Notes on working with LIGO data and computing clusters

Before using this command please follow Eric Thrane's instructions to set up a gsissh-connection with LIGO cluster using LDG client.

Connection to LIGO data centers

Working on LIGO cluster - CIT/pcdev

To copy files to LIGO CIT use this command:

gsiscp file_path/file_name ldas-pcdev2.ligo.caltech.edu:/home/boris.goncharov/file_name

Add -r to copy folders.

Accessing LIGO data

LIGO strain data from Hanford (H1) and Livingston (L1) detectors is stored in .GWF files on the CIT computer cluster. In MATLAB software used by LIGO Stochastic group, i.e. STAMP or Stochastic.m, we need the following:

  1. A job file: list of time segments of data.
    Typical job file has 4 columns: some arbitrary index number, GPS start time (sec), GPS end time (sec), duration (sec)
    Job files can be generated by this script: /home/boris.goncharov/jobgen/make-job-file.sh
  2. Frame files: list of files with paths to .GWF files for our jobs
    Cache files can be generated by this perl script: /home/boris.goncharov/jobgen/cachefiles.pl

GWF files

Files with .gwf format were designed for LIGO-Virgo gravitational-wave frame data. However, there are not many ways to view the list of channels, which are often required to load the data. Here is a useful command to view the list of channels:

FrChannels file_name

Note, this command runs by default on LIGO CIT, but not on OzStar or Virgo CNAF.

HDF5 files

HDF5-type files are very common for storing not just LIGO data, but pretty much any kind of data. Here is a useful command to view what data is contained within an hdf5 file:

h5ls file_name

Note, this command runs by default on LIGO CIT and Virgo CNAF, but not on OzStar.

LIGO PSD and ASD

After taking a Fourier transform of LIGO strain data we get power-spectral density (PSD).
It has units [strain^2 / Hz].

After taking a square root of PSD we get amplitude spectral density (ASD).
It has units [strain / sqrt(Hz)]

Recent Work