Evaluating data analysis challenges in the 3G detector era and discussing the solutions
Python module installation
A module is a list of functions and classes you would like to use. In order to use your owm module in python, you can just write it in a file with an extension ".py", and call this module by a command:
Then all functions or classes from the loaded ".py" file will become available in your code. The ".py" file has to be in the same directory you are working in.
In addition, there are tons of different nice modules for python available here:
pip install --user module_name
Here module_name is a name of module from PyPi. An option "--user" is useful on computer clusters (CIT, OzStar), where user does not have rights for a global installation on the machine.
A collection of modules for specific purposes can be installed as part of the conda environment. First, conda has to be installed. Next, one has to find a conda environment that has packages for one's specific needs. For example, for gravitational wave analyses, an example is. The conda environment manifests as a .yml file, which needs to be installed, after that the environment can be activated. The user guide is available .
From git or external sources
Modules from remote git repositories are usually supplied with "setup.py", which has to be ran for modules to be installed:
python setup.py install.
For developers of the python module
Say, we downloaded (cloned) a git repository with a python module. After a default installation modules are located in ~/.local/lib/python3.6/site-packages/.... If we change the code in a git repository, we would need to reinstall the module again, so that our changes take effect.
A better way would be then to install a module this way:
python setup.py develop. This way changes to the git-repository are automatically reflected in your code, with no need to re-install the module. To uninstall a module in developer mode:
python setup.py develop --uninstall.
A useful module for debbugging in python is "ipdb". After placing
import ipdb in the beginning of the code, we can pause the code execution at any line with
ipdb.set_trace(). Then we can type "c" to continue execution or "q" to quit. We can try out any other command when the code execution is paused this way.
Another way is to run the python script this way:
python -m pdb script_name.py. It is particularly useful when it is not clear at what line an error has occurred. In the beginning "c" should be typed to get the code running.
Also, run your script as
python -m ipdb -c continue SCRIPT.py and it will automatically drop you to an ipython terminal when an error is raised.
Using MATLAB in Python
It is possible to run MATLAB commands and functions in Python: seefor more information. To use MATLAB's functionality with Python one needs to install "MATLAB engine" module for Python:
- In MATLAB do:
- On your own computer do:
system('python setup.py install')
On the cluster do
pwdin MATLAB to see the directory, and in a separate tab enter this directory. Then:
python setup.py build --build-base="/temporary_build_directory" install --user
For more information on MATLAB see: MATLAB code compilation on LIGO cluster.
[!] Warning [!]
When I loaded MATLAB libraries in Python this affected other Python code I used ("libstempo" package in my case).
Copying arrays in python is not the same, as assigning an old array variable to a new array. It worked for me in ipython debugger (ipdb), but did not work in the script: after new arrays were created, assignment of values to their elements did not work. The reason is because python interprets assignment of an array, as directing the new array to the same physical memory, as is occupied by an old array. Instead, we need to use
new_array = old_array.copy().
A precursor to the detection of nanohertz gravitational waves or unmodeled pulsar noise?