Task Manager

document

Installation

Yuzurihara prepared the virtual environment o4_dqr_proto_3 on k1det1.
The current DQR tasks are running with python 3.10.11
This procedure will require the authentication of git.ligo.org.

To read KAGRA data by using NDS, We need to edit gwpy code (see instruction)

--- /home/controls/bin/miniconda2/envs/o4_dqr_proto_3/lib/python3.8/site-packages/gwpy/io/nds2.py_bak   2021-11-05 19:58:53.198393415 +0900
+++ /home/controls/bin/miniconda2/envs/o4_dqr_proto_3/lib/python3.8/site-packages/gwpy/io/nds2.py       2021-11-05 19:38:57.644634458 +0900
@@ -47,6 +47,7 @@
     ('V1', ('nds.ligo.caltech.edu', 31200)),
     ('C1', ('nds40.ligo.caltech.edu', 31200)),
     ('C0', ('nds40.ligo.caltech.edu', 31200)),
+    ('K1', ('10.68.10.123', 8088)),
 ])

conda create --name o4_dqr_proto_3 python=3.8 -y
conda activate o4_dqr_prot_3
conda install gwpy
conda install m2crypto ciecplib python-nds2-client cryptography==3.3.1 -y
conda install igwn-alert -y
conda install tree markuppy -y
conda activate o4_dqr_proto_3

pip install git+https://git.ligo.org/o4-dqr/o4-dqr-configuration.git
pip install git+https://github.com/gwdetchar/gwdetchar.git
pip install git+https://git.ligo.org/detchar/dqrtasks.git

python -m pip install gwpy[astro]

install ligo-proxy-init, from the LDG Client package

wget http://www.lsc-group.phys.uwm.edu/lscdatagrid/doc/ldg-client.sh -O /tmp/ldg-client.sh && sudo bash /tmp/ldg-client.sh

X.509 certification
- For the additional permission to access GraceDB, we need to ask LIGO+Virgo colleague.
- There are two options for KAGRA account. Here Yuzurihara used ligo-proxy-init.
```
ligo-proxy-init -i shibbi.pki.itc.u-tokyo.ac.jp albert.einstein   
```
- To test the certification,
```
gracedb ping
```
```
grid-proxy-info
```
- The usage of ecp-get-cert
```
ecp-get-cert --identity-provider KAGRA --username albert.einstein --verbose
```

Added RETRY option for DAG submission

Edit the code (o4-dqr-configuration/dqr_config/DQRdag.py)

 #self.retry = 0
 # (2022/07/21 yuzu added)
 self.retry = 5

Generate DAG files

By following instruction, prepare ini files for DQR task manager.
- This procedure will NOT require the access GraceDB, because the public alert will be used.
- For test code, see /users/yuzu/work/DQR/Script/o4-dqr-configuration:k1det1
```
mkdir ~/public_html/event
dqr-create-dag -g S190924h -v -c ex_configs/defaults.ini ex_configs/condor.ini ex_configs/stationarity1.ini ex_configs/omega1.ini
```
The generated DAG files are located in /home/controls/public_html/events/

Because we don't store the O3 data in KAGRA site, we need to run this job with gps option.

mkdir ~/public_html/event
dqr-create-dag -g 1318260000 -v -c ex_configs/defaults.ini ex_configs/condor.ini ex_configs/stationarity1.ini ex_configs/omega1.ini

Submit DAG jobs

For example on k1det1

cd /home/controls/public_html/events/
condor_submit_dag S190924h/data_quality_report_S190924h.dag

Result plots and html are ~/public_html/eventsS190924h/omegascan/plots/ and ~/public_html/eventsS190924h/omegascan/about/index.html

KAGRA/Subgroups/DET/DQR/TaskManager (last edited 2025-07-22 13:48:30 by HirotakaYuzurihara)