Differences between revisions 29 and 44 (spanning 15 versions)

Task Manager

document

Installation

Yuzurihara prepared the virtual environment o4_dqr_proto on k1det1.
DQR tasks are supported by python 3.8
This procedure will require the authentication of git.ligo.org.

To read KAGRA data by using NDS, We need to edit gwpy code (see instruction)

--- /home/controls/bin/miniconda2/envs/o4_dqr_proto_3/lib/python3.8/site-packages/gwpy/io/nds2.py_bak   2021-11-05 19:58:53.198393415 +0900
+++ /home/controls/bin/miniconda2/envs/o4_dqr_proto_3/lib/python3.8/site-packages/gwpy/io/nds2.py       2021-11-05 19:38:57.644634458 +0900
@@ -47,6 +47,7 @@
     ('V1', ('nds.ligo.caltech.edu', 31200)),
     ('C1', ('nds40.ligo.caltech.edu', 31200)),
     ('C0', ('nds40.ligo.caltech.edu', 31200)),
+    ('K1', ('10.68.10.123', 8088)),
 ])

conda create --name o4_dqr_proto python=3.8 -y
conda activate o4_dqr_proto
conda install m2crypto ciecplib python-nds2-client cryptography==3.3.1 -y
pip install git+https://git.ligo.org/o4-dqr/o4-dqr-configuration.git
pip install git+https://github.com/gwdetchar/gwdetchar.git
pip install git+https://git.ligo.org/detchar/dqrtasks.git

install ligo-proxy-init, from the LDG Client package

wget http://www.lsc-group.phys.uwm.edu/lscdatagrid/doc/ldg-client.sh -O /tmp/ldg-client.sh && sudo bash /tmp/ldg-client.sh

X.509 certification
- For the additional permission to access GraceDB, we need to ask LIGO+Virgo colleague.
- There are two options for KAGRA account. Here Yuzurihara used ligo-proxy-init.
```
ligo-proxy-init -i shibbi.pki.itc.u-tokyo.ac.jp albert.einstein   
```
- To test the certification,
```
gracedb ping
```
```
grid-proxy-info
```
- The usage of ecp-get-cert
```
ecp-get-cert --identity-provider KAGRA --username albert.einstein --verbose
```

Generate DAG files

By following instruction, prepare ini files for DQR task manager.
- This procedure will NOT require the access GraceDB, because the public alert will be used.
- For test code, see /users/yuzu/work/DQR/Script/o4-dqr-configuration:k1det1
```
mkdir ~/public_html/event
dqr-create-dag -g S190924h -v -c ex_configs/defaults.ini ex_configs/condor.ini ex_configs/stationarity1.ini ex_configs/omega1.ini
```
The generated DAG files are located in /home/controls/public_html/events/

Because we don't store the O3 data in KAGRA site, we need to run this job with gps option.

mkdir ~/public_html/event
dqr-create-dag -g 1318260000 -v -c ex_configs/defaults.ini ex_configs/condor.ini ex_configs/stationarity1.ini ex_configs/omega1.ini

Submit DAG jobs

For example on k1det1

cd /home/controls/public_html/events/
condor_submit_dag S190924h/data_quality_report_S190924h.dag

Result plots and html are ~/public_html/eventsS190924h/omegascan/plots/ and ~/public_html/eventsS190924h/omegascan/about/index.html

KAGRA/Subgroups/DET/DQR/TaskManager (last edited 2022-12-21 15:09:51 by HirotakaYuzurihara)

-  ⇤ ← Revision 29 as of 2021-10-08 01:02:41 → 
  Size: 2384
  Editor: HirotakaYuzurihara
  Comment:
+   ← Revision 44 as of 2021-11-05 20:02:20 → ⇥
  Size: 3398
  Editor: HirotakaYuzurihara
  Comment:
-Deletions are marked like this.
+Additions are marked like this.
 Line 9:
+ * To read KAGRA data by using NDS, We need to edit `gwpy` code (see [[http://gwwiki.icrr.u-tokyo.ac.jp/JGWwiki/KAGRA/Subgroups/DET/OmegaScan|instruction]])
-Line 11:
+Line 12:
-conda create --name o4_dqr_proto python=3.8
+--- /home/controls/bin/miniconda2/envs/o4_dqr_proto_3/lib/python3.8/site-packages/gwpy/io/nds2.py_bak   2021-11-05 19:58:53.198393415 +0900
+++ /home/controls/bin/miniconda2/envs/o4_dqr_proto_3/lib/python3.8/site-packages/gwpy/io/nds2.py       2021-11-05 19:38:57.644634458 +0900
@@ -47,6 +47,7 @@
     ('V1', ('nds.ligo.caltech.edu', 31200)),
     ('C1', ('nds40.ligo.caltech.edu', 31200)),
     ('C0', ('nds40.ligo.caltech.edu', 31200)),
+    ('K1', ('10.68.10.123', 8088)),
 ])
  }}}


  {{{
conda create --name o4_dqr_proto python=3.8 -y
-Line 13:
+Line 26:
-conda install M2Crypto -y
+conda install m2crypto ciecplib python-nds2-client cryptography==3.3.1 -y
-Line 17:
+Line 30:
-conda install ciecplib
conda install python-nds2-client
-Line 44:
+Line 55:
+  * The usage of {{{ecp-get-cert}}} 
  {{{
ecp-get-cert --identity-provider KAGRA --username albert.einstein --verbose
   }}}
-Line 52:
+Line 69:
-dqr-create-dag S190924h -v -c ex_configs/defaults.ini ex_configs/condor.ini ex_configs/stationarity.ini ex_configs/omega.ini
+dqr-create-dag -g S190924h -v -c ex_configs/defaults.ini ex_configs/condor.ini ex_configs/stationarity1.ini ex_configs/omega1.ini
-Line 56:
+Line 73:
-   {{{
ls /home/controls/public_html/events/
+ * Because we don't store the O3 data in KAGRA site, we need to run this job with gps option.

  {{{
mkdir ~/public_html/event
dqr-create-dag -g 1318260000 -v -c ex_configs/defaults.ini ex_configs/condor.ini ex_configs/stationarity1.ini ex_configs/omega1.ini
-Line 59:
+Line 80: