Differences between revisions 15 and 33 (spanning 18 versions)
Revision 15 as of 2021-03-11 15:37:10
Size: 3356
Comment:
Revision 33 as of 2023-02-14 14:28:05
Size: 6690
Comment:
Deletions are marked like this. Additions are marked like this.
Line 6: Line 6:
= Reference =
[[https://github.com/gwdetchar/hveto|Source code (GitHub)]]

[[https://iopscience.iop.org/article/10.1088/0264-9381/28/23/235005|Paper]]

[[https://github.com/gw-detchar/tools|KAGRA scripts (GitHub)]]
== Reference ==
 * [[https://hveto.readthedocs.io/en/stable/index.html|online document]]
 *
[[https://github.com/gwdetchar/hveto|Source code (GitHub)]]
 * [[https://iopscience.iop.org/article/10.1088/0264-9381/28/23/235005|Paper]]
 * [[https://github.com/gw-detchar/tools|KAGRA scripts (GitHub)]]
Line 20: Line 19:
 * Environment
Line 21: Line 21:
--(> source /gpfs/ligo/sw/conda/etc/profile.d/conda.sh
> conda activate igwn-py38)--
> source /gpfs/sw/conda/etc/profile.d/conda.sh
> conda activate igwn-py39-20220707
Line 24: Line 24:
Not yet succeeded.
(Warning!: Hveto does not work properly for the latest one, `igwn-py39-202201102`.
Below, `igwn-py39-202200707` was used.)


 * Create segment lists
  * Segment lists (xml and txt files) are automatically generated and stored under `/home/detchar/Segments/`. If you cannot find the date you want to use, create the segment lists.

  * Scripts are here `/home/detchar/git/kagra-detchar/tools/Segments/Script/` or [[https://github.com/gw-detchar/tools/tree/master/Segments/Script|Scripts (GitHub)]].

  * Run, for example,
  {{{
  > python make24hourSegment.py -c Kashiwa -d 2022-11-09 (-o /output/directory)
  }}}
  * Choose the cluster by `-c` option (necessary). Only `Kashiwa` or `Kamioka` is available.

  * Choose the date by `-d` option (necessary).

  * Default output directory is `/home/detchar/Segments/`. You can specify arbitrary directory by adding `-o` option.

  * See `K1-GRD_LOCKED` for the locked segments.


 * Run Omicron
  * Scripts are here `/home/detchar/git/kagra-detchar/tools/Omicron/Script/`or [[https://github.com/gw-detchar/tools/tree/master/Omicron/Script|GitHub]].

  * Cache files are here `/home/detchar/cache/Cache_GPS/`. Use the Virgo format, `ffl`.

  * Run, for example,
  {{{
  > manual_segment_kashiwa.sh omicron_parameters.txt omicron_segments.txt > log.txt 2>&1
  }}}
  See `log.txt`.

  * By using the `GWpy` function or the useful python script, `OpenEventTable.py`, you can see the summary of triggers. Run `> python OpenEventTable.py` after edited the python script.

  `peak_time peak_time_ns start_time start_time_ns duration peak_frequency central_freq bandwidth channel amplitude snr`

  * Change directory structure to run Hveto by using `mvtoK1.sh`.

  * The output trigger files should be stored in `/home/detchar/triggers/K1` by gwdetchar account.

 * Run Hveto
  * Scripts are here `/home/detchar/git/kagra-detchar/tools/Hveto/Script/`or [[https://github.com/gw-detchar/tools/tree/master/Hveto/Script|GitHub]].

  * Use `hveto_O3GK_arg.sh`. Change the segment list path and ini files for the O4 run.

    * Need a segment list file in the xml format. Specify the path of the segment file within `hveto_O3GK_arg.sh`.

    * Configuration file (ini file) is needed. Details are described below, the ini file section on this page. Specify the path of the ini file within `hveto_O3GK_arg.sh`.
    
    * To run the script, add the date as an option, for example
 
   {{{
    > hveto_O3GK_arg.sh 2022-11-09
    }}}
  * Hveto reads the Omicron triggers file from `/home/detchar/triggers/K1` as a default. Failed to use cache file options for m31 (see [[https://klog.icrr.u-tokyo.ac.jp/osl/?r=22870|klog22870]]).
Line 28: Line 85:
--({{{
> source /cvmfs/oasis.opensciencegrid.org/ligo/sw/conda/etc/profile.d/conda.sh
> conda activate igwn-py38
}}}
)--

--(> source /cvmfs/oasis.opensciencegrid.org/ligo/sw/conda/etc/profile.d/conda.sh)--
--(> conda activate igwn-py38)--
Line 56: Line 113:
 * OUTPUTDIR : Please set it as you like.  * OUTPUTDIR : Please set it as you like. When it is under /mnt/hveto/ or /home/controls/public_html/hveto/ (linked to /mnt/hveto/), the result is available on web browser, https://gwdet.icrr.u-tokyo.ac.jp/~controls/hveto/ .
Line 62: Line 119:
The latest file dedicated for O3GK is [[https://github.com/gw-detchar/tools/blob/master/Hveto/etc/O3GKC20.ini|O3GKC20.ini]]. This is based on the ini file used in LIGO, [[https://git.ligo.org/detchar/ligo-monitors/-/blob/master/configurations/hveto/h1l1-hveto-daily-post-o3.ini|h1l1-hveto-daily-post-o3.ini]] with some modification for KAGRA.  * The latest file dedicated for O3GK is [[https://github.com/gw-detchar/tools/blob/master/Hveto/etc/O3GKC20.ini|O3GKC20.ini]]. This is based on the ini file used in LIGO, [[https://git.ligo.org/detchar/ligo-monitors/-/blob/master/configurations/hveto/h1l1-hveto-daily-post-o3.ini|h1l1-hveto-daily-post-o3.ini]] with some modification for KAGRA.
Line 65: Line 122:
 * For the detail of ini file contents, see [[https://hveto.readthedocs.io/en/stable/api/hveto.config.html|Configuration files for Hveto]]
Line 78: Line 136:
= Note for developper = = Note for developer =
Line 81: Line 139:
 * Please check the link of /home/detchar/triggers/K1. The data under this link is used. Usually, it should be linked to /mnt/triggers/K1/, but for O3GK /mnt/triggers/K1_20200415/K1_20200415/K1/ was used.

Reference

Environment setting

If you need to install from scratch, https://computing.docs.ligo.org/conda/ will be helpful. Error happens with igwn-py38. ligo-summary-3.7 is OK.

In Kashiwa, m31

  • Environment

> source /gpfs/sw/conda/etc/profile.d/conda.sh
> conda activate igwn-py39-20220707

(Warning!: Hveto does not work properly for the latest one, igwn-py39-202201102. Below, igwn-py39-202200707 was used.)

  • Create segment lists
    • Segment lists (xml and txt files) are automatically generated and stored under /home/detchar/Segments/. If you cannot find the date you want to use, create the segment lists.

    • Scripts are here /home/detchar/git/kagra-detchar/tools/Segments/Script/ or Scripts (GitHub).

    • Run, for example,
        > python make24hourSegment.py -c Kashiwa -d 2022-11-09 (-o /output/directory)
    • Choose the cluster by -c option (necessary). Only Kashiwa or Kamioka is available.

    • Choose the date by -d option (necessary).

    • Default output directory is /home/detchar/Segments/. You can specify arbitrary directory by adding -o option.

    • See K1-GRD_LOCKED for the locked segments.

  • Run Omicron
    • Scripts are here /home/detchar/git/kagra-detchar/tools/Omicron/Script/or GitHub.

    • Cache files are here /home/detchar/cache/Cache_GPS/. Use the Virgo format, ffl.

    • Run, for example,
        > manual_segment_kashiwa.sh omicron_parameters.txt omicron_segments.txt > log.txt 2>&1

      See log.txt.

    • By using the GWpy function or the useful python script, OpenEventTable.py, you can see the summary of triggers. Run > python OpenEventTable.py after edited the python script.

      peak_time peak_time_ns start_time start_time_ns duration peak_frequency central_freq bandwidth channel amplitude snr

    • Change directory structure to run Hveto by using mvtoK1.sh.

    • The output trigger files should be stored in /home/detchar/triggers/K1 by gwdetchar account.

  • Run Hveto
    • Scripts are here /home/detchar/git/kagra-detchar/tools/Hveto/Script/or GitHub.

    • Use hveto_O3GK_arg.sh. Change the segment list path and ini files for the O4 run.

      • Need a segment list file in the xml format. Specify the path of the segment file within hveto_O3GK_arg.sh.

      • Configuration file (ini file) is needed. Details are described below, the ini file section on this page. Specify the path of the ini file within hveto_O3GK_arg.sh.

      • To run the script, add the date as an option, for example
      •     > hveto_O3GK_arg.sh 2022-11-09
    • Hveto reads the Omicron triggers file from /home/detchar/triggers/K1 as a default. Failed to use cache file options for m31 (see klog22870).

In Kamioka, k1det0

> source /cvmfs/oasis.opensciencegrid.org/ligo/sw/conda/etc/profile.d/conda.sh > conda activate igwn-py38

> conda activate ligo-summary-3.7

Manual script

Script for kashiwa server

https://github.com/gw-detchar/tools/blob/master/Hveto/Script/hveto_O3GK_arg.sh

Usage

> ./hveto_O3GK_arg.sh 2020-04-10 > log.dat 2>&1

Important parameters

  • SEGMENTFILE : Now it is setting for O3GK. Maybe you need to modify for O4 or later, for example

SEGMENTFILE=/home/detchar/Segments/K1-SCIENCE_MODE/2022/K1-SCIENCE_MODE_UTC_${DIRNAME_DATE}.xml
  • INIFILE : For O3GK, O3GKC20.ini is used. Detail will be written later again.

  • OUTPUTDIR : Please set it as you like. When it is under /mnt/hveto/ or /home/controls/public_html/hveto/ (linked to /mnt/hveto/), the result is available on web browser, https://gwdet.icrr.u-tokyo.ac.jp/~controls/hveto/ .

  • NPROCESS : Originally it was 10 but modified to 1. It is because sometimes multiple process cause error.

ini file

Important parameters

  • For the detail of ini file contents, see Configuration files for Hveto

  • [segment]
    • analysis-flag : You may need to modify it for the online process. The name of the segment list has to be set.
    • padding : Please coordinate with the analysis group whether it covers the necessary range.
  • [primary]
    • channel : K1:DAC-STRAIN_C20 is offline data. For online, channel = %(IFO)s:CAL-CS_PROC_DARM_DISPLACEMENT_DQ or channel = %(IFO)s:CAL-CS_PROC_C00_STRAIN_DBL_DQ will do.

    • snr-threshold : It maybe better to optimize.
    • frequency-range : Please coordinate with the analysis group whether it covers the necessary range.
  • [auxiliary]
    • frequency-range : Please coordinate with the analysis group whether it covers the necessary range.
    • channels : Now the channel list is exactly specified here. Before O4, it should be reconsidered. For example, ASC channels may need to be added.
  • [safety]
    • unsafe-channels : Actually this is not necessary if [auxiliary] channels is set, just written to clarify that they must not be used in hveto. Here is the reference of the discussion for O3GK slides.

Note for developer

  • Maybe it is buggy in the counting of coincidence events ?
  • Not automated in Kashiwa server.
  • Please check the link of /home/detchar/triggers/K1. The data under this link is used. Usually, it should be linked to /mnt/triggers/K1/, but for O3GK /mnt/triggers/K1_20200415/K1_20200415/K1/ was used.
  • It will be good if parameter optimization is done with ER data or something.

KAGRA/Subgroups/DET/Hveto (last edited 2023-02-14 14:28:05 by nami.uchikata)