##master-page:CategoryTemplate
##master-date:Unknown-Date
#format wiki
#language en

== Reference ==
 * [[https://hveto.readthedocs.io/en/stable/index.html|online document]]
 * [[https://github.com/gwdetchar/hveto|Source code (GitHub)]]
 * [[https://iopscience.iop.org/article/10.1088/0264-9381/28/23/235005|Paper]]
 * [[https://github.com/gw-detchar/tools|KAGRA scripts (GitHub)]]

= Environment setting =

--(If you need to install from scratch, [[https://computing.docs.ligo.org/conda/]] will be helpful.)--
Error happens with igwn-py38. ligo-summary-3.7 is OK.

== In Kashiwa, m31 ==

 * Environment
{{{
> source /gpfs/sw/conda/etc/profile.d/conda.sh
> conda activate igwn-py39-20220707
}}}

(Warning!: Hveto does not work properly for the latest one, `igwn-py39-202201102`. 
Below, `igwn-py39-202200707` was used.)


 * Create segment lists
  * Segment lists (xml and txt files) are automatically generated and stored under `/home/detchar/Segments/`. If you cannot find the date you want to use, create the segment lists.

  * Scripts are here `/home/detchar/git/kagra-detchar/tools/Segments/Script/` or [[https://github.com/gw-detchar/tools/tree/master/Segments/Script|Scripts (GitHub)]].

  * Run, for example, 
  {{{
  > python make24hourSegment.py -c Kashiwa -d 2022-11-09 (-o /output/directory)
  }}}
  * Choose the cluster by `-c` option (necessary). Only `Kashiwa` or `Kamioka` is available.

  * Choose the date by `-d` option (necessary). 

  * Default output directory is `/home/detchar/Segments/`. You can specify arbitrary directory by adding `-o` option.

  * See `K1-GRD_LOCKED` for the locked segments.


 * Run Omicron
  * Scripts are here `/home/detchar/git/kagra-detchar/tools/Omicron/Script/`or [[https://github.com/gw-detchar/tools/tree/master/Omicron/Script|GitHub]].

  * Cache files are here `/home/detchar/cache/Cache_GPS/`. Use the Virgo format, `ffl`.

  * Run, for example, 
  {{{
  > manual_segment_kashiwa.sh omicron_parameters.txt omicron_segments.txt > log.txt 2>&1
  }}} 
  See `log.txt`.

  * By using the `GWpy` function or the useful python script, `OpenEventTable.py`, you can see the summary of triggers. Run `> python OpenEventTable.py` after edited the python script.

  `peak_time peak_time_ns start_time start_time_ns duration peak_frequency central_freq bandwidth channel amplitude snr`

  * Change directory structure to run Hveto by using `mvtoK1.sh`.

  * The output trigger files should be stored in `/home/detchar/triggers/K1` by gwdetchar account.

 * Run Hveto
  * Scripts are here `/home/detchar/git/kagra-detchar/tools/Hveto/Script/`or [[https://github.com/gw-detchar/tools/tree/master/Hveto/Script|GitHub]].

  * Use `hveto_O3GK_arg.sh`. Change the segment list path and ini files for the O4 run.

    * Need a segment list file in the xml format. Specify the path of the segment file within  `hveto_O3GK_arg.sh`.

    * Configuration file (ini file) is needed. Details are described below, the ini file section on this page. Specify the path of the ini file within  `hveto_O3GK_arg.sh`.
    
    * To run the script, add the date as an option, for example
 
   {{{
    > hveto_O3GK_arg.sh 2022-11-09
    }}} 
  * Hveto reads the Omicron triggers file from `/home/detchar/triggers/K1` as a default. Failed to use cache file options for m31 (see [[https://klog.icrr.u-tokyo.ac.jp/osl/?r=22870|klog22870]]).


== In Kamioka, k1det0 ==


--(> source /cvmfs/oasis.opensciencegrid.org/ligo/sw/conda/etc/profile.d/conda.sh)--
--(> conda activate igwn-py38)--

{{{
> conda activate ligo-summary-3.7
}}}

= Manual script =

Script for kashiwa server

[[https://github.com/gw-detchar/tools/blob/master/Hveto/Script/hveto_O3GK_arg.sh]]

== Usage ==
{{{
> ./hveto_O3GK_arg.sh 2020-04-10 > log.dat 2>&1
}}}

== Important parameters ==

 * SEGMENTFILE : Now it is setting for O3GK. Maybe you need to modify for O4 or later, for example 
{{{
SEGMENTFILE=/home/detchar/Segments/K1-SCIENCE_MODE/2022/K1-SCIENCE_MODE_UTC_${DIRNAME_DATE}.xml
}}}

 * INIFILE : For O3GK, [[https://github.com/gw-detchar/tools/blob/master/Hveto/etc/O3GKC20.ini|O3GKC20.ini]] is used. Detail will be written later again.

 * OUTPUTDIR : Please set it as you like. When it is under /mnt/hveto/ or /home/controls/public_html/hveto/ (linked to /mnt/hveto/), the result is available on web browser, https://gwdet.icrr.u-tokyo.ac.jp/~controls/hveto/ .

 * NPROCESS : Originally it was 10 but modified to 1. It is because sometimes multiple process cause error. 

= ini file =

 * The latest file dedicated for O3GK is [[https://github.com/gw-detchar/tools/blob/master/Hveto/etc/O3GKC20.ini|O3GKC20.ini]]. This is based on the ini file used in LIGO, [[https://git.ligo.org/detchar/ligo-monitors/-/blob/master/configurations/hveto/h1l1-hveto-daily-post-o3.ini|h1l1-hveto-daily-post-o3.ini]] with some modification for KAGRA.

== Important parameters ==
 * For the detail of ini file contents, see [[https://hveto.readthedocs.io/en/stable/api/hveto.config.html|Configuration files for Hveto]]
 * [segment]
   * analysis-flag : You may need to modify it for the online process. The name of the segment list has to be set.
   * padding : Please coordinate with the analysis group whether it covers the necessary range.
 * [primary]
  * channel : K1:DAC-STRAIN_C20 is offline data. For online, {{{channel = %(IFO)s:CAL-CS_PROC_DARM_DISPLACEMENT_DQ}}} or {{{channel = %(IFO)s:CAL-CS_PROC_C00_STRAIN_DBL_DQ}}} will do.
  * snr-threshold : It maybe better to optimize.
  * frequency-range : Please coordinate with the analysis group whether it covers the necessary range.
 * [auxiliary]
  * frequency-range : Please coordinate with the analysis group whether it covers the necessary range.
  * channels : Now the channel list is exactly specified here. Before O4, it should be reconsidered. For example, ASC channels may need to be added.
 * [safety]
  * unsafe-channels : Actually this is not necessary if [auxiliary] channels is set, just written to clarify that they must not be used in hveto. Here is the reference of the discussion for O3GK [[https://gwdoc.icrr.u-tokyo.ac.jp/cgi-bin/private/DocDB/ShowDocument?docid=11874|slides]].

= Note for developer =
 * Maybe it is buggy in the counting of coincidence events ?
 * Not automated in Kashiwa server.
 * Please check the link of /home/detchar/triggers/K1. The data under this link is used. Usually, it should be linked to /mnt/triggers/K1/, but for O3GK /mnt/triggers/K1_20200415/K1_20200415/K1/ was used. 
 * It will be good if parameter optimization is done with ER data or something.