Differences between revisions 6 and 16 (spanning 10 versions)
⇤ ← Revision 6 as of 2018-06-21 17:39:25 →
Size: 4427
Editor: darkhan.tuyenbayev
Comment:
← Revision 16 as of 2019-07-12 17:40:47 → ⇥
Size: 6834
Editor: darkhan.tuyenbayev
Comment:
Deletions are marked like this. Additions are marked like this.
Line 2: Line 2:
Channels used for low-latency and offline calibration (July 12 - ...)
||<tablewidth="786px">'''Channel Name<<BR>>'''||'''Rate<<BR>>'''||'''Format<<BR>>'''||'''Description<<BR>>'''||
||`K1:CAL-CS_PROC_MICH_RESIDUAL_DBL_DQ` ||16384 ||F64LE ||MICH error ||
||`K1:CAL-CS_PROC_MICH_DELTA_CTRL_TM_DBL_DQ` ||4096 ||F64LE ||MICH TM stage control ||
||`K1:CAL-CS_PROC_MICH_DELTA_CTRL_IM_DBL_DQ` ||4096 ||F64LE ||MICH IM stage control ||
||`K1:CAL-CS_PROC_MICH_DISPLACEMENT_DQ` ||16384 ||F32LE ||MICH displacement (optional) ||
||`K1:CAL-CS_PROC_IMC_RESIDUAL_DQ` ||16384 ||F32LE ||IMC error ||
||`K1:CAL-CS_PROC_IMC_DELTA_CTRL_AOM_DQ` ||4096 ||F32LE ||IMC AOM frequency control ||
||`K1:CAL-CS_PROC_IMC_DELTA_CTRL_TM_DQ` ||4096 ||F32LE ||IMC TM stage control ||
||`K1:CAL-CS_PROC_IMC_FREQUENCY_DQ` ||16384 ||F32LE ||Laser frequency (optional) ||
||`K1:GRD-LSC_LOCK_OK` ||16 ||F32LE ||LSC Guardian OK status ||
||`K1:GRD-LSC_LOCK_STATE_N` ||16 ||F32LE ||LSC lock state ||
||`K1:GRD-IO_OK` ||16 ||F32LE ||IMC Guardian OK status ||
||`K1:GRD-IO_STATE_N` ||16 ||F32LE ||IMC state ||




Where F32LE and F64LE means: floating-point 32-bit and 64-bit numbers in Little-Endian format.

Channels marked "optional" will not be used in the low-latency and offline computations, `K1:CAL-CS_PROC_MICH_DISPLACEMENT_DQ` and `K1:CAL-CS_PROC_IMC_FREQUENCY_DQ` in particular are the channels calibrated with the online system.

----------------

Low-latency h(t) reconstruction server (Aug - Oct/31: DT,SH,TY,NK,...)

 * We already have [[KAGRA/Subgroups/CAL/WG/offline/gstlal|three computing nodes]] to test the offline h(t) reconstruction via frame file but we need to install another for the low-latency h(t) reconstruction via shared memory inside the tunnel

 * (Aug/20-) Connection and software test between a prototype PC and a test bench
  * Prototype PC is just a standard desktop PC and will be prepared by SH

 * (Aug-Sep) Procurement of server (2U height, at least two 10G network card, standard CPU,memory and disk)
  * According to Darkhan’s test with LIGO data and discussion with LIGO people, standard spec should be OK.
  * DT is working to figure out the spec and model.
  * Academia Sinica will make the procurement

 * (Mid.Sep) Connection test between the server and the test bench
  * We also need the collaboration with DMG people for the data transfer

 * (Sep-Oct) Connection test between the server and the KAGRA online system
  * We also need the collaboration with DMG people for the data transfer
Line 4: Line 45:
Line 6: Line 48:
  * Find names and versions of all prerequisite libraries
   * Install prerequisites and gstlal-cal package on a machine at AS
   * Test gstlal-cal pipeline on an AS machine
  * Install prerequisites and gstlal-cal package on a machine at KAGRA
   * Test the pipeline on a KAGRA machine		   * Find names and versions of all prerequisite libraries
  * Install prerequisites and gstlal-cal package on a machine at AS
  * Test gstlal-cal pipeline on an AS machine
  * Install prerequisites and gstlal-cal package on a machine at KAGRA
  * Test the pipeline on a KAGRA machine
Line 12: Line 54:
Progress so far (June 19):
   * Ask LIGO CAL team for the detailed information of gstlal-calibration
     * '''(ongoing) gstlal-calibration maintainers in LIGO are helping to resolve some installation and testing issues.'''
   * Find names and versions of all prerequisite libraries,
     * '''(done, entry from June 7)''' SH made progress on this, see [[KAGRA/Subgroups/CAL/WG/offline/gstlal|gstlal installation]]
   * Install required packages and the gstlal-cal package on a machine at AS
     * '''DT has written scripts for installing lalsuite and gstlal software on a clean CentOS7 distro based on SH's instructions.'''
     Scripts were uploaded to KAGRA-AS group's Gitlab repositories:
       [[https://gitlab.com/kagra-as/docker-lalsuite-cold|containerized "cold" version of lalsuite]]		 Progress so far (June 21):
Line 22: Line 56:
       [[https://gitlab.com/kagra-as/docker-gstlal-cold|containerized "cold" version of gstlal-calibration]]
    * '''CI pipelines that build docker-lalsuite-cold and docker-gstlal-cold containers were set up to run on AS IoP machine.'''
   * Test gstlal-cal pipeline on an AS machine
    * '''As a prerequisite to this task, DT reproduced a portion of LIGO h(t) from raw data with gstlal-calibration-v1.1.8 (w/ a "docker-gstlal-cold" image).

     This test was done on a private server (2-cores on Intel Xeon 2.4GHz, 8GB RAM).'''
    * '''Downloaded a few hours of bKAGRA Phase-1 data on AS machine.'''
    * '''(doing) Will try to reproduce C00 with gstlal-calibration.''' Details in
       [[https://gitlab.com/kagra-as/low-latency-cal/test-gstlal-on-cal-iop|gstlal-calibration testing on AS machine (cal-iop)]]
     * '''(to do) Verify the output of gstlal-calibration.'''
   * Install prerequisites and gstlal-cal package on a machine at KAGRA
    * '''(done, entry from June 7)''' SH made progress on this, see [[KAGRA/Subgroups/CAL/WG/offline/gstlal|gstlal installation]].
   * Test the pipeline on a KAGRA machine [[KAGRA/Subgroups/CAL/WG/offline/tests|gstlal tests]]
     * '''to do'''
   * Documentation
    * '''(doing) Started documenting the work in KAGRA-AS group's Gitlab repositories:'''
       [[https://gitlab.com/kagra-as/docker-lalsuite-cold|containerized "cold" version of lalsuite]]

       [[https://gitlab.com/kagra-as/docker-gstlal-cold|containerized "cold" version of gstlal-calibration]]

       [[https://gitlab.com/kagra-as/low-latency-cal/test-gstlal-on-cal-iop|gstlal-calibration testing on AS machine (cal-iop)]]		  * Ask LIGO CAL team for the detailed information of gstlal-calibration
  * '''(ongoing) gstlal-calibration maintainers in LIGO are helping to resolve some installation and testing issues.'''
 * Find names and versions of all prerequisite libraries,
  * '''(done, entry from June 7)''' SH made progress on this, see [[KAGRA/Subgroups/CAL/WG/offline/gstlal|gstlal installation]]
 * Install required packages and the gstlal-cal package on a machine at AS
  * '''DT has written scripts for installing lalsuite and gstlal software on a clean CentOS7 distro based on SH's instructions.''' Scripts were uploaded to KAGRA-AS group's Gitlab repositories:
   . [[https://gitlab.com/kagra-as/docker-lalsuite-cold|containerized "cold" version of lalsuite]] [[https://gitlab.com/kagra-as/docker-gstlal-cold|containerized "cold" version of gstlal-calibration]]
 * '''CI pipelines that build docker-lalsuite-cold and docker-gstlal-cold containers were set up to run on AS IoP machine.'''
 * Test gstlal-cal pipeline on an AS machine
  * '''As a prerequisite to this task, DT reproduced a portion of LIGO h(t) from raw data with gstlal-calibration-v1.1.8 (w/ a "docker-gstlal-cold" image).''' This test was done on a private server (2-cores on Intel Xeon 2.4GHz, 8GB RAM).
  * '''Downloaded a few hours of bKAGRA Phase-1 data on AS machine.'''
  * '''(doing) Will try to reproduce C00 with gstlal-calibration.''' Details in
   . [[https://gitlab.com/kagra-as/low-latency-cal/test-gstlal-on-cal-iop|gstlal-calibration testing on AS machine (cal-iop)]]
  * '''(to do) Verify the output of gstlal-calibration.'''
 * Install prerequisites and gstlal-cal package on a machine at KAGRA
  * '''(done, entry from June 7)''' SH made progress on this, see [[KAGRA/Subgroups/CAL/WG/offline/gstlal|gstlal installation]].
 * Test the pipeline on a KAGRA machine [[KAGRA/Subgroups/CAL/WG/offline/tests|gstlal tests]]
  * '''to do'''
 * Documentation
  * '''(doing) Started documenting the work in KAGRA-AS group's Gitlab repositories:'''
   . [[https://gitlab.com/kagra-as/docker-lalsuite-cold|containerized "cold" version of lalsuite]] [[https://gitlab.com/kagra-as/docker-gstlal-cold|containerized "cold" version of gstlal-calibration]] [[https://gitlab.com/kagra-as/low-latency-cal/test-gstlal-on-cal-iop|gstlal-calibration testing on AS machine (cal-iop)]]
Line 45: Line 79:
   * Ask LIGO CAL team for the detailed information of gstlal-calibration
     * '''Communication with people working on LIGO gstlal has been initiated.'''
   * Find names and versions of all prerequisite libraries,
    * '''SH made progress on this, see [[KAGRA/Subgroups/CAL/WG/offline/gstlal|gstlal installation]]'''
   * Install prerequisites and gstlal-cal package on a machine at AS
    * '''As a prerequisite to this task, SH worked on installation process for CentOS, DT for Debian.'''
     * '''The goal is to have a better understanding of what is happening under the hood when the gstlal-calibration package is compiled.'''
   * Test gstlal-cal pipeline on an AS machine
    * '''As a prerequisite to this task, DT is currently trying to reproduce LIGO h(t) from raw data.'''
     * '''A sub-goal of this task is to come up with a gstlal code testing procedure.'''
    * '''DT addressed some questions to LIGO gstlal people.'''
   * Install prerequisites and gstlal-cal package on a machine at KAGRA
    * '''SH made progress on this, see [[KAGRA/Subgroups/CAL/WG/offline/gstlal|gstlal installation]]'''
   * Test the pipeline on a KAGRA machine [[KAGRA/Subgroups/CAL/WG/offline/tests|gstlal tests]]
     * '''to do'''
* Ask LIGO CAL team for the detailed information of gstlal-calibration
  * '''Communication with people working on LIGO gstlal has been initiated.'''
 * Find names and versions of all prerequisite libraries,
  * '''SH made progress on this, see [[KAGRA/Subgroups/CAL/WG/offline/gstlal|gstlal installation]]'''
 * Install prerequisites and gstlal-cal package on a machine at AS
  * '''As a prerequisite to this task, SH worked on installation process for CentOS, DT for Debian.'''
  * '''The goal is to have a better understanding of what is happening under the hood when the gstlal-calibration package is compiled.'''
 * Test gstlal-cal pipeline on an AS machine
  * '''As a prerequisite to this task, DT is currently trying to reproduce LIGO h(t) from raw data.'''
  * '''A sub-goal of this task is to come up with a gstlal code testing procedure.'''
  * '''DT addressed some questions to LIGO gstlal people.'''
 * Install prerequisites and gstlal-cal package on a machine at KAGRA
  * '''SH made progress on this, see [[KAGRA/Subgroups/CAL/WG/offline/gstlal|gstlal installation]]'''
 * Test the pipeline on a KAGRA machine [[KAGRA/Subgroups/CAL/WG/offline/tests|gstlal tests]]
  * '''to do'''
Line 62: Line 97:
   * Put more effort on documentation.
* Put more effort on documentation.

Low-latency and offline

Channels used for low-latency and offline calibration (July 12 - ...)

Channel Name

Rate

Format

Description

K1:CAL-CS_PROC_MICH_RESIDUAL_DBL_DQ

16384

F64LE

MICH error

K1:CAL-CS_PROC_MICH_DELTA_CTRL_TM_DBL_DQ

4096

F64LE

MICH TM stage control

K1:CAL-CS_PROC_MICH_DELTA_CTRL_IM_DBL_DQ

4096

F64LE

MICH IM stage control

K1:CAL-CS_PROC_MICH_DISPLACEMENT_DQ

16384

F32LE

MICH displacement (optional)

K1:CAL-CS_PROC_IMC_RESIDUAL_DQ

16384

F32LE

IMC error

K1:CAL-CS_PROC_IMC_DELTA_CTRL_AOM_DQ

4096

F32LE

IMC AOM frequency control

K1:CAL-CS_PROC_IMC_DELTA_CTRL_TM_DQ

4096

F32LE

IMC TM stage control

K1:CAL-CS_PROC_IMC_FREQUENCY_DQ

16384

F32LE

Laser frequency (optional)

K1:GRD-LSC_LOCK_OK

16

F32LE

LSC Guardian OK status

K1:GRD-LSC_LOCK_STATE_N

16

F32LE

LSC lock state

K1:GRD-IO_OK

16

F32LE

IMC Guardian OK status

K1:GRD-IO_STATE_N

16

F32LE

IMC state

Where F32LE and F64LE means: floating-point 32-bit and 64-bit numbers in Little-Endian format.

Channels marked "optional" will not be used in the low-latency and offline computations, K1:CAL-CS_PROC_MICH_DISPLACEMENT_DQ and K1:CAL-CS_PROC_IMC_FREQUENCY_DQ in particular are the channels calibrated with the online system.

Low-latency h(t) reconstruction server (Aug - Oct/31: DT,SH,TY,NK,...)

  • We already have three computing nodes to test the offline h(t) reconstruction via frame file but we need to install another for the low-latency h(t) reconstruction via shared memory inside the tunnel

  • (Aug/20-) Connection and software test between a prototype PC and a test bench
    • Prototype PC is just a standard desktop PC and will be prepared by SH
  • (Aug-Sep) Procurement of server (2U height, at least two 10G network card, standard CPU,memory and disk)
    • According to Darkhan’s test with LIGO data and discussion with LIGO people, standard spec should be OK.
    • DT is working to figure out the spec and model.
    • Academia Sinica will make the procurement
  • (Mid.Sep) Connection test between the server and the test bench
    • We also need the collaboration with DMG people for the data transfer
  • (Sep-Oct) Connection test between the server and the KAGRA online system
    • We also need the collaboration with DMG people for the data transfer

Current tasks (May 26 - July 15):

  • Ask to LIGO CAL team for the detailed information of gstlal-calibration (DT) [status: ongoing | expected finish: N/A]

  • Make gstlal-calibration running on a machine at KAGRA (DT, SH) [status: ongoing | expected finish: Jul 15]

    • Find names and versions of all prerequisite libraries
    • Install prerequisites and gstlal-cal package on a machine at AS
    • Test gstlal-cal pipeline on an AS machine
    • Install prerequisites and gstlal-cal package on a machine at KAGRA
    • Test the pipeline on a KAGRA machine

Progress so far (June 21):

  • Ask LIGO CAL team for the detailed information of gstlal-calibration

    • (ongoing) gstlal-calibration maintainers in LIGO are helping to resolve some installation and testing issues.

  • Find names and versions of all prerequisite libraries,
  • Install required packages and the gstlal-cal package on a machine at AS
  • Test gstlal-cal pipeline on an AS machine

    • As a prerequisite to this task, DT reproduced a portion of LIGO h(t) from raw data with gstlal-calibration-v1.1.8 (w/ a "docker-gstlal-cold" image). This test was done on a private server (2-cores on Intel Xeon 2.4GHz, 8GB RAM).

    • Downloaded a few hours of bKAGRA Phase-1 data on AS machine.

    • (doing) Will try to reproduce C00 with gstlal-calibration. Details in

    • (to do) Verify the output of gstlal-calibration.

  • Install prerequisites and gstlal-cal package on a machine at KAGRA

  • Test the pipeline on a KAGRA machine gstlal tests

    • to do

  • Documentation

Progress so far (June 7):

  • Ask LIGO CAL team for the detailed information of gstlal-calibration

    • Communication with people working on LIGO gstlal has been initiated.

  • Find names and versions of all prerequisite libraries,
  • Install prerequisites and gstlal-cal package on a machine at AS

    • As a prerequisite to this task, SH worked on installation process for CentOS, DT for Debian.

    • The goal is to have a better understanding of what is happening under the hood when the gstlal-calibration package is compiled.

  • Test gstlal-cal pipeline on an AS machine

    • As a prerequisite to this task, DT is currently trying to reproduce LIGO h(t) from raw data.

    • A sub-goal of this task is to come up with a gstlal code testing procedure.

    • DT addressed some questions to LIGO gstlal people.

  • Install prerequisites and gstlal-cal package on a machine at KAGRA

  • Test the pipeline on a KAGRA machine gstlal tests

    • to do

New tasks:

  • Put more effort on documentation.

KAGRA/Subgroups/CAL/Low-latency_and_offline (last edited 2020-07-25 03:17:08 by darkhan.tuyenbayev)