Differences between revisions 4 and 35 (spanning 31 versions)
⇤ ← Revision 4 as of 2018-06-07 18:04:54 →
Size: 1946
Editor: darkhan.tuyenbayev
Comment:
← Revision 35 as of 2020-07-25 03:17:08 → ⇥
Size: 10505
Editor: darkhan.tuyenbayev
Comment:
Deletions are marked like this. Additions are marked like this.
Line 1: Line 1:
= Prepare h(t) reconstruction review =
LIGO time-domain calibration review wiki: https://wiki.ligo.org/Calibration/TDCalibReview

= Calibration State Vector (K1:DCS-CALIB_STATE_VECTOR_C10) =
(Under preparation. It's planned to be used in O3b)
||<tablewidth="786px">'''bit<<BR>>''' ||'''Identifier<<BR>>''' ||'''Description<<BR>>''' ||
||0 ||`HOFT_OK` ||$h(t)$ was successfully computed (logical AND of bits ...) ||
||1 ||`OBS_INTENT` ||The observation intent'' is set by the operator '' ||
||2 ||`OBS_READY_OR_OBSERVING` ||Interferometer is in observation ready mode: the LSC_LOCK+ guardian is in the nominal (observation) state. ||
||3 ||`NO_STOCHASTIC_HW_INJ` ||No stochastic hardware injections present ||
||4 ||`NO_CBC_HW_INJ` ||No compact binary coalescence hardware injections present ||
||5 ||`NO_BURST_HW_INJ` ||No burst hardware injections present ||
||6 ||`NO_DETCHAR_HW_INJ` ||No detector characterization hardware injections present ||
||7 ||`NO_CW_HW_INJ` ||No continuous wave characterization hardware injections present ||
||8 ||`TM_LINE_COH_OK` ||TM line coherence OK ||
||9 ||`IM_LINE_COH_OK` ||IM line coherence OK ||
||10 ||`MN_LINE_COH_OK` ||MN line coherence OK ||
||11 ||`PCAL1_LINE_COH_OK` ||PCAL1 line coherence OK ||
||12 ||`PCAL2_LINE_COH_OK` ||PCAL2 line coherence OK ||
||13 ||`PCAL3_LINE_COH_OK` ||PCAL3 line coherence OK ||
||14 ||`PCAL4_LINE_COH_OK` ||PCAL4 line coherence OK ||
||15 ||`PCAL5_LINE_COH_OK` ||PCAL5 line coherence OK ||
||16 ||`SMOOTH_KAPPA_TM_OK` ||TDCF smoothed $\kappa_\mathrm{TM}$ values are in the expected range ||
||17 ||`SMOOTH_KAPPA_IM_OK` ||TDCF smoothed $\kappa_\mathrm{IM}$ values are in the expected range ||
||18 ||`SMOOTH_KAPPA_MN_OK` ||TDCF smoothed $\kappa_\mathrm{MN}$ values are in the expected range ||
||19 ||`SMOOTH_KAPPA_C_OK` ||TDCF smoothed $\kappa_\mathrm{C}$ value is in the expected range ||
||20 ||`SMOOTH_F_CC_OK` ||TDCF smoothed $f_\mathrm{cc}$ value is in the expected range ||
||21 ||`SMOOTH_F_S_OK` ||(reserved) TDCF smoothed $f_\mathrm{S}$ value is in the expected range ||
||22 ||`SMOOTH_Q_S_OK` ||(reserved) TDCF smoothed $Q_\mathrm{S}$ value is in the expected range ||
||23 ||`TDCFS_APPLIED` ||Time-dependent calibration factors are being applied ||
||24 ||`LINE_SUBTRACTION_APPLIED` ||(reserved) Line subtraction are being applied ||
||25 ||`NO_GAP` ||(reserved) Input data has no gaps, underflow, or overflow values ||
||26 ||(not in use) ||(not in use) ||
||27 ||(not in use) ||(not in use) ||
||28 ||(not in use) ||(not in use) ||
||29 ||(not in use) ||(not in use) ||
||30 ||(not in use) ||(not in use) ||
||31 ||(not in use) ||(not in use) ||




Some channels were copied from DET channels

`K1:DET-DQ_STATE_VECTOR` --> CAL state vector

  # bit 2 : SDF flag (= `K1:GRD-SYS_SDF_OK`) --> # bit 2

  # bit 7 : NO Stoch injection --> # bit 3

  # bit 8 : NO CBC injection --> # bit 4

  # bit 9 : NO Burst injection --> # bit 5

  # bit 10 : NO DET injection --> # bit 6

  # bit 11 : NO CW injection --> # bit 7


`K1:DET-OBS_INTENT` --> CAL state vector: # bit 1


Question: Would we better to change the channel name from `K1:DCS-CALIB_STATE_VECTOR`'''_C10''' to `K1:DCS-CALIB_STATE_VECTOR` ?



At the first half of observing run with PRFPMI for O3 (Late Feb. 2020. - Mid. Mar. 2020),

  `HOFT_OK` bit was computed with the logical AND of bits : 1-7. (All of these bits were just copied from DET channels)


At the latter half of observation (Late Mar. 2020 -),

  `HOFT_OK` bit will be planned to be computed with the logical AND of bits : 1-7, 16-20, 23, 25 (or 1-20, 23, 25).
Line 2: Line 79:
Channels used for low-latency and offline calibration (July 12 - ...)
||<tablewidth="786px">'''Channel Name<<BR>>''' ||'''Rate<<BR>>''' ||'''Format<<BR>>''' ||'''Description<<BR>>''' ||
||`K1:CAL-CS_PROC_MICH_RESIDUAL_DBL_DQ` ||16384 ||F64LE ||MICH error ||
||`K1:CAL-CS_PROC_MICH_DELTA_CTRL_TM_DBL_DQ` ||4096 ||F64LE ||MICH TM stage control ||
||`K1:CAL-CS_PROC_MICH_DELTA_CTRL_IM_DBL_DQ` ||4096 ||F64LE ||MICH IM stage control ||
||`K1:CAL-CS_PROC_MICH_DISPLACEMENT_DQ` ||16384 ||F32LE ||MICH displacement (optional) ||
||`K1:CAL-CS_PROC_IMC_RESIDUAL_DQ` ||16384 ||F32LE ||IMC error ||
||`K1:CAL-CS_PROC_IMC_DELTA_CTRL_AOM_DQ` ||4096 ||F32LE ||IMC AOM frequency control ||
||`K1:CAL-CS_PROC_IMC_DELTA_CTRL_TM_DQ` ||4096 ||F32LE ||IMC TM stage control ||
||`K1:CAL-CS_PROC_IMC_FREQUENCY_DQ` ||16384 ||F32LE ||Laser frequency (optional) ||
||`K1:GRD-LSC_LOCK_OK` ||16 ||F32LE ||LSC Guardian OK status ||
||`K1:GRD-LSC_LOCK_STATE_N` ||16 ||F32LE ||LSC lock state ||
||`K1:GRD-IO_OK` ||16 ||F32LE ||IMC Guardian OK status ||
||`K1:GRD-IO_STATE_N` ||16 ||F32LE ||IMC state ||




Where F32LE and F64LE means: floating-point 32-bit and 64-bit numbers in Little-Endian format.

Channels marked "optional" will not be used in the low-latency and offline computations, `K1:CAL-CS_PROC_MICH_DISPLACEMENT_DQ` and `K1:CAL-CS_PROC_IMC_FREQUENCY_DQ` in particular are the channels calibrated with the online system.

----------
Low-latency h(t) reconstruction server (Aug - Oct/31: DT,SH,TY,NK,...)

 * We already have [[KAGRA/Subgroups/CAL/WG/offline/gstlal|three computing nodes]] to test the offline h(t) reconstruction via frame file but we need to install another for the low-latency h(t) reconstruction via shared memory inside the tunnel

 * (Aug/20-) Connection and software test between a prototype PC and a test bench
  * Prototype PC is just a standard desktop PC and will be prepared by SH

 * (Aug-Sep) Procurement of server (2U height, at least two 10G network card, standard CPU,memory and disk)
  * According to Darkhan’s test with LIGO data and discussion with LIGO people, standard spec should be OK.
  * DT is working to figure out the spec and model.
  * Academia Sinica will make the procurement

 * (Mid.Sep) Connection test between the server and the test bench
  * We also need the collaboration with DMG people for the data transfer

 * (Sep-Oct) Connection test between the server and the KAGRA online system
  * We also need the collaboration with DMG people for the data transfer
Line 4: Line 121:
Line 6: Line 124:
   * Find names and versions of all prerequisite libraries
   * Install prerequisites and gstlal-cal package on a machine at AS
   * Test gstlal-cal pipeline on an AS machine
   * Install prerequisites and gstlal-cal package on a machine at KAGRA
   * Test the pipeline on a KAGRA machine		   * Find names and versions of all prerequisite libraries
  * Install prerequisites and gstlal-cal package on a machine at AS
  * Test gstlal-cal pipeline on an AS machine
  * Install prerequisites and gstlal-cal package on a machine at KAGRA
  * Test the pipeline on a KAGRA machine

Progress so far (June 21):

 * Ask LIGO CAL team for the detailed information of gstlal-calibration
  * '''(ongoing) gstlal-calibration maintainers in LIGO are helping to resolve some installation and testing issues.'''
 * Find names and versions of all prerequisite libraries,
  * '''(done, entry from June 7)''' SH made progress on this, see [[KAGRA/Subgroups/CAL/WG/offline/gstlal|gstlal installation]]
 * Install required packages and the gstlal-cal package on a machine at AS
  * '''DT has written scripts for installing lalsuite and gstlal software on a clean CentOS7 distro based on SH's instructions.''' Scripts were uploaded to KAGRA-AS group's Gitlab repositories:
   . [[https://gitlab.com/kagra-as/docker-lalsuite-cold|containerized "cold" version of lalsuite]] [[https://gitlab.com/kagra-as/docker-gstlal-cold|containerized "cold" version of gstlal-calibration]]
  * '''CI pipelines that build docker-lalsuite-cold and docker-gstlal-cold containers were set up to run on AS IoP machine.'''
 * Test gstlal-cal pipeline on an AS machine
  * '''As a prerequisite to this task, DT reproduced a portion of LIGO h(t) from raw data with gstlal-calibration-v1.1.8 (w/ a "docker-gstlal-cold" image).''' This test was done on a private server (2-cores on Intel Xeon 2.4GHz, 8GB RAM).
  * '''Downloaded a few hours of bKAGRA Phase-1 data on AS machine.'''
  * '''(doing) Will try to reproduce C00 with gstlal-calibration.''' Details in
   . [[https://gitlab.com/kagra-as/low-latency-cal/test-gstlal-on-cal-iop|gstlal-calibration testing on AS machine (cal-iop)]]
  * '''(to do) Verify the output of gstlal-calibration.'''
 * Install prerequisites and gstlal-cal package on a machine at KAGRA
  * '''(done, entry from June 7)''' SH made progress on this, see [[KAGRA/Subgroups/CAL/WG/offline/gstlal|gstlal installation]].
 * Test the pipeline on a KAGRA machine [[KAGRA/Subgroups/CAL/WG/offline/tests|gstlal tests]]
  * '''to do'''
 * Documentation
  * '''(doing) Started documenting the work in KAGRA-AS group's Gitlab repositories:'''
   . [[https://gitlab.com/kagra-as/docker-lalsuite-cold|containerized "cold" version of lalsuite]] [[https://gitlab.com/kagra-as/docker-gstlal-cold|containerized "cold" version of gstlal-calibration]] [[https://gitlab.com/kagra-as/low-latency-cal/test-gstlal-on-cal-iop|gstlal-calibration testing on AS machine (cal-iop)]]
Line 13: Line 155:
   * Ask LIGO CAL team for the detailed information of gstlal-calibration
     * '''Communication with people working on LIGO gstlal has been initiated.'''
   * Find names and versions of all prerequisite libraries,
     * '''SK made progress on this, see [[KAGRA/Subgroups/CAL/WG/offline/gstlal|gstlal installation]]'''
   * Install prerequisites and gstlal-cal package on a machine at AS
    * '''As a prerequisite to this task, SK worked on installation process for CentOS, DT for Debian.'''
     * '''The goal is to have a better understanding of what is happening under the hood when the gstlal-calibration package is compiled.'''
   * Test gstlal-cal pipeline on an AS machine
    * '''As a prerequisite to this task, DT is currently trying to reproduce LIGO h(t) from raw data.'''
     * '''A sub-goal of this task is to come up with a gstlal code testing procedure.'''
    * '''DT addressed some questions to LIGO gstlal people.'''
   * Install prerequisites and gstlal-cal package on a machine at KAGRA
     * '''SK made progress on this, see [[KAGRA/Subgroups/CAL/WG/offline/gstlal|gstlal installation]]'''
   * Test the pipeline on a KAGRA machine [[KAGRA/Subgroups/CAL/WG/offline/tests|gstlal tests]]
     * '''to do'''
* Ask LIGO CAL team for the detailed information of gstlal-calibration
  * '''Communication with people working on LIGO gstlal has been initiated.'''
 * Find names and versions of all prerequisite libraries,
  * '''SH made progress on this, see [[KAGRA/Subgroups/CAL/WG/offline/gstlal|gstlal installation]]'''
 * Install prerequisites and gstlal-cal package on a machine at AS
  * '''As a prerequisite to this task, SH worked on installation process for CentOS, DT for Debian.'''
  * '''The goal is to have a better understanding of what is happening under the hood when the gstlal-calibration package is compiled.'''
 * Test gstlal-cal pipeline on an AS machine
  * '''As a prerequisite to this task, DT is currently trying to reproduce LIGO h(t) from raw data.'''
  * '''A sub-goal of this task is to come up with a gstlal code testing procedure.'''
  * '''DT addressed some questions to LIGO gstlal people.'''
 * Install prerequisites and gstlal-cal package on a machine at KAGRA
  * '''SH made progress on this, see [[KAGRA/Subgroups/CAL/WG/offline/gstlal|gstlal installation]]'''
 * Test the pipeline on a KAGRA machine [[KAGRA/Subgroups/CAL/WG/offline/tests|gstlal tests]]
  * '''to do'''
Line 30: Line 173:
   * Put more effort on documentation.
* Put more effort on documentation.

Prepare h(t) reconstruction review

LIGO time-domain calibration review wiki: https://wiki.ligo.org/Calibration/TDCalibReview

Calibration State Vector (K1:DCS-CALIB_STATE_VECTOR_C10)

(Under preparation. It's planned to be used in O3b)

bit

Identifier

Description

0

HOFT_OK

$h(t)$ was successfully computed (logical AND of bits ...)

1

OBS_INTENT

The observation intent is set by the operator

2

OBS_READY_OR_OBSERVING

Interferometer is in observation ready mode: the LSC_LOCK+ guardian is in the nominal (observation) state.

3

NO_STOCHASTIC_HW_INJ

No stochastic hardware injections present

4

NO_CBC_HW_INJ

No compact binary coalescence hardware injections present

5

NO_BURST_HW_INJ

No burst hardware injections present

6

NO_DETCHAR_HW_INJ

No detector characterization hardware injections present

7

NO_CW_HW_INJ

No continuous wave characterization hardware injections present

8

TM_LINE_COH_OK

TM line coherence OK

9

IM_LINE_COH_OK

IM line coherence OK

10

MN_LINE_COH_OK

MN line coherence OK

11

PCAL1_LINE_COH_OK

PCAL1 line coherence OK

12

PCAL2_LINE_COH_OK

PCAL2 line coherence OK

13

PCAL3_LINE_COH_OK

PCAL3 line coherence OK

14

PCAL4_LINE_COH_OK

PCAL4 line coherence OK

15

PCAL5_LINE_COH_OK

PCAL5 line coherence OK

16

SMOOTH_KAPPA_TM_OK

TDCF smoothed $\kappa_\mathrm{TM}$ values are in the expected range

17

SMOOTH_KAPPA_IM_OK

TDCF smoothed $\kappa_\mathrm{IM}$ values are in the expected range

18

SMOOTH_KAPPA_MN_OK

TDCF smoothed $\kappa_\mathrm{MN}$ values are in the expected range

19

SMOOTH_KAPPA_C_OK

TDCF smoothed $\kappa_\mathrm{C}$ value is in the expected range

20

SMOOTH_F_CC_OK

TDCF smoothed $f_\mathrm{cc}$ value is in the expected range

21

SMOOTH_F_S_OK

(reserved) TDCF smoothed $f_\mathrm{S}$ value is in the expected range

22

SMOOTH_Q_S_OK

(reserved) TDCF smoothed $Q_\mathrm{S}$ value is in the expected range

23

TDCFS_APPLIED

Time-dependent calibration factors are being applied

24

LINE_SUBTRACTION_APPLIED

(reserved) Line subtraction are being applied

25

NO_GAP

(reserved) Input data has no gaps, underflow, or overflow values

26

(not in use)

(not in use)

27

(not in use)

(not in use)

28

(not in use)

(not in use)

29

(not in use)

(not in use)

30

(not in use)

(not in use)

31

(not in use)

(not in use)

Some channels were copied from DET channels

K1:DET-DQ_STATE_VECTOR --> CAL state vector

  • # bit 2 : SDF flag (= K1:GRD-SYS_SDF_OK) --> # bit 2

    # bit 7 : NO Stoch injection --> # bit 3

    # bit 8 : NO CBC injection --> # bit 4

    # bit 9 : NO Burst injection --> # bit 5

    # bit 10 : NO DET injection --> # bit 6

    # bit 11 : NO CW injection --> # bit 7

K1:DET-OBS_INTENT --> CAL state vector: # bit 1

Question: Would we better to change the channel name from K1:DCS-CALIB_STATE_VECTOR_C10 to K1:DCS-CALIB_STATE_VECTOR ?

At the first half of observing run with PRFPMI for O3 (Late Feb. 2020. - Mid. Mar. 2020),

  • HOFT_OK bit was computed with the logical AND of bits : 1-7. (All of these bits were just copied from DET channels)

At the latter half of observation (Late Mar. 2020 -),

  • HOFT_OK bit will be planned to be computed with the logical AND of bits : 1-7, 16-20, 23, 25 (or 1-20, 23, 25).

Low-latency and offline

Channels used for low-latency and offline calibration (July 12 - ...)

Channel Name

Rate

Format

Description

K1:CAL-CS_PROC_MICH_RESIDUAL_DBL_DQ

16384

F64LE

MICH error

K1:CAL-CS_PROC_MICH_DELTA_CTRL_TM_DBL_DQ

4096

F64LE

MICH TM stage control

K1:CAL-CS_PROC_MICH_DELTA_CTRL_IM_DBL_DQ

4096

F64LE

MICH IM stage control

K1:CAL-CS_PROC_MICH_DISPLACEMENT_DQ

16384

F32LE

MICH displacement (optional)

K1:CAL-CS_PROC_IMC_RESIDUAL_DQ

16384

F32LE

IMC error

K1:CAL-CS_PROC_IMC_DELTA_CTRL_AOM_DQ

4096

F32LE

IMC AOM frequency control

K1:CAL-CS_PROC_IMC_DELTA_CTRL_TM_DQ

4096

F32LE

IMC TM stage control

K1:CAL-CS_PROC_IMC_FREQUENCY_DQ

16384

F32LE

Laser frequency (optional)

K1:GRD-LSC_LOCK_OK

16

F32LE

LSC Guardian OK status

K1:GRD-LSC_LOCK_STATE_N

16

F32LE

LSC lock state

K1:GRD-IO_OK

16

F32LE

IMC Guardian OK status

K1:GRD-IO_STATE_N

16

F32LE

IMC state

Where F32LE and F64LE means: floating-point 32-bit and 64-bit numbers in Little-Endian format.

Channels marked "optional" will not be used in the low-latency and offline computations, K1:CAL-CS_PROC_MICH_DISPLACEMENT_DQ and K1:CAL-CS_PROC_IMC_FREQUENCY_DQ in particular are the channels calibrated with the online system.

Low-latency h(t) reconstruction server (Aug - Oct/31: DT,SH,TY,NK,...)

  • We already have three computing nodes to test the offline h(t) reconstruction via frame file but we need to install another for the low-latency h(t) reconstruction via shared memory inside the tunnel

  • (Aug/20-) Connection and software test between a prototype PC and a test bench
    • Prototype PC is just a standard desktop PC and will be prepared by SH
  • (Aug-Sep) Procurement of server (2U height, at least two 10G network card, standard CPU,memory and disk)
    • According to Darkhan’s test with LIGO data and discussion with LIGO people, standard spec should be OK.
    • DT is working to figure out the spec and model.
    • Academia Sinica will make the procurement
  • (Mid.Sep) Connection test between the server and the test bench
    • We also need the collaboration with DMG people for the data transfer
  • (Sep-Oct) Connection test between the server and the KAGRA online system
    • We also need the collaboration with DMG people for the data transfer

Current tasks (May 26 - July 15):

  • Ask to LIGO CAL team for the detailed information of gstlal-calibration (DT) [status: ongoing | expected finish: N/A]

  • Make gstlal-calibration running on a machine at KAGRA (DT, SH) [status: ongoing | expected finish: Jul 15]

    • Find names and versions of all prerequisite libraries
    • Install prerequisites and gstlal-cal package on a machine at AS
    • Test gstlal-cal pipeline on an AS machine
    • Install prerequisites and gstlal-cal package on a machine at KAGRA
    • Test the pipeline on a KAGRA machine

Progress so far (June 21):

  • Ask LIGO CAL team for the detailed information of gstlal-calibration

    • (ongoing) gstlal-calibration maintainers in LIGO are helping to resolve some installation and testing issues.

  • Find names and versions of all prerequisite libraries,
  • Install required packages and the gstlal-cal package on a machine at AS
  • Test gstlal-cal pipeline on an AS machine

    • As a prerequisite to this task, DT reproduced a portion of LIGO h(t) from raw data with gstlal-calibration-v1.1.8 (w/ a "docker-gstlal-cold" image). This test was done on a private server (2-cores on Intel Xeon 2.4GHz, 8GB RAM).

    • Downloaded a few hours of bKAGRA Phase-1 data on AS machine.

    • (doing) Will try to reproduce C00 with gstlal-calibration. Details in

    • (to do) Verify the output of gstlal-calibration.

  • Install prerequisites and gstlal-cal package on a machine at KAGRA

  • Test the pipeline on a KAGRA machine gstlal tests

    • to do

  • Documentation

Progress so far (June 7):

  • Ask LIGO CAL team for the detailed information of gstlal-calibration

    • Communication with people working on LIGO gstlal has been initiated.

  • Find names and versions of all prerequisite libraries,
  • Install prerequisites and gstlal-cal package on a machine at AS

    • As a prerequisite to this task, SH worked on installation process for CentOS, DT for Debian.

    • The goal is to have a better understanding of what is happening under the hood when the gstlal-calibration package is compiled.

  • Test gstlal-cal pipeline on an AS machine

    • As a prerequisite to this task, DT is currently trying to reproduce LIGO h(t) from raw data.

    • A sub-goal of this task is to come up with a gstlal code testing procedure.

    • DT addressed some questions to LIGO gstlal people.

  • Install prerequisites and gstlal-cal package on a machine at KAGRA

  • Test the pipeline on a KAGRA machine gstlal tests

    • to do

New tasks:

  • Put more effort on documentation.

KAGRA/Subgroups/CAL/Low-latency_and_offline (last edited 2020-07-25 03:17:08 by darkhan.tuyenbayev)