Size: 24019
Comment:
|
Size: 27901
Comment:
|
Deletions are marked like this. | Additions are marked like this. |
Line 13: | Line 13: |
* [[https://kgwg.nims.re.kr/cbcwiki/CAGMon | KGWG wiki link]] * [[http://gwwiki.icrr.u-tokyo.ac.jp/JGWwiki/CAGMon| KAGRA wiki link]] |
|
Line 104: | Line 105: |
* Fundamental critarian or guideline of the stride and its data-size | * Fundamental criteria or guideline of CAGMon parameters, such as the stride, the sample rate, and its data-size |
Line 108: | Line 109: |
== Cross-validation == 1. Apply to glitch data on KAGRA during O3GK * Glitch information * [[https://docs.google.com/spreadsheets/d/1JxC3QL6jF3xmA0MnWtWO_dUgNOF_i5enD_j4yUK1X7s/edit#gid=417713112 | KAGRA glitch catalog]] * Purpose * To decide on appropriate parameters when we run CAGMon for searching glitches and correlation * To make recommended parameters in the short-range analysis * Result * [[ | Glitch Catalog for cross-validation of CAGMon]] * Conclusion 2. Apply to the glitch data of GravitySpy on LIGO * Data * TBD 3. Apply to the mid-range data * Data * TBD 4. Apply to the long-range data * Data * TBD |
|
Line 110: | Line 133: |
1. Earthquake effects during O3GK | 1. Earthquake effects during O3GK (with CAGMon Etude Flat) |
Line 124: | Line 147: |
2. Skim through all obs-segments of O3GK * Purpose * Test for calculation time and required resources with all observation segments during O3GK * To figure out trigger events or abnormal behaviors * Computing resource * KISTI-LDG * Requested CPUS: 32cores * Requested memory: 128GB * Results || Date || GPS time || Data length || Stride || Sample rate || Data size || Summary page link || Remarks || || April 7 || 1270287158 - 1270328032 || 11h || 500s || 16Hz || about 8,000 || [[ | summary page]] || processing time: 4h12m / memory usage: 42GB || || || || || 240s || 32Hz || about 8,000 || [[ | summary page]] || processing time: 5h21m / memory usage: 23GB || || || || || 120s || 64Hz || about 8,000 || [[ | summary page]] || processing time: 17h10m / memory usage: 41.9GB || || || || || 60s || 128Hz || about 8,000 || [[ | summary page]] || processing time: 22h03m / memory usage: 28.8GB || || || || || 30s || 256Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 15s || 512Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 8s || 1024Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 4s || 2048Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 2s || 4096Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 1s || 8192Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || April 8 || 1270339218 - 1270425618 || 24h || || || || || Full Data is unavailable in the KISTI cluster || || April 9 || 1270425618 - 1270510167 || 23h || 500s || 16Hz || about 8,000 || [[ | summary page]] || processing time: h / memory usage: GB || || || || || 240s || 32Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 120s || 64Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 60s || 128Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 30s || 256Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 15s || 512Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 8s || 1024Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 4s || 2048Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 2s || 4096Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 1s || 8192Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || April 10 || 1270513160 - 1270596544 || 23h || 500s || 16Hz || about 8,000 || [[ | summary page]] || processing time: h / memory usage: GB || || || || || 240s || 32Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 120s || 64Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 60s || 128Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 30s || 256Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 15s || 512Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 8s || 1024Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 4s || 2048Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 2s || 4096Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 1s || 8192Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || April 11 || 1270598418 - 1270683904 || 23h || 500s || 16Hz || about 8,000 || [[ | summary page]] || processing time: h / memory usage: GB || || || || || 240s || 32Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 120s || 64Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 60s || 128Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 30s || 256Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 15s || 512Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 8s || 1024Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 4s || 2048Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 2s || 4096Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 1s || 8192Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || April 12 || 1270684818 - 1270762046 || 21h || 500s || 16Hz || about 8,000 || [[ | summary page]] || || || || || || 240s || 32Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 120s || 64Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 60s || 128Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 30s || 256Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 15s || 512Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 8s || 1024Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 4s || 2048Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 2s || 4096Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 1s || 8192Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || April 14 || 1270909686 - 1270937768 || 7h || 500s || 16Hz || about 8,000 || [[ | summary page]] || || || || || || 240s || 32Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 120s || 64Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 60s || 128Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 30s || 256Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 15s || 512Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 8s || 1024Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 4s || 2048Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 2s || 4096Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 1s || 8192Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || April 15 || 1270945288 - 1271017582 || 20h || || || || || GRB200415 (08:48:05 UTC) / Full Data is unavailable in the KISTI cluster || || April 16 || 1271030433 - 1271112809 || 22h || 500s || 16Hz || about 8,000 || [[ | summary page]] || processing time: h / memory usage: GB || || || || || 240s || 32Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 120s || 64Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 60s || 128Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 30s || 256Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 15s || 512Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 8s || 1024Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 4s || 2048Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 2s || 4096Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 1s || 8192Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || April 17 || 1271119833 - 1271186507 || 18h || 500s || 16Hz || about 8,000 || [[ | summary page]] || processing time: h / memory usage: GB || || || || || 240s || 32Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 120s || 64Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 60s || 128Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 30s || 256Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 15s || 512Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 8s || 1024Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 4s || 2048Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 2s || 4096Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 1s || 8192Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || April 18 || 1271227441 - 1271288128 || 16h || 500s || 16Hz || about 8,000 || [[ | summary page]] || processing time: h / memory usage: GB || || || || || 240s || 32Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 120s || 64Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 60s || 128Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 30s || 256Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 15s || 512Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 8s || 1024Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 4s || 2048Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 2s || 4096Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 1s || 8192Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || April 19 || 1271289618 - 1271364033 || 20h || 500s || 16Hz || about 8,000 || [[ | summary page]] || processing time: h / memory usage: GB || || || || || 240s || 32Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 120s || 64Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 60s || 128Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 30s || 256Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 15s || 512Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 8s || 1024Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 4s || 2048Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 2s || 4096Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 1s || 8192Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || April 20 || 1271377409 - 1271460608 || 23h || || || || || GRB200420A (2:32:58 UTC) / Full Data is unavailable in the KISTI cluster || 3. With iKAGRA hardware injection data |
2. With iKAGRA hardware injection data (with CAGMon Etude Flat) |
Line 266: | Line 175: |
3. Skim through all obs-segments of O3GK (with CAGMon Etude Octave) * Purpose * Test for calculation time and required resources with all observation segments during O3GK * To figure out trigger events or abnormal behaviors * Computing resource * KISTI-LDG * Requested CPUS: 32cores * Requested memory: 128GB * Results || Date || GPS time || Data length || Stride || Sample rate || Data size || Summary page link || Remarks || || April 7 || 1270287158 - 1270328032 || 11h || 500s || 16Hz || about 8,000 || [[https://ldas-jobs.ligo.caltech.edu/~pil-jong.jung/CAGMon/O3GK/2020-04-07_K1:CAL-CS_PROC_C00_STRAIN_DBL_DQ_1270287158-1270328032_500_16/ | summary page]] || processing time: 4h12m / memory usage: 42GB || || || || || 240s || 32Hz || about 8,000 || [[https://ldas-jobs.ligo.caltech.edu/~pil-jong.jung/CAGMon/O3GK/2020-04-07_K1:CAL-CS_PROC_C00_STRAIN_DBL_DQ_1270287158-1270328032_240_32/ | summary page]] || processing time: 5h21m / memory usage: 23GB || || || || || 120s || 64Hz || about 8,000 || [[https://ldas-jobs.ligo.caltech.edu/~pil-jong.jung/CAGMon/O3GK/2020-04-07_K1:CAL-CS_PROC_C00_STRAIN_DBL_DQ_1270287158-1270328032_120_64/ | summary page]] || processing time: 17h10m / memory usage: 41.9GB || || || || || 60s || 128Hz || about 8,000 || [[https://ldas-jobs.ligo.caltech.edu/~pil-jong.jung/CAGMon/O3GK/2020-04-07_K1:CAL-CS_PROC_C00_STRAIN_DBL_DQ_1270287158-1270328032_60_128/ | summary page]] || processing time: 23h03m / memory usage: 28.8GB || || || || || 30s || 256Hz || about 8,000 || [[https://ldas-jobs.ligo.caltech.edu/~pil-jong.jung/CAGMon/O3GK/2020-04-07_K1:CAL-CS_PROC_C00_STRAIN_DBL_DQ_1270287158-1270328032_30_256/ | summary page]] || processing time: 1d23h / memory usage: 24GB || || || || || 15s || 512Hz || about 8,000 || [[ | summary page]] || processing time: > 3 days => killed || || || || || 8s || 1024Hz || about 8,000 || [[ | summary page]] || processing time: > 3 days => killed || || || || || 4s || 2048Hz || about 8,000 || [[ | summary page]] || processing time: > 3 days => killed || || || || || 2s || 4096Hz || about 8,000 || [[ | summary page]] || processing time: > 3 days => killed || || || || || 1s || 8192Hz || about 8,000 || [[ | summary page]] || processing time: > 3 days => killed || || April 8 || 1270339218 - 1270425618 || 24h || || || || || Full Data is unavailable in the KISTI cluster || || April 9 || 1270425618 - 1270510167 || 23h || 500s || 16Hz || about 8,000 || [[ | summary page]] || processing time: h / memory usage: GB || || || || || 240s || 32Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 120s || 64Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 60s || 128Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 30s || 256Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 15s || 512Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 8s || 1024Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 4s || 2048Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 2s || 4096Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 1s || 8192Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || April 10 || 1270513160 - 1270596544 || 23h || 500s || 16Hz || about 8,000 || [[ | summary page]] || processing time: h / memory usage: GB || || || || || 240s || 32Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 120s || 64Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 60s || 128Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 30s || 256Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 15s || 512Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 8s || 1024Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 4s || 2048Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 2s || 4096Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 1s || 8192Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || April 11 || 1270598418 - 1270683904 || 23h || 500s || 16Hz || about 8,000 || [[ | summary page]] || processing time: h / memory usage: GB || || || || || 240s || 32Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 120s || 64Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 60s || 128Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 30s || 256Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 15s || 512Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 8s || 1024Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 4s || 2048Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 2s || 4096Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 1s || 8192Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || April 12 || 1270684818 - 1270762046 || 21h || 500s || 16Hz || about 8,000 || [[ | summary page]] || || || || || || 240s || 32Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 120s || 64Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 60s || 128Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 30s || 256Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 15s || 512Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 8s || 1024Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 4s || 2048Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 2s || 4096Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 1s || 8192Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || April 14 || 1270909686 - 1270937768 || 7h || 500s || 16Hz || about 8,000 || [[https://ldas-jobs.ligo.caltech.edu/~pil-jong.jung/CAGMon/O3GK/2020-04-14_K1:CAL-CS_PROC_C00_STRAIN_DBL_DQ_1270909686-1270937768_500_16/ | summary page]] || processing time: 50m / memory usage: 1.9GB || || || || || 240s || 32Hz || about 8,000 || [[https://ldas-jobs.ligo.caltech.edu/~pil-jong.jung/CAGMon/O3GK/2020-04-14_K1:CAL-CS_PROC_C00_STRAIN_DBL_DQ_1270909686-1270937768_240_32/ | summary page]] || processing time: 2h8m / memory usage: 2.4GB || || || || || 120s || 64Hz || about 8,000 || [[https://ldas-jobs.ligo.caltech.edu/~pil-jong.jung/CAGMon/O3GK/2020-04-14_K1:CAL-CS_PROC_C00_STRAIN_DBL_DQ_1270909686-1270937768_120_64/ | summary page]] || processing time: 4h40m / memory usage: 3.6GB || || || || || 60s || 128Hz || about 8,000 || [[https://ldas-jobs.ligo.caltech.edu/~pil-jong.jung/CAGMon/O3GK/2020-04-14_K1:CAL-CS_PROC_C00_STRAIN_DBL_DQ_1270909686-1270937768_60_128/ | summary page]] || processing time: 8h30m / memory usage: 2.0GB || || || || || 30s || 256Hz || about 8,000 || [[https://ldas-jobs.ligo.caltech.edu/~pil-jong.jung/CAGMon/O3GK/2020-04-14_K1:CAL-CS_PROC_C00_STRAIN_DBL_DQ_1270909686-1270937768_30_256/ | summary page]] || processing time: 15h30m / memory usage: 2.2GB || || || || || 500s || 32Hz || about 16,000 || [[ | summary page]] || processing time: 50m / memory usage: 3.2GB || || || || || 240s || 64Hz || about 16,000 || [[ | summary page]] || processing time: 5h25m / memory usage: 3.3GB || || || || || 120s || 128Hz || about 16,000 || [[ | summary page]] || processing time: 10h10m / memory usage: 3.4GB || || || || || 60s || 256Hz || about 16,000 || [[ | summary page]] || processing time: 18h50m / memory usage: 3.4GB || || || || || 30s || 512Hz || about 16,000 || [[ | summary page]] || processing time: 17h / memory usage: 3.7GB || || || || || 500s || 64Hz || about 36,000 || [[ | summary page]] || processing time: 6h30m / memory usage: 6.7GB || || || || || 240s || 128Hz || about 36,000 || [[ | summary page]] || processing time: 10h57m / memory usage: 6.7GB || || || || || 120s || 256Hz || about 36,000 || [[ | summary page]] || processing time: 17h3m / memory usage: 7.0GB || || || || || 60s || 512Hz || about 36,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 30s || 1024Hz || about 36,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 500s || 128Hz || about 64,000 || [[ | summary page]] || processing time: 12h10m / memory usage: 14.6GB || || || || || 240s || 256Hz || about 64,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 120s || 512Hz || about 64,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 60s || 1024Hz || about 64,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 30s || 2048Hz || about 64,000 || [[ | summary page]] || processing time: / memory usage: GB || || April 15 || 1270945288 - 1271017582 || 20h || || || || || GRB200415 (08:48:05 UTC) / Full Data is unavailable in the KISTI cluster || || April 16 || 1271030433 - 1271112809 || 22h || 500s || 16Hz || about 8,000 || [[ | summary page]] || processing time: h / memory usage: GB || || || || || 240s || 32Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 120s || 64Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 60s || 128Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 30s || 256Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 15s || 512Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 8s || 1024Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 4s || 2048Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 2s || 4096Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 1s || 8192Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || April 17 || 1271119833 - 1271186507 || 18h || 500s || 16Hz || about 8,000 || [[ | summary page]] || processing time: h / memory usage: GB || || || || || 240s || 32Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 120s || 64Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 60s || 128Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 30s || 256Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 15s || 512Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 8s || 1024Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 4s || 2048Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 2s || 4096Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 1s || 8192Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || April 18 || 1271227441 - 1271288128 || 16h || 500s || 16Hz || about 8,000 || [[ | summary page]] || processing time: h / memory usage: GB || || || || || 240s || 32Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 120s || 64Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 60s || 128Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 30s || 256Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 15s || 512Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 8s || 1024Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 4s || 2048Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 2s || 4096Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 1s || 8192Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || April 19 || 1271289618 - 1271364033 || 20h || 500s || 16Hz || about 8,000 || [[ | summary page]] || processing time: h / memory usage: GB || || || || || 240s || 32Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 120s || 64Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 60s || 128Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 30s || 256Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 15s || 512Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 8s || 1024Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 4s || 2048Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 2s || 4096Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || || || || 1s || 8192Hz || about 8,000 || [[ | summary page]] || processing time: / memory usage: GB || || April 20 || 1271377409 - 1271460608 || 23h || || || || || GRB200420A (2:32:58 UTC) / Full Data is unavailable in the KISTI cluster || |
,-----. ,---. ,----. ,--. ,--. ,--. ,--. ' .--./ / O \ ' .-./ | `.' | ,---. ,--,--, ,---. ,-' '-.,--.,--. ,-| | ,---. | | | .-. || | .---.| |'.'| || .-. || \ | .-. :'-. .-'| || |' .-. || .-. : ' '--'\| | | |' '--' || | | |' '-' '| || | \ --. | | ' '' '\ `-' |\ --. `-----'`--' `--' `------' `--' `--' `---' `--''--' `----' `--' `----' `---' `----'
Description
The CAGMon etude is a study version of CAGMon that evaluates the dependence between the primary and auxiliary channels.
Project goal
The goal of this project is to find a systematic way of identifying the abnormal glitches in the gravitational-wave data using various methods of correlation analysis. Usually, the community such as LIGO, Virgo, and KAGRA uses a conventional way of finding glitches in auxiliary channels of the detector - Klein-Welle, Omicron, Ordered Veto Lists, etc. However, some different ways can be possible to find and monitor them in a (quasi-) realtime. Also, the method can point out which channel is responsible for the found glitch. In this project, we study its possible to apply three different correlation methods - maximal information coefficient, Pearson's correlation coefficient, and Kendall's tau coefficient - in the gravitational wave data from the KAGRA detector.
Participants
- John.J Oh (NIMS)
- Young-Min Kim (UNIST)
- Pil-Jong Jung (NIMS)
Methods and Frameworks
Maximal Information Coefficient (MIC)
the Maximal Information coefficient(MIC) of a set D of two-variable data with sample size n and the grid less than B(n) is given by
\[ MIC(D)=\underset{xy<B(n)}{\max}{\left\{ \frac{I^{*}(D,x,y)}{\log \min \left\{x,y \right\}} \right \}} \],
where \[\omega(1)<B(n)\le O(n^{1-\epsilon}) \] for some \[ 0<\epsilon<1 \]
Pearson's Correlation Coefficient (PCC)
Pearson Correlation Coefficient(PCC) is a statistic that explains the amount of variance accounted for in the relationship between two (or more) variables by \[ R=} \],
where \[ \overline{X} \] and \[ \overline{Y} \] are the mean of X and Y, respectively
Kendall's tau Coefficient
Kendall’s tau with a random samples n of observations from two variables measures the strength of the relationship between two ordinal level variables by
\[ \tau =\frac{c-d} \],
where c is the number of concordant pairs, and d is the number of discordant pairs
Flow chart
Code development
GitHub
Code versions
- CAGMon Etude Alpha
- for the basic test and evaluation of the LASSO regression method developed by LIGO
- reproduced original CAGMon methods and idea
- CAGMon Etude Beta
- added coefficient trend plots with LASSO beta, coherence, MIC, PCC, and Kendall's tau
- CAGMon Etude Delta
- fixed a critical problem that sucked enormous memory when it used the matplotlib module
- CAGMon Etude Eta
- fixed minor issues
- added the range limitation of stride
- CAGMon Etude Flat (current version)
- fixed minor issues and optimized scripts
- added the script of HTML summary page
- added coefficient distribution plots
- CAGMon Etude Octave (development version)
- remove some processes that make Time-series and Scatter plots. Even though it required tremendous memory, this information is not useful
- adjust HTML code
- fixed minor issues and optimized scripts
- added the analysis option whether or not the algorithm proceeds in the active segment only
- improve script efficiency
Series of scripts
- Agrement.py
- the script gathered functions the model required
- Melody.py
- the script to calcutate each coefficient and to save trend data as csv
- Conchord.py
- the script to make plots
- Echo.py
- the script to save the result as HTML web page
- CAGMonEtude{Version}.py
- the script to run each script
User guide
Needs of code development
- Fundamental criteria or guideline of CAGMon parameters, such as the stride, the sample rate, and its data-size
- Daily running on KAGRA
Cross-validation
- Apply to glitch data on KAGRA during O3GK
- Glitch information
- Purpose
- To decide on appropriate parameters when we run CAGMon for searching glitches and correlation
- To make recommended parameters in the short-range analysis
- Result
- Conclusion
Apply to the glitch data of GravitySpy on LIGO
- Data
- TBD
- Data
- Apply to the mid-range data
- Data
- TBD
- Data
- Apply to the long-range data
- Data
- TBD
- Data
Exemplary results
1. Earthquake effects during O3GK (with CAGMon Etude Flat)
- Datetime: 19 April 2020 20:39 UTC
- Purpose
- Test to run CAGMon algorithm with a remarkable event
- To figure out the cause of lock-loss in KAGRA
- Computing resource
- KISTI-LDG
- Requested CPUS: 32cores
- Requested memory: 128GB
- Results
stride 5 seconds Summary page
stride 20 seconds Summary page
stride 30 seconds Summary page
2. With iKAGRA hardware injection data (with CAGMon Etude Flat)
- Event
- Phenomenon: the strain channel and seismometer channels in iKAGRA had a high correlation during the hardware injection test
- Cause: still unknown
- Hypothesis: the glitches have relatively the same behavior as the vacuum rotary pump
More detail analysis: h-veto brief Report for K1 and KGWG Face-to-Face Meeting
- Purpose
- To verify whether this model senses injected signals and abnormal glitches
- To test noise resistance and data-size limitation
- Computing resource
- KISTI-LDG
- Requested CPUS: 32cores
- Requested memory: 64GB
- Results
Stride
Sample sata
Data size
Dada length
Summary page link
10s
512Hz
about 5,000
about 12m
10s
1024Hz
about 10,000
about 12m
10s
2048Hz
about 20,000
about 12m
10s
3072Hz
about 30,000
about 12m
10s
4096Hz
about 40,000
about 12m
2s
4096Hz
about 8,000
about 12m
5s
4096Hz
about 20,000
about 12m
60s
128Hz
about 7,500
whole iKAGRA data
150s
64Hz
about 10,000
whole iKAGRA data
300s
64Hz
about 20,000
whole iKAGRA data
600s
16Hz
about 10,000
whole iKAGRA data
3. Skim through all obs-segments of O3GK (with CAGMon Etude Octave)
- Purpose
- Test for calculation time and required resources with all observation segments during O3GK
- To figure out trigger events or abnormal behaviors
- Computing resource
- KISTI-LDG
- Requested CPUS: 32cores
- Requested memory: 128GB
- Results
Date
GPS time
Data length
Stride
Sample rate
Data size
Summary page link
Remarks
April 7
1270287158 - 1270328032
11h
500s
16Hz
about 8,000
processing time: 4h12m / memory usage: 42GB
240s
32Hz
about 8,000
processing time: 5h21m / memory usage: 23GB
120s
64Hz
about 8,000
processing time: 17h10m / memory usage: 41.9GB
60s
128Hz
about 8,000
processing time: 23h03m / memory usage: 28.8GB
30s
256Hz
about 8,000
processing time: 1d23h / memory usage: 24GB
15s
512Hz
about 8,000
processing time: > 3 days => killed
8s
1024Hz
about 8,000
processing time: > 3 days => killed
4s
2048Hz
about 8,000
processing time: > 3 days => killed
2s
4096Hz
about 8,000
processing time: > 3 days => killed
1s
8192Hz
about 8,000
processing time: > 3 days => killed
April 8
1270339218 - 1270425618
24h
Full Data is unavailable in the KISTI cluster
April 9
1270425618 - 1270510167
23h
500s
16Hz
about 8,000
processing time: h / memory usage: GB
240s
32Hz
about 8,000
processing time: / memory usage: GB
120s
64Hz
about 8,000
processing time: / memory usage: GB
60s
128Hz
about 8,000
processing time: / memory usage: GB
30s
256Hz
about 8,000
processing time: / memory usage: GB
15s
512Hz
about 8,000
processing time: / memory usage: GB
8s
1024Hz
about 8,000
processing time: / memory usage: GB
4s
2048Hz
about 8,000
processing time: / memory usage: GB
2s
4096Hz
about 8,000
processing time: / memory usage: GB
1s
8192Hz
about 8,000
processing time: / memory usage: GB
April 10
1270513160 - 1270596544
23h
500s
16Hz
about 8,000
processing time: h / memory usage: GB
240s
32Hz
about 8,000
processing time: / memory usage: GB
120s
64Hz
about 8,000
processing time: / memory usage: GB
60s
128Hz
about 8,000
processing time: / memory usage: GB
30s
256Hz
about 8,000
processing time: / memory usage: GB
15s
512Hz
about 8,000
processing time: / memory usage: GB
8s
1024Hz
about 8,000
processing time: / memory usage: GB
4s
2048Hz
about 8,000
processing time: / memory usage: GB
2s
4096Hz
about 8,000
processing time: / memory usage: GB
1s
8192Hz
about 8,000
processing time: / memory usage: GB
April 11
1270598418 - 1270683904
23h
500s
16Hz
about 8,000
processing time: h / memory usage: GB
240s
32Hz
about 8,000
processing time: / memory usage: GB
120s
64Hz
about 8,000
processing time: / memory usage: GB
60s
128Hz
about 8,000
processing time: / memory usage: GB
30s
256Hz
about 8,000
processing time: / memory usage: GB
15s
512Hz
about 8,000
processing time: / memory usage: GB
8s
1024Hz
about 8,000
processing time: / memory usage: GB
4s
2048Hz
about 8,000
processing time: / memory usage: GB
2s
4096Hz
about 8,000
processing time: / memory usage: GB
1s
8192Hz
about 8,000
processing time: / memory usage: GB
April 12
1270684818 - 1270762046
21h
500s
16Hz
about 8,000
240s
32Hz
about 8,000
processing time: / memory usage: GB
120s
64Hz
about 8,000
processing time: / memory usage: GB
60s
128Hz
about 8,000
processing time: / memory usage: GB
30s
256Hz
about 8,000
processing time: / memory usage: GB
15s
512Hz
about 8,000
processing time: / memory usage: GB
8s
1024Hz
about 8,000
processing time: / memory usage: GB
4s
2048Hz
about 8,000
processing time: / memory usage: GB
2s
4096Hz
about 8,000
processing time: / memory usage: GB
1s
8192Hz
about 8,000
processing time: / memory usage: GB
April 14
1270909686 - 1270937768
7h
500s
16Hz
about 8,000
processing time: 50m / memory usage: 1.9GB
240s
32Hz
about 8,000
processing time: 2h8m / memory usage: 2.4GB
120s
64Hz
about 8,000
processing time: 4h40m / memory usage: 3.6GB
60s
128Hz
about 8,000
processing time: 8h30m / memory usage: 2.0GB
30s
256Hz
about 8,000
processing time: 15h30m / memory usage: 2.2GB
500s
32Hz
about 16,000
processing time: 50m / memory usage: 3.2GB
240s
64Hz
about 16,000
processing time: 5h25m / memory usage: 3.3GB
120s
128Hz
about 16,000
processing time: 10h10m / memory usage: 3.4GB
60s
256Hz
about 16,000
processing time: 18h50m / memory usage: 3.4GB
30s
512Hz
about 16,000
processing time: 17h / memory usage: 3.7GB
500s
64Hz
about 36,000
processing time: 6h30m / memory usage: 6.7GB
240s
128Hz
about 36,000
processing time: 10h57m / memory usage: 6.7GB
120s
256Hz
about 36,000
processing time: 17h3m / memory usage: 7.0GB
60s
512Hz
about 36,000
processing time: / memory usage: GB
30s
1024Hz
about 36,000
processing time: / memory usage: GB
500s
128Hz
about 64,000
processing time: 12h10m / memory usage: 14.6GB
240s
256Hz
about 64,000
processing time: / memory usage: GB
120s
512Hz
about 64,000
processing time: / memory usage: GB
60s
1024Hz
about 64,000
processing time: / memory usage: GB
30s
2048Hz
about 64,000
processing time: / memory usage: GB
April 15
1270945288 - 1271017582
20h
GRB200415 (08:48:05 UTC) / Full Data is unavailable in the KISTI cluster
April 16
1271030433 - 1271112809
22h
500s
16Hz
about 8,000
processing time: h / memory usage: GB
240s
32Hz
about 8,000
processing time: / memory usage: GB
120s
64Hz
about 8,000
processing time: / memory usage: GB
60s
128Hz
about 8,000
processing time: / memory usage: GB
30s
256Hz
about 8,000
processing time: / memory usage: GB
15s
512Hz
about 8,000
processing time: / memory usage: GB
8s
1024Hz
about 8,000
processing time: / memory usage: GB
4s
2048Hz
about 8,000
processing time: / memory usage: GB
2s
4096Hz
about 8,000
processing time: / memory usage: GB
1s
8192Hz
about 8,000
processing time: / memory usage: GB
April 17
1271119833 - 1271186507
18h
500s
16Hz
about 8,000
processing time: h / memory usage: GB
240s
32Hz
about 8,000
processing time: / memory usage: GB
120s
64Hz
about 8,000
processing time: / memory usage: GB
60s
128Hz
about 8,000
processing time: / memory usage: GB
30s
256Hz
about 8,000
processing time: / memory usage: GB
15s
512Hz
about 8,000
processing time: / memory usage: GB
8s
1024Hz
about 8,000
processing time: / memory usage: GB
4s
2048Hz
about 8,000
processing time: / memory usage: GB
2s
4096Hz
about 8,000
processing time: / memory usage: GB
1s
8192Hz
about 8,000
processing time: / memory usage: GB
April 18
1271227441 - 1271288128
16h
500s
16Hz
about 8,000
processing time: h / memory usage: GB
240s
32Hz
about 8,000
processing time: / memory usage: GB
120s
64Hz
about 8,000
processing time: / memory usage: GB
60s
128Hz
about 8,000
processing time: / memory usage: GB
30s
256Hz
about 8,000
processing time: / memory usage: GB
15s
512Hz
about 8,000
processing time: / memory usage: GB
8s
1024Hz
about 8,000
processing time: / memory usage: GB
4s
2048Hz
about 8,000
processing time: / memory usage: GB
2s
4096Hz
about 8,000
processing time: / memory usage: GB
1s
8192Hz
about 8,000
processing time: / memory usage: GB
April 19
1271289618 - 1271364033
20h
500s
16Hz
about 8,000
processing time: h / memory usage: GB
240s
32Hz
about 8,000
processing time: / memory usage: GB
120s
64Hz
about 8,000
processing time: / memory usage: GB
60s
128Hz
about 8,000
processing time: / memory usage: GB
30s
256Hz
about 8,000
processing time: / memory usage: GB
15s
512Hz
about 8,000
processing time: / memory usage: GB
8s
1024Hz
about 8,000
processing time: / memory usage: GB
4s
2048Hz
about 8,000
processing time: / memory usage: GB
2s
4096Hz
about 8,000
processing time: / memory usage: GB
1s
8192Hz
about 8,000
processing time: / memory usage: GB
April 20
1271377409 - 1271460608
23h
GRB200420A (2:32:58 UTC) / Full Data is unavailable in the KISTI cluster
Beyond
References
Presentation materials
Papers
Science.1518; Detecting Novel Associations in Large Data Sets