Differences between revisions 2 and 14 (spanning 12 versions)
⇤ ← Revision 2 as of 2021-01-20 13:51:03 →
Size: 1502
Editor: PJJung
Comment:
← Revision 14 as of 2021-01-21 15:09:47 → ⇥
Size: 4435
Editor: PJJung
Comment:
Deletions are marked like this. Additions are marked like this.
Line 2: Line 2:
== Descripstion == == Description ==
Line 7: Line 7:
== Project Goal == == Project goal ==
Line 22: Line 22:
the Maximal Information coefficient(MIC) of a set D of two-variable data with sample size n and grid less than B(n) is given by
Line 24: Line 25:
r=\frac{\sum_{i=1}^{n} (x_i - \bar{x})(y_i-\bar{y})}{\sqrt{\sum_{i=1}^{n} (x_i-\bar{x})^2} \sqrt{\sum_{i=1}^{n} (y_i -\bar{y})^2}}
\]		 MIC(D)=\underset{xy<B(n)}{\max}{\left\{ \frac{I^{*}(D,x,y)}{\log \min \left\{x,y \right\}} \right \}}
\],
Line 27: Line 28:
where \[\omega(1)<B(n)\le O(n^{1-\epsilon}) \] for some \[ 0<\epsilon<1 \]
==== Pearson's Correlation Coefficient (PCC) ====
Line 28: Line 31:
==== Pearson's Correlation Coefficient (PCC) ==== Pearson Correlation Coefficient(PCC) is a statistic that explains the amount of variance accounted for in the relationship between two (or more) variables by
\[
R={{\sum_{i=1}^{n} (X_i - \overline{X})(Y_i - \overline{Y})} \over {\sqrt{\sum_{i=1}^{n} (X_i - \overline{X}) \sum_{i=1}^{n} (Y_i - \overline{Y})}}}
\],

where \[ \overline{X} \] and \[ \overline{Y} \] are the mean of X and Y, respectively
Line 32: Line 41:
Kendall’s tau with a random samples n of observations from two variables measures the strength of the relationship between two ordinal level variables by

\[
\tau =\frac{c-d}{{n \choose 2}}
\],

where c is the number of concordant pairs, and d is the number of discordant pairs
Line 34: Line 50:
== Exemplary Results == == Code development ==

==== GitHub ====
TBA

==== Code version ====
 1. CAGMon Etude Alpha
   * for the basic test and evaluation of the LASSO regression method developed by LIGO
   * reproduced original CAGMon methods and idea
 2. CAGMon Etude Beta
  * added coefficient trend plots with LASSO beta, coherence, MIC, PCC, and Kendall's tau
 3. CAGMon Etude Delta
  * fixed a critical problem that sucked enormous memory when it used the matplotlib module
 4. CAGMon Etude Eta
  * fixed memory issues
  * fixed minor bugs
  * added the range limitation of stride
 5. CAGMon Etude Flat (latest version)
  * fixed minor issues and optimized scripts
  * added the script of HTML summary page
  * added coefficient distribution plots

==== Structure of scripts ====
 * Agrement.py
  * the script gathered functions the medel required
 * Melody.py
  * the script to calcutate each coefficient and to save trend data as csv
 * Conchord.py
  * the script to make plots, such as coefficient trend, coefficient distribution trend, time-series, and scatter plots
 * Echo.py
  * the script to save the result as HTML web page
 * CAGMonEtudeFlat.py
  * the script to run each script

== Exemplary results ==

1. Earthquake effects during O3GK
 * Datetime: 19 April 2020 20:39 UTC
 * Purpose
  * Test to run CAGMon algorithm with a remarkable event
  * To figure out the cause of lock-loss in KAGRA
 * Result
  * [[https://ldas-jobs.ligo.caltech.edu/~pil-jong.jung/CAGMon/2020-04-19_K1:CAL-CS_PROC_C00_STRAIN_DBL_DQ_1271363358-1271364078(5)/ | Summary page with stride 5 seconds]]
  * [[https://ldas-jobs.ligo.caltech.edu/~pil-jong.jung/CAGMon/2020-04-19_K1:CAL-CS_PROC_C00_STRAIN_DBL_DQ_1271363358-1271364078/ | Summary page with stride 20 seconds]]
  * [[https://ldas-jobs.ligo.caltech.edu/~pil-jong.jung/CAGMon/2020-04-19_K1:CAL-CS_PROC_C00_STRAIN_DBL_DQ_1271363358-1271364078(30)/ | Summary page with stride 30 seconds]]
Line 40: Line 100:
==== Presentation Materials ==== ==== Presentation materials ====
[[https://gwdoc.icrr.u-tokyo.ac.jp/cgi-bin/private/DocDB/ShowDocument?docid=12481|JGW-G2112481-v1]]

CAGMon etude

Description

The CAGMon etude is a study version of CAGMon that evaluates the dependence between the primary and auxiliary channels.

Project goal

The goal of this project is to find a systematic way of identifying the abnormal glitches in the gravitational-wave data using various methods of correlation analysis. Usually, the community such as LIGO, Virgo, and KAGRA uses a conventional way of finding glitches in auxiliary channels of the detector - Klein-Welle, Omicron, Ordered Veto Lists, etc. However, some different ways can be possible to find and monitor them in a (quasi-) realtime. Also, the method can point out which channel is responsible for the found glitch. In this project, we study its possible to apply three different correlation methods - maximal information coefficient, Pearson's correlation coefficient, and Kendall's tau coefficient - in the gravitational wave data from the KAGRA detector.

Participants

  • John.J Oh (NIMS)
  • Young-Min Kim (UNIST)
  • Pil-Jong Jung (NIMS)

Methods and Frameworks

Maximal Information Coefficient (MIC)

the Maximal Information coefficient(MIC) of a set D of two-variable data with sample size n and grid less than B(n) is given by

\[ MIC(D)=\underset{xy<B(n)}{\max}{\left\{ \frac{I^{*}(D,x,y)}{\log \min \left\{x,y \right\}} \right \}} \],

where \[\omega(1)<B(n)\le O(n^{1-\epsilon}) \] for some \[ 0<\epsilon<1 \]

Pearson's Correlation Coefficient (PCC)

Pearson Correlation Coefficient(PCC) is a statistic that explains the amount of variance accounted for in the relationship between two (or more) variables by \[ R=\sum_{i=1}^{n} (X_i - \overline{X})(Y_i - \overline{Y})} \over {\sqrt{\sum_{i=1}^{n} (X_i - \overline{X}) \sum_{i=1}^{n} (Y_i - \overline{Y})} \],

where \[ \overline{X} \] and \[ \overline{Y} \] are the mean of X and Y, respectively

Kendall's tau Coefficient

Kendall’s tau with a random samples n of observations from two variables measures the strength of the relationship between two ordinal level variables by

\[ \tau =\frac{c-d}n \choose 2 \],

where c is the number of concordant pairs, and d is the number of discordant pairs

Code development

GitHub

TBA

Code version

  1. CAGMon Etude Alpha
    • for the basic test and evaluation of the LASSO regression method developed by LIGO
    • reproduced original CAGMon methods and idea
  2. CAGMon Etude Beta
    • added coefficient trend plots with LASSO beta, coherence, MIC, PCC, and Kendall's tau
  3. CAGMon Etude Delta
    • fixed a critical problem that sucked enormous memory when it used the matplotlib module
  4. CAGMon Etude Eta
    • fixed memory issues
    • fixed minor bugs
    • added the range limitation of stride
  5. CAGMon Etude Flat (latest version)
    • fixed minor issues and optimized scripts
    • added the script of HTML summary page
    • added coefficient distribution plots

Structure of scripts

  • Agrement.py
    • the script gathered functions the medel required
  • Melody.py
    • the script to calcutate each coefficient and to save trend data as csv
  • Conchord.py
    • the script to make plots, such as coefficient trend, coefficient distribution trend, time-series, and scatter plots
  • Echo.py
    • the script to save the result as HTML web page
  • CAGMonEtudeFlat.py
    • the script to run each script

Exemplary results

1. Earthquake effects during O3GK

Beyond

References

Presentation materials

JGW-G2112481-v1

Papers

PJJung/CAGMonEtude (last edited 2021-07-28 08:43:57 by PJJung)