Differences between revisions 7 and 11 (spanning 4 versions)

NoEMi

The Noise Frequency Event Miner (NoEMi) is a Virgo software tool dedicated to the monitoring and identification of spectral noise lines.

It uses codes for CW analysis to find and track spectral lines found in a given list of channels. It also looks for spectral features in coincidence between the "main channel" and the "auxiliary channels".

NoEMi documentation: https://apps.virgo-gw.eu/noemi/docs/index.html

This version, has 3 main components:

the NoEMi events database (DB);
the Line-Finder script (LF);
the web user interface (WUI).

In addition to NoEMi the LinesDB will be installed soon.

Requirements

Main requirements:

HTCondor
MySQL
Python
Apache HTTP server
PHP

The following Python libraries can be installed with pip

pip install htcondor
pip install pymysql
pip install numpy

The PHP Data Objects module could be necessary.

MySQL has been installed from scratch, the admin user credentials are (this is the root user for any MySQL operation):

Username: root
Password: user_admin_pass

Get NoEMi

The full NoEMi package is available in the Virgo SVN software repository

Note that, to access the repository, you will need to use your EGO Active Directory credentials. More information on obtaining these is available here.

Download the full package

svn co https://svn.ego-gw.it/svn/advsw/Noemi/branches/v2/trunk \
          NoEMi

The folder will contain the three components of NoEMi and the documentation

     NoEMi/
          /db      --> MySQL Database
          /doc     --> Documentation
          /noemi   --> Line finder python scripts
          /wui     --> Web user interface

Host machines and Users

The 3 components of NoEMi: the DB, the Line finder and the WUI are installed on the following machines

 DB host: localhost
 LF hosts: localhost
 WUI host: localhost --> web page work in progress

The following MySQL users have been created:

1. The web user interface (WUI) MySQL user

Username: noemi_wui
Password: noemi_wui_pass

2. The Line-Finder (LF) MySQL user

Username: noemi_user
Password: noemi_user_pass

Changes for KAGRA

The "K1" detector case has been added to the main SFDB (Short Fourier Transform DataBase) executable. The SFDBs are used also for CW analysis in Virgo.

The list of channels/sampling frequency of the channel used is:

Staus

The NoEMi events database is installed
The Line finder scripts are installed
The WUI output pages are not pubicly visible (coordination with Gary Hemming ongoing)
Communication between the LF scripts and Condor BQS is working
Communication between the LF scripts and the NoEMi events MySQL DB is working

What's missing

The main executable for the creation of SFDBs needs minor modification this is a C compilable code, test ongoing
Automatic creation of ffl files (ffl is simply the list of .gwf frame files where in the first column you have the path to the .gwf file and as a second column the starting GPS time of the frame). A script already exist, needs to be adapted.

-  ⇤ ← Revision 7 as of 2019-10-31 15:16:16 → 
  Size: 2467
  Editor: ornella.piccinni
  Comment:
+   ← Revision 11 as of 2019-11-01 09:57:58 → ⇥
  Size: 3504
  Editor: ornella.piccinni
  Comment:
-Deletions are marked like this.
+Additions are marked like this.
 Line 12:
+This version, has 3 main components:

 * the NoEMi events database (DB);
 * the Line-Finder script (LF);
 * the web user interface (WUI).

In addition to NoEMi the LinesDB will be installed soon.
-Line 22:
+Line 30:
-Line 30:
+Line 37:
 The [[https://www.php.net/manual/en/book.pdo.php|PHP Data Objects module]] could be necessary.
-Line 34:
+Line 40:
+{{{
-Line 36:
+Line 44:
+}}}
-Line 50:
+Line 58:
- *The folder will contain the three components of NoEMi and the documentation
+ * The folder will contain the three components of NoEMi and the documentation
-Line 57:
+Line 66:
-Line 59:
+Line 67:
-Line 61:
+Line 68:
-Line 67:
+Line 73:
- WUI host: localhost
+ WUI host: localhost --> web page work in progress
}}}
The following MySQL users have been created:

1. The web user interface (WUI) MySQL user 

{{{
Username: noemi_wui
Password: noemi_wui_pass
-Line 70:
+Line 84:
-The following MySQL users have been created:
+. The Line-Finder (LF) MySQL user
-Line 72:
+Line 86:
-. The web user interface (WUI) MySQL user
Username: noemi_wui
Password: noemi_wui_pass

2. The Line-Finder (LF) MySQL user
+{{{
-Line 79:
+Line 89:
+}}}
-Line 81:
+Line 92:
-The "K1" detector case has been added to the main SFDB (Short Fourier Transform DataBase) executable.
+The "K1" detector case has been added to the main SFDB (Short Fourier Transform DataBase) executable. The SFDBs are used also for CW analysis in Virgo.
-Line 86:
+Line 97:
+}}}

== Staus ==
 
 * The NoEMi events database is installed
 * The Line finder scripts are installed
 * The WUI output pages are not pubicly visible (coordination with Gary Hemming ongoing)
 * Communication between the LF scripts and Condor BQS is working
 * Communication between the LF scripts and the NoEMi events MySQL DB is working

What's missing 

 * The main executable for the creation of SFDBs needs minor modification this is a C compilable code, test ongoing
 * Automatic creation of ffl files (ffl is simply the list of .gwf frame files where in the first column you have the path to the .gwf file and as a second column the starting GPS time of the frame). A script already exist, needs to be adapted.
-Line 88:
+Line 113:
-}}}