Differences between revisions 13 and 24 (spanning 11 versions)
⇤ ← Revision 13 as of 2018-08-03 03:31:51 →
Size: 2265
Editor: SadakazuHaino
Comment:
← Revision 24 as of 2018-08-18 11:08:59 → ⇥
Size: 4179
Editor: SadakazuHaino
Comment:
Deletions are marked like this. Additions are marked like this.
Line 2: Line 2:
 * For KAGRA CAL there are (at least) two shared system where gstlal can run and KAGRA data is accessible
   * Kashiwa Tier-0 data center
   * Kamioka surface building		  * For KAGRA CAL, (at least) three shared system are being prepared where gstlal can run and KAGRA data is accessible
   * [[#Tunnel|Kamioka tunnel]]
   * [[#Surface|Kamioka surface building]]
   * [[#Tier-0|Kashiwa Tier-0 data center]]
 * [[#LAL|LAL]] and [[#GstLAL|GstLAL]] are installed in all the system and GDS will be installed into the [[#Tunnel|Kamioka tunnel]] system
 * [[https://gwdoc.icrr.u-tokyo.ac.jp/cgi-bin/private/DocDB/ShowDocument?docid=8710|Development plan of CAL machine and infrastructure for the low-latency h(t) reconstruction]]
Line 6: Line 9:
<<Anchor(Tunnel)>>
== Kamioka tunnel (low latency) ==
 * Scientific Linux 7 (SL7)
 * This will be the dedicated server for low-latency h(t) reconstruction
 * The selection of the hardware model is on-going
 * Prototype machine is being prepared
   * Installation of SL7 with "General purpose system"
   * Additional RPMs are identified
   * [[https://wiki.ligo.org/DASWG/LALSuite|Installation of LAL suite]] and [[https://wiki.ligo.org/DASWG/GstLAL|GstLAL]] with RPM at [[http://software.ligo.org/lscsoft/scientific/7/x86_64/production/|SL7-lscsoft]]
   * A few RPMs (healpy-1.10.3, python-seaborn-0.9.0, gst-python-1.4.0) were prepared by ourselves
   * Due to the recent update of [[https://dl.fedoraproject.org/pub/epel/7/x86_64/Packages/r/|root-related packages]] from 6.12 to 6.14 we need to rebuild GDS RPMs from [[http://software.ligo.org/lscsoft/scientific/7/SRPMS/production/gds-2.18.7-1.el7.src.rpm|SRPM]]
 * Useful links and documentations
   * [[https://labcit.ligo.caltech.edu/~jzweizig/dmt/DMTProject/|LIGO DMT project]]
   * [[https://labcit.ligo.caltech.edu/~jzweizig/dmt/DMTProject/DMT-DD.html|LIGO DMT Data Distribution subsystem]]
   * [[https://labcit.ligo.caltech.edu/~jzweizig/dmt/DMTProject/DMT-Hardware.html|LIGO Observatory Hardware configuration]]
   * [[https://llocds.ligo-la.caltech.edu/cdsindex.html|LIGO Livingston Observatory CDS website]]


<<Anchor(Surface)>>
== Kamioka surface building ==
 * Red Hat Enterprise 6
 * Cluster's overall first priority is given for the DMG data transfer. Thus, some of hosts is not allowed to login/use for non-DMG usage and data traffic should not be overloaded.
 * CAL subgroup can use one or two calculation host with first priority.
 * This machine can be directly connected to DMG system for the frame file based h(t) reconstruction
 * The output files can be transferred with the DMG standard pipeline
 * Many libraries have been installed from the source at shared home to have similar environment at Kashiwa Tier-0 center
 * Following the DMG policy, access to the shared account is limited with SSH public key only (no shared password)
 * If you want to use this machine with the shared user account, please contact DMG chief (for VPN) and S.Haino (for SSH public key)

<<Anchor(Tier-0)>>
Line 14: Line 47:
== Kamioka surface building ==
 * Red Hat Enterprise 6
 * Cluster's overall first priority is given for the DMG data transfer. Thus, some of hosts is not allowed to login/use for non-DMG usage and data traffic should not be overloaded.
 * CAL subgroup can use one or two calculation host with first priority.
 * This machine can be directly connected to KAGRA online system for the low-latency h(t) pipeline
 * The output files can be transferred with the DMG standard pipeline
 * Many libraries have been installed from the source at shared home to have similar environment at Kashiwa Tier-0 center
 * Following the DMG policy, access to the shared account is limited with SSH public key only (no shared password)
 * If you want to use this machine with the shared user account, please contact DMG chief (for VPN) and S.Haino (for SSH public key)
 <<Anchor(LAL)>>
Line 41: Line 65:
<<Anchor(GstLAL)>>
Line 42: Line 67:
 * [[https://wiki.ligo.org/DASWG/GstLAL|LIGO wiki]]

gstlal-calibration installation

Kamioka tunnel (low latency)

Kamioka surface building

  • Red Hat Enterprise 6
  • Cluster's overall first priority is given for the DMG data transfer. Thus, some of hosts is not allowed to login/use for non-DMG usage and data traffic should not be overloaded.
  • CAL subgroup can use one or two calculation host with first priority.
  • This machine can be directly connected to DMG system for the frame file based h(t) reconstruction
  • The output files can be transferred with the DMG standard pipeline
  • Many libraries have been installed from the source at shared home to have similar environment at Kashiwa Tier-0 center
  • Following the DMG policy, access to the shared account is limited with SSH public key only (no shared password)
  • If you want to use this machine with the shared user account, please contact DMG chief (for VPN) and S.Haino (for SSH public key)

Kashiwa Tier-0 data center

  • CentOS-7 (equivalent to SL7 at LIGO)
  • LAL and GstLAL are installed as rpm (should be same as LIGO)
  • The priority of CPU usage is given to DAS sub-system for the search pipeline development
  • This machine can be used only for offline analysis
  • For CAL, it is recommended to use this machine only to test the codes at the similar environment as DAS (and maybe LIGO)

  • It is known that numpy.save doesn't work in numpy 1.14 with python 2.7.5 https://github.com/numpy/numpy/issues/10348 use numpy 1.13 instead

LAL version

  • Same version is installed both in Kashiwa and Kamioka 

lalapps_version
  • LAL: 6.18.0 (CLEAN ea18...)
  • LALFrame: 1.4.3 (CLEAN 5224...)
  • LALMetaIO: 1.3.1 (CLEAN 7619...)
  • LALXML: 1.2.4 (CLEAN 5fd4...)
  • LALSimulation: 1.7.3 (CLEAN 9eb4...)
  • LALBurst: 1.4.4 (CLEAN ceb2...)
  • LALInspiral: 1.7.7 (CLEAN a00b...)
  • LALPulsar: 1.16.0 (CLEAN b8da...)
  • LALInference: 1.9.4 (CLEAN cbb2...)
  • LALStochastic: 1.1.20 (CLEAN 983b...)
  • LALApps: 6.21.0 (CLEAN ba16...)

GstLAL version

  • LIGO wiki

  • gstlal-1.3.0
  • gstlal-ugly-1.4.1
  • gstlal-calibration-1.1.9
  • gstlal-inspiral-1.4.2

KAGRA/Subgroups/CAL/WG/offline/gstlal (last edited 2018-08-18 11:08:59 by SadakazuHaino)