Setup CDS software with front-end

1. change controls UID to 1001, and controls GID to 1001 because all of the CDS files owned by controls account with UID=1001 and GID=1001. see https://awiki.ligo-wa.caltech.edu/aLIGO/CdsUbuntuDesktopBuild

2. sudo chown 1001 /frames
   sudo chown -R 1001:1001 /frames/*
   sudo chown 1001 /opt
   sudo chown 1001 /opt/apps
   sudo chown 1001 /opt/rtcds
   sudo chown -R 1001:1001 /opt/apps/*
   sudo chown -R 1001:1001 /opt/rtcds/*

3. sudo apt-get install tcsh

4. change default shell to tcsh

5. sudo apt-get install g++

6. Install Matlab with simulink at /opt/apps/linux64/matlab

7. install NFS client as

   sudo apt-get install nfs-common

8. add in /etc/fstab

   standalone:/opt/rtcds   /opt/rtcds      nfs     rw,bg,soft       0       0
   standalone:/opt/rtapps  /opt/rtapps     nfs     rw,bg,soft       0       0
   standalone:/opt/cdscfg  /opt/cdscfg     nfs     rw,bg,soft       0       0

9. add in /etc/hosts

   131.215.113.100 standalone
   131.215.113.110 desktop1

10. add /etc/host on standalone(master)

    131.215.113.100       standalone
    131.215.113.110       desktop1

11. sudo reboot 

12. Add slow channel file name at /opt/rtcds/tst/x1/target/fb/master

    /opt/rtcds/tst/x1/chans/daq/X1EDCU.ini

    and add slow channels at /opt/rtcds/tst/x1/chans/daq/X1EDCU.ini

    [default]
    dcuid=4
    datarate=16
    gain=1.0
    acquire=1
    ifoid=0
    datatype=4
    units=units
    slope=1.0
    offset=0
    
    
    [X1:FEC-24_ADC_WAIT]
    [X1:X14-ADC_FILTER_2_INMON]

13. Change /opt/rtcds/tst/x1/target/fb/start_daqd

    exec su controls -c 'nice -19 /opt/rtcds/tst/x1/target/fb/daqd -c /opt/rtcds/tst
    /x1/target/fb/daqdrc' > /opt/rtcds/tst/x1/target/fb/logs/daqd.log.$$ 2>&1 

    and /opt/rtcds/tst/x1/target/fb/start_nds

    exec su controls -c '/opt/rtcds/tst/x1/target/fb/nds /opt/rtcds/tst/x1/target/fb
    /pipe' > /opt/rtcds/tst/x1/target/fb/logs/nds.log.$$ 2>&1 

    to write framedata as controls owned.

14. To run daqconfig properly

    sudo emerge tk

15. Run daqconfig, define DAQ channels and activate them.
16. modifi /opt/rtcds/tst/x1/target/fb/daqd
    • change epics server name to "X0:DAQ-FB0_" "X1:DAQ-FB0_"

Other important files

• /etc/motd
• /etc/rtsystab
• /etc/rc.local

How to

How to change model from x1x14 to x1x30 for example

1. On 'desktop1' PC

   >cd /opt/rtcds/tst/x1/core/advLigoRTS/trunk/src/epics/simLink/
   >cp x1x14.mdl x1x30.mdl

2. build x1x30

   >cd /opt/rtcds/tst/x1/core/advLigoRTS/trunk
   >make uninstall-x1x30
   >make x1x30
   >make install-x1x30
   >make install-daq-x1x30
   >make install-screens-x1x30
   
    1. If you want to change dcuid, start matlab, open the x1x30.mdl, and change dcuid on left top box.
   
    1. {{{
   >sudo emacs /etc/stsysteb

and put the new model name or exchange existing model to the new one.

1. restart 'standalone'

   >sudo reboot

2. change /opt/rtcds/tst/x1/target/fb/master
3. define and activate daq channels using daqdconfig

How to change IP address

1. Edit /etc/conf.d/net for IP address and router address
2. Edit /etc/hosts
3. Edit /opt/rtcds/tst/x1/target/gds/param/diag
4. Edit /etc/xinet.d/diagconf

Trouble shooting

Test point, DAQ

• If diag -i is OK and everything seems OK but you cannot use test point on diaggui for expample, suspect IP address is not correct. Check /etc/conf.d/net for IP address and router address.
• If you cannot see past data on diaggui or dataviewer, define daq channel using daqconfig. Be careful that channels are activated.
• If dmesg says DAQ size is too small, increase sampling rate on DAQ channels
• If diaggui says test point is not available, suspect awg, and run /opt/rtcds/tst/x1/target/gds/startup_x1x14.cmd

• diagguiなどがうまく立ち上がらないトラブルに見舞われた。

  原因は2点。
  
  1. 余分なDCU nodeを立ち上げていた
  
  diagguiやdataviewerでtest pointが現れたり現れなかったり、不安定。いずれの場合も実際の測定はtest pointにアクセスできない等のエラーが出てしまい不可能。
  
  diag -iでみてやると本来
  
  controls@standalone ~ $ diag -i
  Diagnostics configuration:
  awg 24 0 192.168.11.100 822095896 1 192.168.11.100
  nds * * 192.168.11.100 8088 * 192.168.11.100
  tp 24 0 192.168.11.100 822091800 1 192.168.11.100
  
  となるはずなのに node 24のみでなく26やら何やらいくつか余分なものが走っていた。結局
  
  /opt/rtcds/tst/x1/target/gds/param/testpoint.par
  
  にnode 24以外のものが書いてあったのが原因で、余分なものを消したら、不安定なことはなくなった。
  
  2. awgtpmanがうまく立ち上がっていなかった
  
  startx1x14 でたちあげるとawgtpmanが途中でこける。直接ターミナル上で立ち上げると、うまく立ち上がる。立ち上がったかどうかはdiag -iでわかる。スクリプトで立ち上げた際のログ（/opt/rtcds/tst/x1/target/gds/x1x14）を見てやると、途中でログが切れている、もしくはcycle time outとかなっている。スクリプトを見てやると
  
  (cd /opt/rtcds/tst/x1/target/gds && ./startup_x1x14.cmd)
  sleep 5; sudo killall -q daqd
  
  とawgtpman立ち上げ後にsleepがはいっている。これが足りないのかと当たりをつけ、
  
  /opt/rtcds/tst/x1/core/advLigoRTS/trunk/Makefileの１０６行目のsleep 5をsleep 10にして、startx1x14を生成する際に、余分なsleepを入れてやることで解決。

diagnostic for front end

• Make it sure the timing signal is injected into ADC like 65536Hz, 0-5V square waves.

  killx1x14
  startx1x14
  sudo /sbin/init q

  controls@standalone ~ $ diag -i
  Diagnostics configuration:
  awg 24 0 131.215.113.100 822095896 1 131.215.113.100
  nds * * 131.215.113.100 8088 * 131.215.113.100
  tp 24 0 131.215.113.100 822091800 1 131.215.113.100

  controls@standalone ~ $ ps -ef | grep nds
  root      4051     1  0 11:30 ?        00:00:00 /opt/rtcds/tst/x1/target/fb/nds /opt/rtcds/tst/x1/target/fb/pipe
  controls 15479 15384  0 15:48 pts/1    00:00:00 grep --colour=auto nds
  
  controls@standalone ~ $ ps -ef | grep daqd
  root     15316     1  0 15:40 ?        00:00:02 /opt/rtcds/tst/x1/target/fb/daqd -c /opt/rtcds/tst/x1/target/fb/daqdrc
  controls 15490 15384  0 15:49 pts/1    00:00:00 grep --colour=auto daqd
  
  controls@standalone ~ $ ps -ef | grep awg 
  root     15273     1  0 15:38 pts/0    00:00:03 /opt/rtcds/tst/x1/target/gds/bin/awgtpman -s x1x14 -4
  controls 15606 15384  0 15:58 pts/1    00:00:00 grep --colour=auto awg

  controls@standalone ~ $ lsmod
  Module                  Size  Used by
  x1x14fe              4486882  0 
  mbuf                    7546  3 x1x14fe
  
  controls@standalone ~ $ more /proc/modules 
  x1x14fe 4486882 0 - Live 0xffffffffa045e000
  mbuf 7546 3 x1x14fe, Live 0xffffffffa0000000

• Check some configuration files

  controls@standalone ~ $ more /opt/rtcds/tst/x1/target/fb/master     
  /opt/rtcds/tst/x1/chans/daq/X1X14.ini
  /opt/rtcds/tst/x1/target/gds/param/tpchn_x1x14.par

  controls@standalone ~ $ more /opt/rtcds/tst/x1/target/gds/param/testpoint.par 
  [X-node24]
  hostname=standalone
  system=x1x14

• Trace from startupx1x14

  controls@standalone /opt/rtcds/tst/x1/scripts $ more /opt/rtcds/tst/x1/target/x1x14/x1x14epics/startupX1 
  #!/bin/sh
  ./x1x14epics x1x14epicsX1.cmd >& iocX1.log &
  echo x1x14epics X1 IOC Server started

  • epics

    controls@standalone /opt/rtcds/tst/x1/scripts $ more /opt/rtcds/tst/x1/target/x1x14/x1x14epics/iocX1.log 
    dbLoadDatabase "dbd/a.dbd"
    dbLoadDatabase "dbd/x1x14.dbd"
    registerRecordDeviceDriver(pdbbase)
    dbLoadRecords "db/X1/x1x141.db"
    iocInit
    Starting iocInit
    ############################################################################
    ## EPICS R3.14.9 $R3-14-9$ $2007/02/05 16:31:45$
    ## EPICS Base built Sep 29 2008
    ############################################################################
    iocInit: All initialization complete
    seq &x1x14,("ifo=X1, site=tst, sys=X14, sysnum= 24, sysfile=X1X14 ")
    SEQ Version 2.0.11: Mon Sep 29 14:55:46 2008
    cas warning: Configured TCP port was unavailable.
    cas warning: Using dynamically assigned TCP port 37288,
    cas warning: but now two or more servers share the same UDP port.
    cas warning: Depending on your IP kernel this server may not be
    cas warning: reachable with UDP unicast (a host's IP in EPICS_CA_ADDR_LIST)

  • real time core

    controls@standalone /opt/rtcds/tst/x1/scripts $ more /opt/rtcds/tst/x1/target/x1x14/scripts/startupX1rt
    sudo /sbin/insmod /opt/rtcds/tst/x1/target/x1x14/bin/x1x14fe.ko  >  /opt/rtcds/tst/x1/target/x1x14/logs/log.txt  2>& 1 &

  • awgtpman

    controls@standalone ~ $ more /opt/rtcds/tst/x1/target/gds/startup_x1x14.cmd 
    #!/bin/bash
    cd /opt/rtcds/tst/x1/target/gds; sudo /opt/rtcds/tst/x1/target/gds/bin/awgtpman 
    -s x1x14 -4 > x1x14.log 2>& 1 &
    
    controls@standalone ~ $ more /opt/rtcds/tst/x1/target/gds/x1x14.log         
    64 kHz system
    /dev/mbuf mmapped address is 0x7fbc97714000
    /dev/mbuf mmapped address is 0x7fbc97314000
    My config file is /opt/rtcds/tst/x1/target/gds/param/tpchn_x1x14.par
    IPC at 0x97314000
    Total PCI cards from the master: 2
    Model 0 = 0
    Found ADC at 0
    Model 1 = 2
    Spawn testpoint manager
    waiting to sync 26304
    remotehost = ; section= X-node0
    remotehost = ; section= X-node1
    remotehost = ; section= X-node2
    remotehost = ; section= X-node3
    remotehost = ; section= X-node4
    remotehost = ; section= X-node5
    remotehost = ; section= X-node6
    remotehost = ; section= X-node7
    remotehost = ; section= X-node8
    remotehost = ; section= X-node9
    remotehost = ; section= X-node10
    remotehost = ; section= X-node11
    remotehost = ; section= X-node12
    remotehost = ; section= X-node13
    remotehost = ; section= X-node14
    remotehost = ; section= X-node15
    remotehost = ; section= X-node16
    remotehost = ; section= X-node17
    remotehost = ; section= X-node18
    remotehost = ; section= X-node19
    remotehost = ; section= X-node20
    remotehost = ; section= X-node21
    remotehost = ; section= X-node22
    remotehost = ; section= X-node23
    remotehost = standalone; section= X-node24
    sysname = x1x14
    this is my node
    remotehost = ; section= X-node25
    remotehost = ; section= X-node26
    remotehost = ; section= X-node27
    remotehost = ; section= X-node28
    remotehost = ; section= X-node29
    remotehost = ; section= X-node30
    remotehost = ; section= X-node31
    remotehost = ; section= X-node32
    remotehost = ; section= X-node33
    remotehost = ; section= X-node34
    remotehost = ; section= X-node35
    remotehost = ; section= X-node36
    remotehost = ; section= X-node37
    remotehost = ; section= X-node38
    remotehost = ; section= X-node39
    remotehost = ; section= X-node40
    remotehost = ; section= X-node41
    remotehost = ; section= X-node42
    remotehost = ; section= X-node43
    remotehost = ; section= X-node44
    remotehost = ; section= X-node45
    remotehost = ; section= X-node46
    remotehost = ; section= X-node47
    remotehost = ; section= X-node48
    remotehost = ; section= X-node49
    remotehost = ; section= X-node50
    remotehost = ; section= X-node51
    remotehost = ; section= X-node52
    remotehost = ; section= X-node53
    remotehost = ; section= X-node54
    remotehost = ; section= X-node55
    remotehost = ; section= X-node56
    remotehost = ; section= X-node57
    remotehost = ; section= X-node58
    remotehost = ; section= X-node59
    remotehost = ; section= X-node60
    remotehost = ; section= X-node61
    remotehost = ; section= X-node62
    remotehost = ; section= X-node63
    �`b!��`b!�X1:X14-ADC_FILTER_00_EXC rmid 24
    X1:X14-ADC_FILTER_01_EXC rmid 24
    X1:X14-ADC_FILTER_02_EXC rmid 24
    X1:X14-ADC_FILTER_03_EXC rmid 24
    X1:X14-ADC_FILTER_04_EXC rmid 24
    X1:X14-ADC_FILTER_05_EXC rmid 24
    X1:X14-ADC_FILTER_06_EXC rmid 24
    X1:X14-ADC_FILTER_07_EXC rmid 24
    X1:X14-ADC_FILTER_00_IN1 rmid 24
    X1:X14-ADC_FILTER_00_IN2 rmid 24
    X1:X14-ADC_FILTER_00_OUT rmid 24
    X1:X14-ADC_FILTER_01_IN1 rmid 24
    X1:X14-ADC_FILTER_01_IN2 rmid 24
    X1:X14-ADC_FILTER_01_OUT rmid 24
    X1:X14-ADC_FILTER_02_IN1 rmid 24
    X1:X14-ADC_FILTER_02_IN2 rmid 24
    X1:X14-ADC_FILTER_02_OUT rmid 24
    X1:X14-ADC_FILTER_03_IN1 rmid 24
    X1:X14-ADC_FILTER_03_IN2 rmid 24
    X1:X14-ADC_FILTER_03_OUT rmid 24
    X1:X14-ADC_FILTER_04_IN1 rmid 24
    X1:X14-ADC_FILTER_04_IN2 rmid 24
    X1:X14-ADC_FILTER_04_OUT rmid 24
    X1:X14-ADC_FILTER_05_IN1 rmid 24
    X1:X14-ADC_FILTER_05_IN2 rmid 24
    X1:X14-ADC_FILTER_05_OUT rmid 24
    X1:X14-ADC_FILTER_06_IN1 rmid 24
    X1:X14-ADC_FILTER_06_IN2 rmid 24
    X1:X14-ADC_FILTER_06_OUT rmid 24
    X1:X14-ADC_FILTER_07_IN1 rmid 24
    X1:X14-ADC_FILTER_07_IN2 rmid 24
    X1:X14-ADC_FILTER_07_OUT rmid 24
    Channel list length for node 24 is 32
    node 0 invalid
    node 1 invalid
    node 2 invalid
    node 3 invalid
    node 4 invalid
    node 5 invalid
    node 6 invalid
    node 7 invalid
    node 8 invalid
    node 9 invalid
    node 10 invalid
    node 11 invalid
    node 12 invalid
    node 13 invalid
    node 14 invalid
    node 15 invalid
    node 16 invalid
    node 17 invalid
    node 18 invalid
    node 19 invalid
    node 20 invalid
    node 21 invalid
    node 22 invalid
    node 23 invalid
    Test point manager (31002018 / 1): node 24
    node 25 invalid
    node 26 invalid
    node 27 invalid
    node 28 invalid
    node 29 invalid
    node 30 invalid
    node 31 invalid
    node 32

  • daqd

    controls@standalone ~ $ more /opt/rtcds/tst/x1/target/fb/daqdrc
    set thread_stack_size=102400;
    set avoid_reconnect;
    set allow_tpman_connect_fail;
    set dcu_status_check=5;
    #set symm_gps_offset=2;
    set debug=0;
    set log=2;
    set zero_bad_data=0;
    set dcu_status_check=9;
    set controller_dcu=24;
    
    set master_config="/opt/rtcds/tst/x1/target/fb/master";
    configure channels begin end;
    tpconfig "/opt/rtcds/tst/x1/target/gds/param/testpoint.par";
    
    set gps_leaps = 820108813;
    
    set detector_name="CIT";
    set detector_prefix="C1";
    set detector_longitude=-90.7742403889;
    set detector_latitude=30.5628943337;
    set detector_elevation=.0;
    set detector_azimuths=1.1,4.7123889804;
    set detector_altitudes=1.0,2.0;
    set detector_midpoints=2000.0, 2000.0;
    
    set num_dirs = 10;
    set frames_per_dir=225;
    set full_frames_per_file=1;
    set full_frames_blocks_per_frame=16;
    set frame_dir="/frames/full", "C-R-", ".gwf";
    set trend_num_dirs=10;
    set trend_frames_per_dir=1440;
    set trend_frame_dir= "/frames/trend/second", "C-T-", ".gwf";
    set raw-minute-trend-dir="/frames/trend/minute_raw";
    
    set nds-jobs-dir="/opt/rtcds/tst/x1/target/fb";
    
    set minute-trend-num-dirs=10;
    set minute-trend-frames-per-dir=24;
    set minute-trend-frame-dir="/frames/trend/minute", "C-M-", ".gwf";
    
    start main 10;
    start profiler;
    
    # comment out this block to stop saving data
    /*
    start frame-saver;
    sync frame-saver;
    start trender;
    start trend-frame-saver;
    sync trend-frame-saver;
    start raw_minute_trend_saver;
    */
    
    #start frame-writer "225.225.225.1" broadcast="131.215.113.0" all;
    #sleep 5;
    
    start producer;
    start epics dcu;
    start epics server "C0:DAQ-FB0_" "C1:DAQ-FB0_";
    
    start listener 8087;
    start listener 8088 1;
    sleep 60;
    clear crc;