KAGRA VIS Operations Manual - Developing models in Simulink

Back to Operation Manual main page

Software running on front end computers is written in Simulink per T080135 AdvLigo CDS Realtime Code Generator (RCG) Application Developer’s Guide.

Simulink models live in the userapps repository, which by convention is checked out as /opt/rtcds/userapps. VIS models can be found at

/opt/rtcds/userapps/release/vis/${ifo}/models, e.g, /opt/rtcds/userapps/release/vis/k1/models

Only certain computers (k1boot) are configured for building models from the Simulink source - they also have the appropriate release of the rtbuild system linked to as

/opt/rtcds/${site}/${ifo}/release

and rtbuild knows to look in userapps for models.

The CDS parts library is at /opt/rtcds/rtscore/release/src/epics/simLink (note capital L in simLink).

Models currently running

The most recent rebuild of each Simulink model is archived at one of the following locations and can be accessed from outside the CDS network with ligo.org credentials:

Building models

Get advice from the DGS system manager (Miyakawa-san) on what other parts of the CDS system are likely to be affected. For example:
1. Restarting a VIS model may interfere with commissioning of the physical suspension, or the beam path that it is a part of.
2. Restarting a VIS model may interfere with ISC models that receive IPC signals from it.
3. If new channels are to be added, it may be necessary to restart the DAC, which is disruptive to people needing trend data or making measurement with DTT.
Check with users who may be working on the suspension or may be otherwise affected and confirm that it is OK to proceed.
Open a work permit and get all necessary signoffs.
Open a fresh terminal window and log into a build machine (k1boot, etc).
```
$ ssh controls@k1boot
```
Go to the build directory for the most recent release of rtcds.
```
$ cdscode # cd /opt/rtcds/kamioka/k1/rtbuild/current
```
Do a make with the model name less .mdl:
```
$ make k1vispr3
```
Diagnose and correct any errors. If the model depends on IPC from other models that have not yet been compiled, compile those at this point and try again.
Do a make install- with the model name less .mdl:
```
$ make install-k1vispr3
```
Open the DGS Overview screen, and identify the computer that runs the model.
Click on the associated button to open the GDS TP screen for the model.
Also open the main control screen for the suspension.
Let potentially affected users know that the model is about to be restarted and check that this is still OK.
Alert the operator that you are about to restart a model, and which one, so that the alarm generated in the next step does not come as a surprise.
Disable the Dolphin connection to the front end computer using SITEMAP->CDS->Dolphin (click on the associated button till it goes grey). If you don't, many other front ends in the same area are likely to crash.
Use the Guardian to take the suspension to the SAFE state.
Open a second fresh terminal window and log into the front end that is to run the model and (re)start the model from the newly compiled version.
```
$ ssh k1pr0
$ startk1vispr3
```

Note however that as of a recent RTS software version, the start... and kill... scripts do a time-consuming and not-very-useful backup before stopping the current process. If there are no other user models running on the front end that need to remain running, it is quicker to restart the entire machine:

```
$ ssh k1pr0
$ sudo shutdown -r now
```

Alternatively the front end can be powered down and brought back up using the web interface. See ../Remote#webserver. This is a slightly more thorough power down than with shutdown -r and can sometimes be effective against odd hardware/software states.

Check that the GDS TP screen and the main MEDM screen for the suspension go white briefly and then recovers after a few tens of seconds.
Check that the line for the model on the CDS Detailed Overview screen goes white briefly and then recovers (this normally takes a few tens of seconds longer to come back than the MEDM screen for the suspension).
Check that the terminal window says that the safe.snap file has been loaded. Also check that appropriate gains and matrix values have been initialized on the MEDM screen for the suspension.
If the safe.snap does not load, try loading it via the Restore safe.snap command in the popup on the MEDM screen for the suspension.
If the safe.snap file still doesn't load, or has problems, open a third fresh terminal window (on one of the ops workstations) and, without logging onto anywhere else, load a more recent BURT snapshot, e.g., the most recent hourly backup:
```
$ burtrestore k1vispr3epics 09:00
```
If you have added new channels to the model, check with DGS staff (Miyakawa-san et al.) open yet another terminal window and restart the DAC:
```
$ ssh k1dc0
$ ps -ef | grep daqd
$ kill nnnn # nnnn=process number of the daqd.dc process as revealed by the previous command
```

If you had to restart the DAQ, also restart mxstream:

$ ssh k1boot
$ /home/controls/admin_script/restart_mxstream.sh

If everything looks OK on the MEDM screen for the suspension, make sure the MASTER switch is off, reenable the watchdogs and use the Guardian to go to a suitable state, e.g., DAMPED.
On the DGS Detailed Overview screen, press the [DR] (Diag Reset) button to clear any transitory errors, and debug any that persist.
When all looks OK, close the work permit and do an alog describing the changes.

Inserting SVN revision numbers

Note that Simulink models can have a macro string in a text box which inserts the SVN revision number when the model is compiled, so it can be displayed later and serve as documentation of which revision was most recently compiled (this is usually the one running). See InsertingSvnVersionStringIntoSimulinkModels.

KAGRA/Subgroups/VIS/OpsManual/Simulink (last edited 2019-04-26 09:40:21 by MarkBarton)