Differences between revisions 1 and 12 (spanning 11 versions)
Revision 1 as of 2017-02-21 16:14:21
Size: 8676
Editor: MarkBarton
Comment:
Revision 12 as of 2019-04-16 11:23:30
Size: 9524
Editor: MarkBarton
Comment:
Deletions are marked like this. Additions are marked like this.
Line 2: Line 2:

Cut and pasted from PDF of LIGO version - needs major editing!
Line 6: Line 9:
burtgooey which invokes two command line tools, burtwb and burtrb. The burtrb tool needs a BURT request
file (.req) specifying a list of channels to be backed up and generates a BURT snapshot file (.snap) containing a list of
`burtgooey` which invokes two command line tools, `burtwb` and `burtrb`. The `burtrb` tool needs a BURT request
file (`.req`) specifying a list of channels to be backed up and generates a BURT snapshot file (`.snap`) containing a list of
Line 9: Line 12:
autoBurt.req file specifying a default list of channels to be backed up (not all will be of interest). A background task
backs up all models according to their autoBurt.req files at regular intervals (currently hourly), but additional
snapshots can be taken manually. The kill scripts which shut down models in an orderly fashion (e.g., killh1susmc2
for H1:MC2) do a backup first. (Startup scripts, e.g., starth1susmc2 for H1:MC2, invoke the corresponding kill script
`autoBurt.req` file specifying a default list of channels to be backed up (not all will be of interest). A background task
backs up all models according to their `autoBurt.req` files at regular intervals (currently hourly), but additional
snapshots can be taken manually. The kill scripts which shut down models in an orderly fashion (e.g., killk1visbs
for K1:BS) do a backup first. (Startup scripts, e.g., startk1visbs for K1:BS, invoke the corresponding kill script
Line 15: Line 18:
Each model should have a safe.snap file representing a safe state in which the suspension or other system is least likely
to be a danger to itself or the rest of the system. Each model is automatically restored to the safe.snap state after a reboot
Each model should have a `safe.snap` file representing a safe state in which the suspension or other system is least likely
to be a danger to itself or the rest of the system. Each model is automatically restored to the `safe.snap` state after a reboot
Line 20: Line 23:
The Guardian system will use additional files similar to safe.snap to define standard states. The lock.snap files The Guardian system will use additional files similar to safe.snap to define standard states. The `lock.snap` files
Line 26: Line 29:
/opt/rtcds/${site}/${ifo}/target/ , e.g., /opt/rtcds/lho/h1/target/ .
`
/opt/rtcds/${site}/${ifo}/target/` , e.g., `/opt/rtcds/kamioka/k1/target/` .
Line 28: Line 33:
/opt/rtcds/${site}/${ifo}/target/${model}/${model}epics/, e.g., for H1:MC2,
/opt/rtcds/lho/h1/target/h1susmc2/h1susmc2epics/ .

The autoBurt.req file is
/opt/rtcds/${site}/${ifo}/target/${model}/${model}epics/autoBurt.req , e.g. for
H1:MC2, /opt/rtcds/lho/h1/target/h1susmc2/h1susmc2epics/autoBurt.req.

`
/opt/rtcds/${site}/${ifo}/target/${model}/${model}epics/`, e.g.,
`/opt/rtcds/kamioka/k1/target/k1visbst/k1visbstepics/` and `/opt/rtcds/kamioka/k1/target/k1visbsp/k1visbspepics/`
for K1:BS
(split top and payload models).

The `autoBurt.req` file is

`
/opt/rtcds/${site}/${ifo}/target/${model}/${model}epics/autoBurt.req` , e.g. for
K1:BS, `/opt/rtcds/kamioka/k1/target/k1visbst/k1visbsepicst/autoBurt.req`.
Line 34: Line 44:
/opt/rtcds/${site}/${ifo}/target/${model}/${model}epics/burt/safe.snap e.g., for
H1:MC2, /opt/rtcds/lho/h1/target/h1susmc2/h1susmc2epics/burt/safe.snap .
However by convention, all safe.snap files are now symbolic links into the SVN-controlled area at
/opt/rtcds/userapps/release/sus/${ifo}/burtfiles/, e.g,
/opt/rtcds/lho/h1/target/h1susmc2/h1susmc2epics/burt/safe.snap ->
/opt/rtcds/userapps/release/sus/burtfiles/h1susmc2_safe.snap
Because BURT (burtgooey etc) is not smart enough to follow a symbolic link and write to the location pointed to

`/opt/rtcds/${site}/${ifo}/target/${model}/${model}epics/burt/safe.snap` e.g., for
H1:BS, `/opt/rtcds/kamioka/k1/target/k1visbs/k1visbsepics/burt/safe.snap` .

At LIGO, the convention is that safe.snap files are symbolic links into the SVN-controlled area at

`/opt/rtcds/userapps/release/vis/${ifo}/burtfiles/`, e.g,
`/opt/rtcds/kamioka/k1/target/k1visbst/k1visbstepics/burt/safe.snap` ->
`/opt/rtcds/userapps/release/vis/burtfiles/k1visbst_safe.snap`.

'''However this has not been completely implemented at KAGRA (yet). Type B will be done the week of 2019-04-15.'''

If/when it is, because BURT (`burtgooey` etc) is not smart enough to follow a symbolic link and write to the location pointed to
Line 43: Line 59:
Snapshot files generated by kill scripts are at
/opt/rtcds/${site}/${ifo}/target/${model}/${model}epics/burt , e.g., for H1:MC2,
/opt/rtcds/lho/h1/target/h1susmc2/h1susmc2epics/burt/ .

with names like h1susmc2_burt_121204_124643.snap (i.e., with a yymmdd_hhmmss date/time stamp).
Hourly backups live at /ligo/cds/${site}/${ifo}/burt organized by year, month, day, time, and model, e.g.,
/ligo/cds/lho/h1/burt/2013/01/04/09:00/h1susmc2epics.snap.

Snapshot files generated by kill scripts (generated by the build system) are at

`
/opt/rtcds/${site}/${ifo}/target/${model}/${model}epics/burt` , e.g., for H1:BS,
`/opt/rtcds/kamioka/k1/target/k1visbst/k1visbstepics/burt/` and `/opt/rtcds/kamioka/k1/target/k1visbsp/k1visbspepics/burt/`,

with names like `k1visbst_burt_121204_124643.snap` (i.e., with a yymmdd_hhmmss date/time stamp).

Hourly backups live at `/kagra/burt/autoburt/snapshots` organized by year, month, day, time, and model, e.g.,
`/kagra/burt/autoburt/snapshots/2017/Feb/20/18:40/k1visbst.snap` .
Line 51: Line 72:
Line 52: Line 74:
There is a Matlab function save_safe_snap() to make new safe.snap files conveniently. (Currently there is no
equivalent save_lock_snap(), but stay tuned.) It saves the current state of a suspension, puts it into the safe state,

'''Need to recreate/adapt this script! for KAGRA'''

There is a Matlab function `save_safe_snap()` to make new safe.snap files conveniently. (Currently there is no
equivalent `save_lock_snap()`, but stay tuned.) It saves the current state of a suspension, puts it into the safe state,
Line 56: Line 81:
1. Run Matlab and navigate to the `^trunk/Common/MatlabTools directory.  1. Run Matlab and navigate to the `^trunk/Common/MatlabTools directory.
{{{
Line 58: Line 84:
> cd /ligo/svncommon/SusSVN/sus/trunk/Common/MatlabTools
2. Invoke save_safe_snap() with the desired ifo and suspension.
> save_safe_snap('H1','ITMY')
3. Copy and paste the svn command that is printed by the function and paste it into the Matlab window. (Note the use of
> cd /kagra/svncommon/SusSVN/vis/trunk/Common/MatlabTools
}}}
 1
. Invoke save_safe_snap() with the desired ifo and suspension.
{{{
> save_safe_snap('K1','BS')
}}}
 1
. Copy and paste the svn command that is printed by the function and paste it into the Matlab window. (Note the use of
Line 63: Line 92:
{{{
Line 64: Line 94:
/opt/rtcds/userapps/release/sus/h1/burtfiles/h1susitmy_safe.snap /opt/rtcds/userapps/release/vis/k1/burtfiles/k1visitmy_safe.snap
}}}
Line 66: Line 98:
1. Put the suspension in the desired state. For safe.snap files, check the following:
[DAMP] filter outputs should be ON.
[MASTER_SWITCH] should be OFF.
[TEST] filter outputs should be OFF.
[LOCK] filter outputs should be OFF.
[OPTICALIGN] filter outputs should be OFF.
 1. Put the suspension in the desired state. For safe.snap files, check the following:
 * [DAMP] filter outputs should be ON.
 * [MASTER_SWITCH] should be OFF.
 * [TEST] filter outputs should be OFF.
 * [LOCK] filter outputs should be OFF.
 * [OPTICALIGN] filter outputs should be OFF.
Line 73: Line 105:
[DAMP] filter outputs should be ON.
[MASTER_SWITCH] should be ON.
[TEST] filter outputs should be OFF.
[LOCK] filter outputs should be ???.
[OPTICALIGN] filter outputs should be ON.
2. Open burtgooey in the command window:
 * [DAMP] filter outputs should be ON.
 * [MASTER_SWITCH] should be ON.
 * [TEST] filter outputs should be OFF.
 * [LOCK] filter outputs should be ???.
 * [OPTICALIGN] filter outputs should be ON.
 1.#2 Open burtgooey in the command window:
{{{
Line 80: Line 113:
3. Click [Backup]. In the new window, click [Request Files]. The Request File Selector dialog appears.
4. Navigate to the /opt/rtcds/${site}/${ifo}/target/ area for your site and ifo, and go down two
levels to locate the autoBurt.req file for your suspension, e.g.,
/opt/rtcds/lho/h1/target/h1susmc2/h1susmc2epics/ for H1 MC2. Click on
autoBurt.req, then [OK] and [Cancel] (the UI allows for adding more than one file, hence the need for
[Cancel]).
5. In the [Snapshot Filename] field, enter in THE ENTIRE PATH and desired filename for the snapshot. For
}}}
 1.#
3 Click `[Backup]`. In the new window, click `[Request Files]`. The Request File Selector dialog appears.
 1.#4 Navigate to the `/opt/rtcds/${site}/${ifo}/target/` area for your site and ifo, and go down two
levels to locate the `autoBurt.req` file for your suspension, e.g.,
`/opt/rtcds/kamioka/k1/target/k1visbs/k1visbsepics/` for H1:BS. Click on
`
autoBurt.req`, then `[OK]` and `[Cancel]` (the UI allows for adding more than one file, hence the need for
`[Cancel]`).
 1.#5 In the `[Snapshot Filename]` field, enter in THE ENTIRE PATH and desired filename for the snapshot. For
Line 89: Line 123:
/opt/rtcds/userapps/release/sus/h1/burtfiles/h1susmc2_safe.snap.
6. Click the [Backup] then the [Done] buttons.
7. If the snapshot is in the userapps SVN directory sure to commit the newly-written file:
$ cd /opt/rtcds/userapps/release/sus/h1/burtfiles/
$ svn commit -m "New safe.snap for MC2"
`/opt/rtcds/userapps/release/vis/k1/burtfiles/k1visbs_safe.snap.`
 1.#
6 Click the [Backup] then the [Done] buttons.
 1.#7 If the snapshot is in the userapps SVN directory sure to commit the newly-written file:
{{{
$ cd /opt/rtcds/userapps/release/vis/k1/burtfiles/
$ svn commit -m "New safe.snap for BS"
}}}
Line 96: Line 132:
1. Open the SITEMAP.
2. Open the screen for the suspension.
3. Click on the button which opens the GDS TP screen for the user model, e.g., for H1:MC2, [H1SUSMC2 GDS TP].
4. Click on the [BURT] button.
 1. Open the SITEMAP.
 2. Open the screen for the suspension.
 3. Click on the button which opens the GDS TP screen for the user model, e.g., for H1:BS, [H1SUSBS GDS TP].
 4. Click on the [BURT] button.
Line 101: Line 137:
1. Open burtgooey in the command window:  1. Open burtgooey in the command window:
Line 103: Line 139:
2. Click on [Restore]. The Restore window appears.
3. Click on [Snapshot Files ...]. The Snapshot File Selector dialog appears.
4. Navigate to the directory with the backup file
 2. Click on [Restore]. The Restore window appears.
 3. Click on [Snapshot Files ...]. The Snapshot File Selector dialog appears.
 4. Navigate to the directory with the backup file
Line 107: Line 143:
/opt/rtcds/${site}/${ifo}/target/${model}/${model}epics/burt/ e.g., for H1
MC2
, /opt/rtcds/lho/h1/target/h1susmc2/h1susmc2epics/burt/.
`/opt/rtcds/${site}/${ifo}/target/${model}/${model}epics/burt/` e.g., for K1:BS, `/opt/rtcds/kamioka/k1/target/k1visbs/k1visbsepics/burt/`.
Line 110: Line 145:
/ligo/cds/${site}/${ifo}/burt/${YYYY}/${MM}/${DD}/${hh}:{mm}/
5. Identify and a select a file from a known good time, then click [OK] and [Cancel] (the UI allows for adding more
than one file, hence the need for [Cancel]).
Kill script snapshots have a date/time in yymmdd_hhmmss format at the end of their name, e.g.,
h1susmc2_burt_121204_124643.snap.
Hourly backups are called ${model}epics.snap, e.g, for H1:MC2, h1susmcsepics.snap.
6. Check that the file selected in the previous step is the only one in the list. If not clear unwanted files from the list with
[Remove Selected].
7. Click [Restore]. After a few seconds the indicator immediately to the right should turn green and display "OK". If
`/kagra/burt/autoburt/snapshots/${YYYY}/${Mon}/${DD}/${hh}:{mm}/`
 5. Identify and a select a file from a known good time, then click `[OK]` and `[Cancel]` (the UI allows for adding more
than one file, hence the need for `[Cancel]`).
Kill script snapshots have a date/time in `yymmdd_hhmmss` format at the end of their name, e.g.,
`k1visbs_burt_121204_124643.snap`.
Hourly backups are called `${model}.snap`, e.g, for H1:BS, k1visbs.snap.
 6.
Check that the file selected in the previous step is the only one in the list. If not clear unwanted files from the list with
`[Remove Selected]`.
 7. Click `[Restore]`. After a few seconds the indicator immediately to the right should turn green and display "OK". If
Line 122: Line 157:
'''This worked at LIGO but doesn't seem to be implemented at KAGRA - try navigating to the snapshot with burtgooey as described above.'''
Line 124: Line 160:
$ burtrestore h1susmc2epics 09:00 % restore to 9 o'clock this morning
$ burtrestore h1susmc2epics 09:00 02 % restore to 9 o'clock on the 2nd of this month
$ burtrestore h1susmc2epics 09:00 02 01 % restore to 9 o'clock on the 2nd of January this
{{{
$ burtrestore k1visbsepics 09:00 % restore to 9 o'clock this morning
$ burtrestore k1visbsepics 09:00 02 % restore to 9 o'clock on the 2nd of this month
$ burtrestore k1visbsepics 09:00 02 01 % restore to 9 o'clock on the 2nd of January this
Line 128: Line 165:
$ burtrestore h1susmc2epics 09:00 02 01 2012 % restore to 9 o'clock on 1/2/12 $ burtrestore k1visbsepics 09:00 02 01 2012 % restore to 9 o'clock on 1/2/12
}}}

KAGRA VIS Operations Manual - Info on BURT

Cut and pasted from PDF of LIGO version - needs major editing!

Back to Operation Manual main page

Background

BURT is a tool for backing up and restoring some or all of the EPICS state of one or more models. There is a GUI interface, burtgooey which invokes two command line tools, burtwb and burtrb. The burtrb tool needs a BURT request file (.req) specifying a list of channels to be backed up and generates a BURT snapshot file (.snap) containing a list of channel names and saved values. The burtwb tool then loads the values back into the model. Each model should have a autoBurt.req file specifying a default list of channels to be backed up (not all will be of interest). A background task backs up all models according to their autoBurt.req files at regular intervals (currently hourly), but additional snapshots can be taken manually. The kill scripts which shut down models in an orderly fashion (e.g., killk1visbs for K1:BS) do a backup first. (Startup scripts, e.g., startk1visbs for K1:BS, invoke the corresponding kill script first and thus also trigger a backup.)

Each model should have a safe.snap file representing a safe state in which the suspension or other system is least likely to be a danger to itself or the rest of the system. Each model is automatically restored to the safe.snap state after a reboot of the front-end computer. The "safe" state is currently defined as having the damping filters ON but the master switch and most other sources of excitation OFF.

The Guardian system will use additional files similar to safe.snap to define standard states. The lock.snap files define the desired state for locking the cavity of which the suspension is a part, with the damping, master switch, alignment offsets and locking filters all on. Need to check this!!!

Directory structure

Many BURT related utilities put or expect to find BURT-related files at

/opt/rtcds/${site}/${ifo}/target/ , e.g., /opt/rtcds/kamioka/k1/target/ .

These are organized by the name of the Simulink model controlling the suspension:

/opt/rtcds/${site}/${ifo}/target/${model}/${model}epics/, e.g., /opt/rtcds/kamioka/k1/target/k1visbst/k1visbstepics/ and /opt/rtcds/kamioka/k1/target/k1visbsp/k1visbspepics/ for K1:BS (split top and payload models).

The autoBurt.req file is

/opt/rtcds/${site}/${ifo}/target/${model}/${model}epics/autoBurt.req , e.g. for K1:BS, /opt/rtcds/kamioka/k1/target/k1visbst/k1visbsepicst/autoBurt.req.

The safe.snap file used when a model is started is

/opt/rtcds/${site}/${ifo}/target/${model}/${model}epics/burt/safe.snap e.g., for H1:BS, /opt/rtcds/kamioka/k1/target/k1visbs/k1visbsepics/burt/safe.snap .

At LIGO, the convention is that safe.snap files are symbolic links into the SVN-controlled area at

/opt/rtcds/userapps/release/vis/${ifo}/burtfiles/, e.g, /opt/rtcds/kamioka/k1/target/k1visbst/k1visbstepics/burt/safe.snap -> /opt/rtcds/userapps/release/vis/burtfiles/k1visbst_safe.snap.

However this has not been completely implemented at KAGRA (yet). Type B will be done the week of 2019-04-15.

If/when it is, because BURT (burtgooey etc) is not smart enough to follow a symbolic link and write to the location pointed to (rather, it replaces the link with a plain data file), it is important always to give it a explicit path into the userapps area so as not to break the link.

Snapshot files generated by kill scripts (generated by the build system) are at

/opt/rtcds/${site}/${ifo}/target/${model}/${model}epics/burt , e.g., for H1:BS, /opt/rtcds/kamioka/k1/target/k1visbst/k1visbstepics/burt/ and /opt/rtcds/kamioka/k1/target/k1visbsp/k1visbspepics/burt/,

with names like k1visbst_burt_121204_124643.snap (i.e., with a yymmdd_hhmmss date/time stamp).

Hourly backups live at /kagra/burt/autoburt/snapshots organized by year, month, day, time, and model, e.g., /kagra/burt/autoburt/snapshots/2017/Feb/20/18:40/k1visbst.snap .

Saving snapshots

The following sections explain several different tools for making snapshots.

Updating a safe.snap file via Matlab

Need to recreate/adapt this script! for KAGRA

There is a Matlab function save_safe_snap() to make new safe.snap files conveniently. (Currently there is no equivalent save_lock_snap(), but stay tuned.) It saves the current state of a suspension, puts it into the safe state, saves the state again with the appropriate name in the userapps working copy area, restores the original state and prints an svn command that can be used to commit the new file to the userapps repository.

  1. Run Matlab and navigate to the `^trunk/Common/MatlabTools directory.

$ matlab&
> cd /kagra/svncommon/SusSVN/vis/trunk/Common/MatlabTools
  1. Invoke save_safe_snap() with the desired ifo and suspension.

> save_safe_snap('K1','BS')
  1. Copy and paste the svn command that is printed by the function and paste it into the Matlab window. (Note the use of

! to execute shell commands within Matlab - omit it if you prefer to use a regular terminal window.)

> !svn commit -m "New safe.snap file for H1:ITMY"
/opt/rtcds/userapps/release/vis/k1/burtfiles/k1visitmy_safe.snap

Creating a snapshot manually

  1. Put the suspension in the desired state. For safe.snap files, check the following:
  2. [DAMP] filter outputs should be ON.
  3. [MASTER_SWITCH] should be OFF.
  4. [TEST] filter outputs should be OFF.
  5. [LOCK] filter outputs should be OFF.
  6. [OPTICALIGN] filter outputs should be OFF.

For lock.snap files check the following [Need to check this!]:

  • [DAMP] filter outputs should be ON.
  • [MASTER_SWITCH] should be ON.
  • [TEST] filter outputs should be OFF.
  • [LOCK] filter outputs should be ???.
  • [OPTICALIGN] filter outputs should be ON.
  • Open burtgooey in the command window:

$ burtgooey &
  1. Click [Backup]. In the new window, click [Request Files]. The Request File Selector dialog appears.

  2. Navigate to the /opt/rtcds/${site}/${ifo}/target/ area for your site and ifo, and go down two

levels to locate the autoBurt.req file for your suspension, e.g., /opt/rtcds/kamioka/k1/target/k1visbs/k1visbsepics/ for H1:BS. Click on autoBurt.req, then [OK] and [Cancel] (the UI allows for adding more than one file, hence the need for [Cancel]).

  1. In the [Snapshot Filename] field, enter in THE ENTIRE PATH and desired filename for the snapshot. For

safe.snap and lock.snap files, this should be in the part of the userapps directory containing the suspension's individual safe.snap file, e.g., /opt/rtcds/userapps/release/vis/k1/burtfiles/k1visbs_safe.snap.

  1. Click the [Backup] then the [Done] buttons.
  2. If the snapshot is in the userapps SVN directory sure to commit the newly-written file:

$ cd /opt/rtcds/userapps/release/vis/k1/burtfiles/
$ svn commit -m "New safe.snap for BS"

Restoring from snapshots

Restoring to a snap.snap file with the GDS_TP screen

  1. Open the SITEMAP.
  2. Open the screen for the suspension.
  3. Click on the button which opens the GDS TP screen for the user model, e.g., for H1:BS, [H1SUSBS GDS TP].
  4. Click on the [BURT] button.

Restoring from an arbitrary saved state with the BURT GUI

  1. Open burtgooey in the command window:

$ burtgooey &

  1. Click on [Restore]. The Restore window appears.
  2. Click on [Snapshot Files ...]. The Snapshot File Selector dialog appears.
  3. Navigate to the directory with the backup file

For kill script backups, navigate to /opt/rtcds/${site}/${ifo}/target/${model}/${model}epics/burt/ e.g., for K1:BS, /opt/rtcds/kamioka/k1/target/k1visbs/k1visbsepics/burt/. For hourly backups, navigate to /kagra/burt/autoburt/snapshots/${YYYY}/${Mon}/${DD}/${hh}:{mm}/

  1. Identify and a select a file from a known good time, then click [OK] and [Cancel] (the UI allows for adding more

than one file, hence the need for [Cancel]). Kill script snapshots have a date/time in yymmdd_hhmmss format at the end of their name, e.g., k1visbs_burt_121204_124643.snap. Hourly backups are called ${model}.snap, e.g, for H1:BS, k1visbs.snap.

  1. Check that the file selected in the previous step is the only one in the list. If not clear unwanted files from the list with

[Remove Selected].

  1. Click [Restore]. After a few seconds the indicator immediately to the right should turn green and display "OK". If

there is a problem (the indicator turns red and says "Not OK"), in which clear the file from the list and choose a better one.

Restoring from an hourly backup

This worked at LIGO but doesn't seem to be implemented at KAGRA - try navigating to the snapshot with burtgooey as described above. The burtrestore utility is a convenient way of restoring to an hourly backup. It takes the hourly snapshot file name for a particular model (i.e., the name of the model plus "epics") and as many of time, day, month and year as are necessary:

$ burtrestore k1visbsepics 09:00 % restore to 9 o'clock this morning
$ burtrestore k1visbsepics 09:00 02 % restore to 9 o'clock on the 2nd of this month
$ burtrestore k1visbsepics 09:00 02 01 % restore to 9 o'clock on the 2nd of January this
year
$ burtrestore k1visbsepics 09:00 02 01 2012 % restore to 9 o'clock on 1/2/12

KAGRA/Subgroups/VIS/OpsManual/BURT (last edited 2019-04-16 11:23:30 by MarkBarton)