Differences between revisions 6 and 18 (spanning 12 versions)
⇤ ← Revision 6 as of 2018-10-18 14:39:38 →
Size: 548
Editor: shoichi.oshino
Comment:
← Revision 18 as of 2023-06-28 13:35:49 → ⇥
Size: 3801
Editor: shoichi.oshino
Comment:
Deletions are marked like this. Additions are marked like this.
Line 3: Line 3:

== Policy ==
=== Target ===
List of directories to obtain the backup.

 * k1boot
  * /opt
  * /diskless/root
 * k1nfs0
  * /export/apps
  * /export/ligo
  * /export/kagra
  * /export/users

=== Method ===

 * k1boot
  * backup to k1bck0 by using borg
 * k1nfs0
  * backup to k1bck0 by using borg
  * backup to local HDD by using rsync

== How to backup ==
=== Initialization ===
{{{
# borg init -e none /BACKUP_DIR
}}}

=== Backup from source directory ===
{{{
# borg create /BACKUP_DIR::ARCHIVE /SRC
}}}
ARCHIVE is name of archive in this time.

=== Daily ===

 * Once a day the script is executed by cron.
 * The script is located at /root/bin/backup_borg.sh
 * This script performs backup and organizing past backups.

=== Policy of prune backup ===

 1. Keep the last seven days of backups
 1. Keep the latest backup on each week, up to 8
 1. Keep the latest backup on each month, up to 36

=== Check backup status ===

 * Assign repository
 {{{
# borg list /BACKUP_DIR
 }}}
 This command displays the list of archives.
 * Assign archive
 {{{
# borg list /BACKUP_DIR::ARCHIVE
 }}}
 This command displays the list of directories and files of this archive.

=== Restore from backup ===

 1. Mount disk on k1bck0
 1. Change directory to mounted disk
 1. Start restore
 {{{
# borg extract /BACKUP_DIR::ARCHIVE
 }}}
 This command write backup data to the current directory.

=== Mount from backup ===
 1. Mount disk on k1bck0
 {{{
$ sudo brog mount /zbackup/BACKUP_DIR::DATE {MOUNT_DIR}
 }}}
 1. Unmount disk
 {{{
$ sudo brog umount {MOUNT_DIR}
 }}}

[[KAGRA/Subgroups/DGS/Projects/BackupJP|Backup Japanese]]
Line 19: Line 100:
 {{{
# pkg install tmux py36-borgbackup
 }}}
 1. Configuration
  1. SSH
  {{{
/etc/rc/conf
sshd_enable="YES"
   }}}
  1. NFS & autofs
  {{{
/etc/rc.local
nfs_client_enable="YES"
autofs_enable="YES"
  }}}
  {{{
/etc/auto_master
/- -noauto
  }}}
  {{{
/etc/fstab
# NFS
k1nfs0:/export/users /export/users nfs ro,noauto 0 0
k1nfs0:/export/ligo /export/ligo nfs ro,noauto 0 0
k1nfs0:/export/apps /export/apps nfs ro,noauto 0 0
k1nfs0:/export/kagra /export/kagra nfs ro,noauto 0 0
k1boot:/opt /export/opt nfs ro,noauto 0 0
k1boot:/diskless/root /export/root nfs ro,noauto 0 0
  }}}
 1. Create RAID
 {{{
# zpool create zbackup raidz2 ada1 ada2 ada3 da0 da1
# zpool status
  pool: zbackup
 state: ONLINE
  scan: none requested
config:
Line 20: Line 138:
        NAME STATE READ WRITE CKSUM
        zbackup ONLINE 0 0 0
          raidz2-0 ONLINE 0 0 0
            ada1 ONLINE 0 0 0
            ada2 ONLINE 0 0 0
            ada3 ONLINE 0 0 0
            da0 ONLINE 0 0 0
            da1 ONLINE 0 0 0

errors: No known data errors
 }}}
 1. Reboot
 {{{
# reboot
 }}}


== Monitor ==

=== EPICS IOC ===

 * Working on k1script1
 * Directory: /opt/rtcds/kamioka/k1/target/k1sys/

=== Script ===

 * /kagra/bin/SYS/backup_status.sh
 * Run daily at 9:00 am
Line 25: Line 171:
|| device || Serial || file system || || device || Serial || for ||
Line 27: Line 173:
|| ada0 || || zfs  ||
|| ada1 || 1SHW665Z ||<|5> raidz2 ||		 || ada0 || || system ||
|| ada1 || 1SHW665Z ||<|5> data ||

Backup System

Contents

  1. Backup System
    1. Policy
      1. Target
      2. Method
    2. How to backup
      1. Initialization
      2. Backup from source directory
      3. Daily
      4. Policy of prune backup
      5. Check backup status
      6. Restore from backup
      7. Mount from backup
    3. Maintenance
      1. OS update
    4. Installation
    5. Monitor
      1. EPICS IOC
      2. Script
    6. HDD list

Policy

Target

List of directories to obtain the backup.

  • k1boot
    • /opt
    • /diskless/root
  • k1nfs0
    • /export/apps
    • /export/ligo
    • /export/kagra
    • /export/users

Method

  • k1boot
    • backup to k1bck0 by using borg
  • k1nfs0
    • backup to k1bck0 by using borg
    • backup to local HDD by using rsync

How to backup

Initialization

# borg init -e none /BACKUP_DIR

Backup from source directory

# borg create /BACKUP_DIR::ARCHIVE /SRC

ARCHIVE is name of archive in this time.

Daily

  • Once a day the script is executed by cron.
  • The script is located at /root/bin/backup_borg.sh
  • This script performs backup and organizing past backups.

Policy of prune backup

  1. Keep the last seven days of backups
  2. Keep the latest backup on each week, up to 8
  3. Keep the latest backup on each month, up to 36

Check backup status

  • Assign repository 
    # borg list /BACKUP_DIR
    This command displays the list of archives.
  • Assign archive 
    # borg list /BACKUP_DIR::ARCHIVE
    This command displays the list of directories and files of this archive.

Restore from backup

  1. Mount disk on k1bck0
  2. Change directory to mounted disk
  3. Start restore 
    # borg extract /BACKUP_DIR::ARCHIVE
    This command write backup data to the current directory.

Mount from backup

  1. Mount disk on k1bck0 
    $ sudo brog mount /zbackup/BACKUP_DIR::DATE {MOUNT_DIR}
  2. Unmount disk 
    $ sudo brog umount {MOUNT_DIR}

Backup Japanese

Maintenance

OS update

# freebsd-update fetch
# freebsd-update install
# pkg upgrade
# reboot

Installation

  1. Install FreeBSD 11
  2. Install packages 
    # pkg install tmux py36-borgbackup
  3. Configuration
    1. SSH 
      /etc/rc/conf
sshd_enable="YES"

    2. NFS & autofs 

      /etc/rc.local
nfs_client_enable="YES"
autofs_enable="YES"
      /etc/auto_master
/-              -noauto
      /etc/fstab
# NFS
k1nfs0:/export/users    /export/users   nfs     ro,noauto       0       0
k1nfs0:/export/ligo     /export/ligo    nfs     ro,noauto       0       0
k1nfs0:/export/apps     /export/apps    nfs     ro,noauto       0       0
k1nfs0:/export/kagra    /export/kagra   nfs     ro,noauto       0       0
k1boot:/opt     /export/opt     nfs     ro,noauto       0       0
k1boot:/diskless/root     /export/root     nfs     ro,noauto       0       0
  4. Create RAID 
    # zpool create zbackup raidz2 ada1 ada2 ada3 da0 da1
# zpool status
  pool: zbackup
 state: ONLINE
  scan: none requested
config:

        NAME        STATE     READ WRITE CKSUM
        zbackup     ONLINE       0     0     0
          raidz2-0  ONLINE       0     0     0
            ada1    ONLINE       0     0     0
            ada2    ONLINE       0     0     0
            ada3    ONLINE       0     0     0
            da0     ONLINE       0     0     0
            da1     ONLINE       0     0     0

errors: No known data errors
  5. Reboot 
    # reboot

Monitor

EPICS IOC

  • Working on k1script1
  • Directory: /opt/rtcds/kamioka/k1/target/k1sys/

Script

  • /kagra/bin/SYS/backup_status.sh
  • Run daily at 9:00 am

HDD list

HDD list

device

Serial

for

ada0

system

ada1

1SHW665Z

data

ada2

1SHM8GSZ

ada3

1SHX5YDZ

da0

1SHWVU4Z

da1

1SHWE8JZ

KAGRA/Subgroups/DGS/Projects/Backup (last edited 2023-06-28 13:35:49 by shoichi.oshino)