Back to the main page

Dirvish script and config file

About Dirvish Main configuration script

Master conf file for Dirvish is /opt/csw/etc/dirvish/master.conf (if you install Dirvish using pkg-get, this is most likely location). 

Check dirvish.org for man pages about this conf file. 

Most commonly used options for master.conf are: 

 bank  : defines path to dirs containing vaults (FS), supports multiple banks

 client  : specify client to backup (Usually, it's localhost in /opt/csw/etc/dirvish/master.conf and remote machine [that is backed up] in $VAULT/dirvish/default.conf) 

 exclude  : what you want to exclude, maybe logs, lost+found, etc. 

 expire-default   : works if 'expire' is not used (see man page). This tells how long you want to keep backup.

 expire-rules   : tells rules for expiration.  

 image-default   : tells default image name, if you have one image per day, say your name is like: %Y-%m-%d

 image-perm   : tells image permissions, by default is 755

 index   : created index file containing list of all files in image, also defines how to compress it, like with gzip

 meta-perm   : tells permission of image meta-data files (this is security concern since logs have file names and they can have some sensitive info)

 stats   : rsync will report transfer statistics, 0 = yes

 summary   : short(default)/long summary format 

 xdev   : don't cross FS mount points 

Example: 

/opt/csw/etc/dirvish> cat master.conf
# Master configuration file
# For OS backup

# Banks should be defined on different lines
bank:
        /backup/bankOS

# Don't cross FS mount points 
xdev: 1 

# Create index file and compress it
index: gzip

# Try to handle sparse files efficiently
sparse: 1

# Secure (meta)-data
image-perm: 700
meta-perm: 600

# Image name
image-default: %Y-%m-%d-%Hh

# Exclude temp files we really do not need to backup 
exclude:
        **/var/log/**
        **/var/tmp/**
        **/var/cron/*log
        **/var/svc/log/**

# Default expiration time, keep images for 10 days
expire-default: +10 days

# Expiration strategy
expire-rule:
        hr { 5-20 }                             +2 days
        wd { 6 }                                +1 month
        wd { 6 } md { 22-28 }                   +3 months
        wd { 6 } md { 22-28 } mo { 6 }          +12 months
        wd { 6 } md { 22-28 } mo { 12 }         never

# Default client (=hostname) 
client: hostname 


Let me explain more the expiration strategy.
Note: just example for better understanding, don't buy the story for your production environment. 

hr { 5-20 }                             +2 days      ---> images taken between 5-20 h (any day) are kept only 2 days 
wd { 6 }                                +1 month     ---> images taken any other Fridays are kept 1 month
wd { 6 } md { 22-28 }                   +3 months    ---> images taken on last Friday (all other months) are kept 3 months 
wd { 6 } md { 22-28 } mo { 6 }          +12 months   ---> images taken on last Friday in June are kept 12 months  
wd { 6 } md { 22-28 } mo { 12 }         never        ---> images taken on last Friday in December are kept forever

You see that the days of the week are numbered 1-Sunday .... 7-Saturday

This calendar can help for better understanding of expiration strategy.








 

Jan

1 2 3 4 5 6 7
 

8 9 10 11 12 13 14
 

15 16 17 18 19 20 21
 

22 23 24 25 26 27 28
 

29 30 31





 

Feb

1 2 3 4 5 6 7
 

8 9 10 11 12 13 14
 

15 16 17 18 19 20 21
 

22 23 24 25 26 27 28
 

29





 

Mar

1 2 3 4 5 6 7
 

8 9 10 11 12 13 14
 

15 16 17 18 19 20 21
 

22 23 24 25 26 27 28
 

29 30 31











 

Apr

1 2 3 4 5 6 7
 

8 9 10 11 12 13 14
 

15 16 17 18 19 20 21
 

22 23 24 25 26 27 28
 

29 30 





 

May

1 2 3 4 5 6 7
 

8 9 10 11 12 13 14
 

15 16 17 18 19 20 21
 

22 23 24 25 26 27 28
 

29 30 31





 

Jun

1 2 3 4 5 6 7
 

8 9 10 11 12 13 14
 

15 16 17 18 19 20 21
 

22 23 24 25 26 27 28
 

29 30 










 

Jul

1 2 3 4 5 6 7
 

8 9 10 11 12 13 14
 

15 16 17 18 19 20 21
 

22 23 24 25 26 27 28
 

29 30 31





 

Aug

1 2 3 4 5 6 7
 

8 9 10 11 12 13 14
 

15 16 17 18 19 20 21
 

22 23 24 25 26 27 28
 

29 30 31





 

Sep

1 2 3 4 5 6 7
 

8 9 10 11 12 13 14
 

15 16 17 18 19 20 21
 

22 23 24 25 26 27 28
 

29 30 










 

Oct

1 2 3 4 5 6 7
 

8 9 10 11 12 13 14
 

15 16 17 18 19 20 21
 

22 23 24 25 26 27 28
 

29 30 31





 

Nov

1 2 3 4 5 6 7
 

8 9 10 11 12 13 14
 

15 16 17 18 19 20 21
 

22 23 24 25 26 27 28
 

29 30 





 

Dec

1 2 3 4 5 6 7
 

8 9 10 11 12 13 14
 

15 16 17 18 19 20 21
 

22 23 24 25 26 27 28
 

29 30 31






 
Note: the calendar is just for example as you can see it's not real proper layout. 



        images taken between 5-20 h (any day) are kept only 2 days


        images taken any other Fridays are kept 1 month


        images taken on last Friday (all other months) are kept 3 months 


        images taken on last Friday in June are kept 12 months 


        images taken on last Friday in December are kept forever

Vault (FS) configuration file 

Each vault has its own conf file, $VAULT/dirvish/default.conf   

In $VAULT/dirvish/default.conf you may want to place "tree" for specific FS, pre/post scripts, and excluded directories for that FS.  

It can look like this for /var

# client is remote host that is backed up on our dirvish server
#
# If there is no 'client' here, that means that we backup localhost's /var
# In this case, client is specified in /opt/csw/etc/dirvish/master.conf
client: remote_hostname

# we backup client's /var FS
tree:   /var
exclude:
        /tmp/**
        /log/**
        /cron/*log
        /yp/ypxfr.log

The command dirvish

The command dirvish creates a backup image of vault (FS). 

The image is a directory containing: 

- transfer log - output of pre/post commands and rsync log with modified files with statistics
- summary - show how image was created and how to manage it
- tree - copy of FS
- rsync_error - if there is transfer error 

Tips: 

Check verion: 

# dirvish --version
dirvish version 1.2

Create initial image from source FS (since there is no any reference image in this moment)

# dirvish --vault /var --init
 
See what would have happened if you run backup on FS /var:

# dirvish --vault /var --no-run

Basically dirvish reads config files: 

1. /opt/csw/etc/dirvish/master.conf
2. $VAULT/dirvish/default.conf

The command dirvish-expire

The command dirvish-expire deletes expired images. 

The command checks image's summary file and looks for field like "Expire: +2 days".

If field says the image expired, the command will delete the image.  

Tip: use dirvish-expire --no-run to see what would have happened, no real action will be performed. 

# dirvish-expire --no-run
Expiring images as of 2009-09-08 16:11:24
VAULT:BRANCH    IMAGE           CREATED           EXPIRED
root:default    2009-09-06-12h  2009-09-06 12:06  +2 days == 2009-09-08 12:01
var:default     2009-09-06-12h  2009-09-06 12:08  +2 days == 2009-09-08 12:08

During expiration, you can notice 'remove' command works hard to expire/delete the image. 

# ps -ef
rm -rf /backup/bank0/root/2009-09-06-12h/tree

History file 

In the directory $VAULT/dirvish/ there is the file default.hist

It gives you: image name, when it is created, image reference for this image (from where non changed files are copied) and when image will expire. 

Something like this: 

#IMAGE  		CREATED 		REFERECE        EXPIRES 
2009-09-03-18h  	2009-09-03 18:07:00     2009-09-03-13h  +1 month == 2009-10-03 18:06:14
2009-09-03-23h  	2009-09-03 23:06:49     2009-09-03-18h  +1 month == 2009-10-03 23:06:02
2009-09-04-00h  	2009-09-04 00:06:54     2009-09-03-23h  +10 days == 2009-09-14 00:06:12
2009-09-04-06h  	2009-09-04 06:07:21     2009-09-04-00h  +2 days == 2009-09-06 06:06:37
2009-09-04-12h  	2009-09-04 12:06:59     2009-09-04-06h  +2 days == 2009-09-06 12:06:14
Back to the main page