The HOME directory contains the important settings and configurations of a user. It can be accessed on any nodes as /home/scc/<user> (or /home/user/<user> if you have an Theoretical Physics account). There is a default quota of 100 GB for the HOME directory, so bigger data should be stored on the DATA areas (see below). You can check your quota with /software/bin/quota.

Please use the directories that fits best for your work. All nodes have a /localscratch directory for temporary data storage with no backup. A bigger and faster scratch area is available under /work/scratch/ (also excluded from backup). There may be a link to the scratch area in your data directory named nobackup.

When the filesystem supports compression, only use compression with higher rates (not zip or gzip, see Compression tutorial).

Following data directories are available:

 Storage  Shortcut Size   IOPS max. Bandwidth Quota Compression Backup Usage
/home/user, /home/scc  $HOME 11 TB   100000 56Gb/s 100 GB yes daily (+15 min snapshots) User Home

/data/scc

(/data/scc4)

/work 640 TB   100000 100Gb/s none yes monthly Live data storage
/data/scc2 /work2 180 TB   100000 56Gb/s none no monthly Live data storage of group Peter
/data/scc3 /work3 320 TB   100000 56Gb/s none no monthly Live data storage of group Mayer and Peter
/data/archiv   640 TB   10000 56Gb/s none yes monthly

Long term data archive

/data/www   250 GB   10000 1Gb/s none no daily Web server data
/localscratch /scratch/* 1-3 TB/node   100 150Mb/s none no no Short time/temporary data storage

Usage monitoring can be found here (internal).

Backup and Archive

Depending on the data area there is a daily or monthly backup of all data. There is also an archive (with backup) to store data that is not changing anymore (=cold data) for at least 10 years. Contact Support when you want to archive your data or your account. Inactive accounts will be deleted after 5 years and data archived for at least 10 years (when not specified differently).

Please let us know if you like to have something restored from the backup or have questions about the archive or where to store data safely.

Data transfer

Never use "cp" or "mv" to copy/move your data. Use "rsync" instead, because rsync support compression and sparse files, maintains time stamps and attributes and only transfers file changes.

Here are some typical use cases:

  • local copy/move directory dir1 to dir2 : rsync -avhSP dir1/ dir2/
  • copy/move from host over slow network (100 MB/s): rsync -avhSPz -e "ssh -T -c aes128-ctr -o Compression=no -x" host:dir1/ dir2/
  • copy/move to host over slow network (100 MB/s): rsync -avhSPz -e "ssh -T -c aes128-ctr -o Compression=no -x" dir1/ host:dir2/
  • copy/move from host over fast network (1 GB/s or more): rsync -avhSP -e "ssh -T -c aes128-ctr -o Compression=no -x" host:dir1/ dir2/
  • copy/move to host over slow network (1 GB/s or more): rsync -avhSP -e "ssh -T -c aes128-ctr -o Compression=no -x" dir1/ host:dir2/

rsync compression (option -z) is only useful für slow networks. Also the default encryption and compression of ssh is slow, so use "aes128-ctr" and no ssh compression. The options -a and -v turns on archive mode preserving time stamps, links etc and verbosity. The options -S and -P care about sparse and partial transfer.