For members of the Theoretical Physics (TP) in Konstanz the HOME directory is located on the standard file server. For all other users the HOME directory is located on the storage server of SCC. There is a quota of 100 GB per user.

Please use the data directory that fits best for your simulations. All nodes have a /localscratch directory for temporary data storage which has no backup. A bigger and faster scratch area is available under /work/scratch/ (also excluded from backup).

Following data directories are available:

 Storage  Shortcut Size Used IOPS max. Bandwidth Quota Usage
/home/watt, /home/newton  $HOME 11 TB 5 TB 10000 1Gb/s 100 GB Long term data storage (daily backup)
/home/scc  $HOME  3 TB 2 TB 100000 56Gb/s 100 GB Long term data storage (daily backup)
/data/scc /work/ 120 TB 100 TB 100000 56Gb/s none Long term data storage (weekly backup)
/data/scc2 /work2/ 180 TB 140 TB 100000 56Gb/s none Long term data storage of group Peter (weekly backup)
/data/data   61 TB 50 TB 10000 1Gb/s none

Long term data storage (weekly backup)

  /data/archiv     10000 1Gb/s none

Long term data archive (weekly backup)

/data/sphinx   5.3 TB 3.6 TB 10000 1Gb/s none Long term data storage of SPHINX cluster (weekly backup)
/data/grendel   7.2 TB 6 TB 2000 100Mb/s none Long term data storage of GRENDEL cluster (weekly backup)
/data/nessie   19 TB 14 TB 10000 1Gb/s none Long term data storage of NESSIE cluster (weekly backup)
/data/www   250 GB 70 GB 10000 1Gb/s none Web server data (daily backup)
/localscratch /scratch/* 1.7-2.7 TB/node   100 150Mb/s none Short time/temporary data storage (no backup)

Usage monitoring can be found here (internal).

All data areas have a scratch/ directory for temporary data without backup. Please use it when possible. Also please use compression when practical (see Compression tutorial).

Never use "cp" or "mv" to copy/move your data. Use "rsync" instead, because rsync support compression and sparse files, maintains time stamps and attributes and only transfers file changes.

Here are some typical use cases:

  • local copy/move directory dir1 to dir2 : rsync -avhSP dir1/ dir2/
  • copy/move from host over slow network (100 MB/s): rsync -avzhSP -e "ssh -T -c arcfour -o Compression=no -x" host:dir1/ dir2/
  • copy/move to host over slow network (100 MB/s): rsync -avzhSP -e "ssh -T -c arcfour -o Compression=no -x" dir1/ host:dir2/
  • copy/move from host over fast network (1 GB/s): rsync -avhSP -e "ssh -T -c arcfour -o Compression=no -x" host:dir1/ dir2/
  • copy/move to host over slow network (1 GB/s): rsync -avhSP -e "ssh -T -c arcfour -o Compression=no -x" dir1/ host:dir2/

rsync compression (option -z) is only useful für slow networks. Also the default encryption and compression of ssh is slow, so use "arcfour" and no ssh compression. The options -a and -v turns on archive mode preserving time stamps, links etc and verbosity. The options -S and -P care about sparse and partial transfer.

Depending on the data area there is a daily or weekly backup of all data for one year. There is also an archive to store data forever that is not changing anymore. Usage monitoring can be found here (internal).

Please let us know if you like to have something restored from the backup or have questions about the archive or where to store data safely.