... data.1.1
Additional comments: FAT32 is usually supported on Win,Linux and MAC OS; NTFS(MBR) is supported by Linux and Windows XP/7. Hard drives with capacities larger than 2.2TB as well as nowadays available pre-formatted drives are set up as NTFS(GPS) drives. This set up is supported by Windows 7. Such drives usually work on SL 6.x systems available, but are NOT supported on SL 5.x systems as well as on Windows XP systems (only read support at best). USB 3 is NOT supported on SL 5.x systems. On Linux systems the external drive needs to be connected to a system with keyboard/screen login, otherwise the automounter won't work.
Depending on filesystem, additional limitations may exist (e.g. limits for single file size on FAT or file attributes).
The transfer rates, i.e. the amount of time needed to copy data, is NOT only defined by the interface definition, which gives the upper theoretical limit of the bandwidth. It highly depends for example on the hardware itself, the file sizes of the data to transfer, the number of files, or the filesystem of the storage device. For example: The USB 2.0 protocol has a theoretical bandwidth of 480 MBit/s (= 60MiB/s). Due to protocol overhead, chipset limitations etc. the effective bandwidth turns out on the order of $\approx 70$% of the theoretical bandwidth ($\approx 40$MiB/s) at best. Typical data rates are $\approx 45$% lower than theroy ($\approx 30$MiB/s) and in particular when copying some ten thousands or even hundreds of thousands of files of small size (some ten to 100kb per file) the transfer rate will most likely result much closer to 1MiB/s.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
... III1.2
Photon Science user operation ended in Oct. 2012
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
...$ manager.$1.3
More precisely, a beamtime application ID is created when an application for beamtime has been submitted via DOOR. The associated meta data 'specific period' and 'participants' (who initially will have read permission to the data set) are created by scheduling the beamtime (beamline manager) and by providing the names participant names (declaration by the user via DOOR), respectivley.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
... instance1.4
More precisely it creates a file in the traditional dCache with tape back end.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
... /path/to/file.tar1.5
The tarball will be extracted into directory /to/
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
... 'j').1.6
Please note that other compression schemes might work as well, depending on system setup etc. Usually 'gzip' ('z'-option) and 'compress' ('Z'-option) are supported and allow faster compression compared to bzip2, but both schemes do not really allow for partial archive recovery in case of archive corruption.
Note A.R.: DESY-IT has to tell what 'J' currently means ('J' has been reassigned to xz in GNU tar 2009 (?), before 'J'-option denoted lzma(?)). Can we assume that -xz, -lzip,-lzma or -lzop are supported by default? As far as I've realised, tar setup / staging is executed with '-no-auto-compress' and thus do not uses file suffix to determine the compression program. Do we have to define a file extension for compressed tarballs? '.tbz' or '.tbz2' are common for bzip2 compression.
Do we have to explicitly highlight that one should not use the -use-compress-program=PROG option, i.e. with 'zip' program - depending on zip program / zip format implementation installed, there are several limitations regarding archive length, number of files .... Btw. this is also the case for POSIX tar implementation, which we hopefully do not find any more at DESY.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
... migration1.7
When migrating from other sources, e.g. data of DESY staff taken at external facilities, one has to account for the underlying file system [for example ACL settings in AFS). Inventing ACLs/pnfs 4.1 for the OnlineFS is under discussion, but would make it necessary to set up OnlineFS from scratch [i.e. fully copy/delete/restore all data from OnlineFS]. Note: Even with ACLs on the OnlineFS we still have the problem writing data from Pilatus detectors to it - The first generation Pilatus detectors have a local account 'det' which is not covered by kerberos or something similar.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
... themself1.8
Until now, FS-EC (i.e. J. Grabitz and A. Rothkirch) migrated the data to the dCache in coordination with beamline responsibles.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
... '/scratch'1.9
When using /scratch as storage space one has to remind that one writes to the local disk of a specific computer, i.e.  one has to remember on which work group server processing was done and data was written
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
... system3.1
Take over is limited to a single system, i.e. if both heads 1 and 2 fail, the attached storage is offline.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
... aggregates3.2
More precisely, the OnlineFS has eight aggregates, four management aggregates and four aggregates for the data.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.