News: Maintenance; BL responsibles (IT); hasyutils; Remote BL Operation; Fs Data Management

• Maintenance work: in about 3 weeks there will be a downtime of 3 hours of the mail service for each user. Each user will be notified 1 week in advance.

  There will be a network maintenance on 18.5. from 6 to 8 h. It is in a maintenance week.

• BL responsibles (IT) : IT asks for a list of persons whom they could contact in case of IT-related issues, mostly GPFS.

  Is the beamline handy an option?

• hasyutils: SardanaAIO.py has been upgraded to better handle additional Pools that have been installed. If a Pool has ever been exported, it will be deleted by AIO. Furthermore, Pool not pointed to byh the Macroserver will be ignored.

  Countermeasure for recent incidents.

• Remote BL Operation: the following requirements have be mentioned during a survey and in a BL mgr meeting
  • *** gamma portal: DESY should offer ssh + rsync for download.

    Other ways to transfer data from DESY to a remote institutes are globus-online and rsync. Julia is currently collecting information which can then be given to our users.

  • *** access multiple hosts per beamline: complicated discussion is ongoing. For urgent cases ask Marcus to change the configuration on the DMZ proxy manually.

    Several options have been discussed:

    • users see a single host for a beamline. The actual target host is selected by configuring the httpd in the DMZ.

    • users connect from outside to the target host, the selection is based on the used experimental setup, this process requires a substantial change in Door, which is currently based on beamlines, not experimental setups.

    • install an extra remote target host for each beamline requesting this service, from where to operate the different experimental setups. Pro: the remote desktop runs on a separate PC.

      A script will swithch from one host to another: copy the online.xml, update the macros, perform AIO.

      The extra remote target host cannot be a VM. Marcus did some tests.

  • zoom licences at the beam lines: can be ordered via AMS, create a ticket in FS-EC.

  • kiosk mode for multiple applications, ongoing

  • scaling problem due to different monitor resolutions: prepare the session on a low resolution monitor, ask users to use high resolution monitors or use browser zoom.

• Fs Data Management:

  News:

  • *** 12.5.2021: Inhouse research data have to stay longer on disk than 90 days after beamtime.

    In discussion: how to identify these data. Door gives us the proposal leader and the PI. There may be other DESY collaborations not falling into these categories. Option: make a declaration via the gamma portal.

  • *** 12.5.2021: improving the dCache perfomance is still worked on.

  • 21.4.2021: C-FEL will use 1.6 PiB of the new server.

  • 20.4.2021: Improved dCache performance, bandwidth about 100 TiB per day, needs further testing.

  • 14.4.2021: GPFS core will by extended, decided to purchase a new server 5.5 PiB, but: C-FEL buys a part of it and an old 900 TiB server ran out of maintenance.

  • 31.3.2021, dCache need better performance, esp. for many small files.

  • 31.3.2021, Science accounts: the registry-2 is needed for self-service password resets and resource management of science accounts. Currently certain platform adaptors are still missing.

  • Science accounts: request for simplifying the procedure. A discussion with IT has started. This problem is related to registry-2.

  • The pass-though mode is in preparation.

  • The pass-though will be enabled for beamtimes greater than X TB. ( first guess for X: 1 TB, AR and JH are investigating).

  • The GPFS core file system will be upgraded in the near future, 5.6 PiB (- 1.5 PiB for C-FEL).

  New demanding detectors forced us to reconsider our data management strategies. In collaboration with IT a note has been prepared proposing several measures which are currently being discussed:

  Summary:

  • the data-on-disk time will be limited to 90 days after the beamtime as ended.

  • the beamline file system will no longer store complete beamtimes. Instead this server will act as a pass-through buffer for data to be moved to the core file system.

  • a substantial upgrades of the FS-related IT infrastructure of the computer center are needed.

  • storage demands may become part of the proposal evaluation

  • maturity of the data analysis pipeline may also be considered during the proposal evaluation

  • the purchase of new detectors has to be reviewed for impact on data volumes.

  Concerns:

  • implement a simplified procedure for creating scientific accounts?

    The reason for this request is of course that the data are quickly moved to the core where they can only (except for exceptions) be accessed from real accounts.

  • can small beamtimes stay in the BL FS?

  • what about commissioning data? Does the 90 days rule also apply?

    The confluence page about the commissioning path can be found here

    Andre: commissioning data are created when groups overlap and when BL scientists align their setup. Commissioning runs are not saved to tape and they are not deleted. To answer the question: No.

  • FLASH: for beamtime > xTB 90 days, for beamtimes < xTB 360 days, with x being in the range 5-10 TB