File Spooler and Copy System for Fast Data Transfer.

The ALICE (A Large Ion Collider Experiment) experiment at the CERN (European Organization for Nuclear Research) LHC (Large Hadron Collider) is preparing for the LHC Run3, beginning in 2021, with a detector and computing upgrade. On the computing side, a large, purpose-build computing farm (O2) consisting of CPU and GPU will process the data coming from the experimental setup at an average input rate of some 2TB/sec and output rate of 100GB/sec. The farm will consist of few hundred off-the-shelve servers, called Event Processing Nodes (EPN), collectively connected to a remote disk-based storage system. The EPNs will process data in near-real time during the ALICE detector operation with expected output rate to storage of ~100GB/sec. To avoid interruptions of processing due to network glitches or overload, we foresee to equip the EPNs with fast high-capacity SSDs for temporary data storage. The data stored on the SSDs must be transferred asynchronously to the remote storage element. The transfer operation is time-critical, as the SSDs will be able to hold at most a few hours of data accumulation. This paper presents a method for fast copying the files generated by the EPNs while ensuring no data loss and caching all the encountered errors.