ERCIM News No.45 - April 2001 [contents]

The Particle Physics Grid Programme at CLRC

by John Gordon and David Boyd

---

Particle physics will present some of the most challenging data handling and computing problems over the next few years, culminating in operation of the Large Hadron Collider (LHC) at CERN near Geneva in 2005. The volume of data which will be produced is staggering, several petabytes per year from all the LHC experiments combined. Meanwhile experiments at other particle physics laboratories are producing data which is driving prototype Grid developments.

LHC Tier1 Centre
The deluge of real data will start in 2005 but its forerunner, the Monte Carlo simulations needed to develop the experiments and prepare the data analysis system, is already here. The proposed solution is to devolve much of the data processing to about 10 regional centres, known as Tier1 centres (CERN itself is the Tier0 centre). Each centre will be equipped with petabyte-scale data storage facilities and computing farms containing thousands of commodity processors to process the incoming data from CERN and pass it on to the hundreds of research institutes and groups which will analyse it further to extract new physics results.

CLRC’s Rutherford Appleton Laboratory (RAL) will act as the Tier1 centre for the UK and is now working closely with other UK particle physics groups to build a prototype testbed system using the Globus toolkit. This testbed is already carrying out simple computing tasks using these tools such as remote job submission and data retrieval. The diagram shows schematically the multi-tier hierarchy of centres.

Schematic of the 5 Tier model for LHC data processing.

With such a large distributed system, controlling access becomes a serious problem. The current solution to this is to establish a trusted Certification Authority (CA) which issues digital certificates to members of the community. These certificates are the passports to entry to the system and access to its growing computing resources. RAL now operates a CA on behalf of the UK particle physics community.

CDF
CDF is an experiment on the Tevatron at Fermilab. The CDF institutions in the UK (Glasgow, Liverpool, Oxford, University College/London) and RAL are planning to use Grid distributed computing ideas partly to help with CDF data analysis and partly as a rehearsal for computing for the LHC experiments at CERN. They are currently developing a demonstration project.

The plan is to implement three user services over five sites coupled using components of the Globus software toolkit. The three services are:

• The presentation of datafile catalogue metadata about datasets from the local catalogues at each participating site. The Globus capability used for this service is an LDAP server acting as a front end to the existing local metadata catalogue which has been implemented using an RDBMS. The LDAP server is used currently to supply information about local computing and disk resources over the Grid network.
• Remote job entry between sites. These Globus tools exist and interface to various batch systems for running the jobs. In the initial version, the user chooses where to send the job but a later development will choose the best location.
• Copying data between sites when bandwidth is available. A user interface accessing the information from the various catalogues will present the user with the information he needs to copy data and metadata to a convenient location and to submit his job. The replication would initially be driven by the user. This will be developed into an automatic policy-driven system later.

After successful completion of this demonstration project, the CDF-UK groups plan to ask the CDF team at Fermilab to install Grid software so that their central analysis cluster could participate in these three services.

BaBar
BaBar is an experiment on CP violation at the Stanford Linear Accelerator Centre (SLAC). The BaBar collaboration already operates a distributed database cache in the UK which is used to hold experimental data for analysis at 9 collaborating universities. This cache system is currently being augmented by a 600 processor, distributed, simulation and analysis facility. The resources located at each site primarily support local researchers, but developments have started to implement a Grid infrastructure based on the distributed data catalogue and to give all collaborators access to all sites to take advantage of data cached elsewhere and to use spare CPU capacity for simulation studies.

The main data store for the UK BaBar collaboration is at RAL and plans are now well developed to significantly enhance this centre. Current plans call for disk and tape storage of order 100TB by the end of 2002. This storage at RAL will be augmented by processing facilities to support bulk data queries/processing and the system will be fully integrated into the existing distributed database cache. These BaBar developments are complimentary to the prototype LHC Tier1 centre which will be developed at RAL.

Link:
UK High Energy Physics Grid web site: http://www.hep.grid.ac.uk/

Please contact:
John Gordon - CLRC
Tel: +44 1235 446574
E-mail: j.c.gordon@rl.ac.uk