Experience with ATLAS MySQL PanDA database service

Y. Smirnov, T. Wlodek, K. De, J. Hover, N. Ozturk, J. Smith, T. Wenaus, D. Yu

Research output: Contribution to journalConference articlepeer-review

Abstract

The PanDA distributed production and analysis system has been in production use for ATLAS data processing and analysis since late 2005 in the US, and globally throughout ATLAS since early 2008. Its core architecture is based on a set of stateless web services served by Apache and backed by a suite of MySQL databases that are the repository for all PanDA information: active and archival job queues, dataset and file catalogs, site configuration information, monitoring information, system control parameters, and so on. This database system is one of the most critical components of PanDA, and has successfully delivered the functional and scaling performance required by PanDA, currently operating at a scale of half a million jobs per week, with much growth still to come. In this paper we describe the design and implementation of the PanDA database system, its architecture of MySQL servers deployed at BNL and CERN, backup strategy and monitoring tools. The system has been developed, thoroughly tested, and brought to production to provide highly reliable, scalable, flexible and available database services for ATLAS Monte Carlo production, reconstruction and physics analysis.

Original languageEnglish (US)
Article number042059
JournalJournal of Physics: Conference Series
Volume219
Issue number1 PART 4
DOIs
StatePublished - 2010
Event17th International Conference on Computing in High Energy and Nuclear Physics, CHEP 2009 - Prague, Czech Republic
Duration: Mar 21 2009Mar 27 2009

All Science Journal Classification (ASJC) codes

  • Physics and Astronomy(all)

Fingerprint Dive into the research topics of 'Experience with ATLAS MySQL PanDA database service'. Together they form a unique fingerprint.

Cite this