The Grid2003 Production Grid: Principles and Practice

I. Foster; J. Gieraltowski; S. Gose; N. Maltsev; E. May; A. Rodriguez; D. Sulakhe; A. Vaniachine; J. Shank; S. Youssef; D. Adams; R. Baker; W. Deng; J. Smith; D. Yu; I. Legrand; S. Singh; C. Steenberg; Y. Xia; A. Afaq; E. Berman; J. Annis; L. A.T. Bauerdick; M. Ernst; I. Fisk; L. Giacchetti; G. Graham; A. Heavey; J. Kaiser; N. Kuropatkin; R. Pordes; V. Sekhri; J. Weigand; Y. Wu; K. Baker; L. Sorrillo; J. Huth; M. Allen; L. Grundhoefer; J. Hicks; F. Luehring; S. Peck; R. Quick; S. Simms; G. Fekete; J. van den Berg; K. Cho; K. Kwon; D. Son; H. Park; S. Canon; K. Jackson; D. E. Konerding; J. Lee; D. Olson; I. Sakrejda; B. Tierney; M. Green; R. Miller; J. Letts; T. Martin; D. Bury; C. Dumitrescu; D. Engh; R. Gardner; M. Mambelli; Y. Smirnov; J. Voeckler; M. Wilde; Y. Zhao; X. Zhao; P. Avery; R. Cavanaugh; B. Kim; C. Prescott; J. Rodriguez; A. Zahn; S. McKee; C. Jordan; J. Prewett; T. Thomas; H. Severini; B. Clifford; E. Deelman; L. Flon; C. Kesselman; G. Mehta; N. Olomu; K. Vahi; K. De; P. McGuigan; M. Sosebee; D. Bradley; P. Couvares; A. de Smet; C. Kireyev; E. Paulson; A. Roy; S. Koranda; B. Moe; B. Brown; P. Sheldon

The Grid2003 Production Grid: Principles and Practice

I. Foster
, J. Gieraltowski
, S. Gose
, N. Maltsev
, E. May
, A. Rodriguez
, D. Sulakhe
, A. Vaniachine
, J. Shank
, S. Youssef
, D. Adams
, R. Baker
, W. Deng
, J. Smith
, D. Yu
, I. Legrand
, S. Singh
, C. Steenberg
, Y. Xia
, A. Afaq

E. Berman, J. Annis, L. A.T. Bauerdick, M. Ernst, I. Fisk, L. Giacchetti, G. Graham, A. Heavey, J. Kaiser, N. Kuropatkin, R. Pordes, V. Sekhri, J. Weigand, Y. Wu, K. Baker, L. Sorrillo, J. Huth, M. Allen, L. Grundhoefer, J. Hicks, F. Luehring, S. Peck, R. Quick, S. Simms, G. Fekete, J. van den Berg, K. Cho, K. Kwon, D. Son, H. Park, S. Canon, K. Jackson, D. E. Konerding, J. Lee, D. Olson, I. Sakrejda, B. Tierney, M. Green, R. Miller, J. Letts, T. Martin, D. Bury, C. Dumitrescu, D. Engh, R. Gardner, M. Mambelli, Y. Smirnov, J. Voeckler, M. Wilde, Y. Zhao, X. Zhao, P. Avery, R. Cavanaugh, B. Kim, C. Prescott, J. Rodriguez, A. Zahn, S. McKee, C. Jordan, J. Prewett, T. Thomas, H. Severini, B. Clifford, E. Deelman, L. Flon, C. Kesselman, G. Mehta, N. Olomu, K. Vahi, K. De, P. McGuigan, M. Sosebee, D. Bradley, P. Couvares, A. de Smet, C. Kireyev, E. Paulson, A. Roy, S. Koranda, B. Moe, B. Brown, P. Sheldon

Department of Computer Science and Engineering

Argonne National Laboratory
Boston University
Brookhaven National Laboratory
California Institute of Technology
Fermi National Accelerator Laboratory
Hampton U
Harvard University
Indiana University Bloomington
Johns Hopkins University
Kyungpook National University
Lawrence Berkeley National Laboratory
SUNY Buffalo
University of California at San Diego
The University of Chicago
University of Florida
University of Michigan, Ann Arbor
University of New Mexico
University of Oklahoma
University of Southern California
University of Texas at Arlington
University of Wisconsin-Madison
University of Wisconsin-Milwaukee
Vanderbilt University

Research output: Contribution to journal › Conference article › peer-review

68 Scopus citations

Abstract

The Grid2003 Project has deployed a multi-virtual organization, application-driven grid laboratory ("GridS") that has sustained for several months the production-level services required by physics experiments of the Large Hadron Collider at CERN (ATLAS and CMS), the Sloan Digital Sky Survey project, the gravitational wave search experiment LIGO, the BTeV experiment at Fermilab, as well as applications in molecular structure analysis and genome analysis, and computer science research projects in such areas as job and data scheduling. The deployed infrastructure has been operating since November 2003 with 27 sites, a peak of 2800 processors, work loads from 10 different applications exceeding 1300 simultaneous jobs, and data transfers among sites of greater than 2 TB/day. We describe the principles that have guided the development of this unique infrastructure and the practical experiences that have resulted from its creation and use. We discuss application requirements for grid services deployment and configuration, monitoring infrastructure, application performance, metrics, and operational experiences. We also summarize lessons learned.

Original language	English
Pages (from-to)	236-245
Number of pages	10
Journal	IEEE International Symposium on High Performance Distributed Computing, Proceedings
State	Published - 2004
Event	Proceedings - 13th IEEE International Symposium on High Performance Distributed Computing - Honolulu, HI, United States Duration: Jun 4 2004 → Jun 6 2004

Cite this

Foster, I., Gieraltowski, J., Gose, S., Maltsev, N., May, E., Rodriguez, A., Sulakhe, D., Vaniachine, A., Shank, J., Youssef, S., Adams, D., Baker, R., Deng, W., Smith, J., Yu, D., Legrand, I., Singh, S., Steenberg, C., Xia, Y., ... Sheldon, P. (2004). The Grid2003 Production Grid: Principles and Practice. IEEE International Symposium on High Performance Distributed Computing, Proceedings, 236-245.

@article{0df99caffc01448ab49af1e5ad5b7a62,

title = "The Grid2003 Production Grid: Principles and Practice",

abstract = "The Grid2003 Project has deployed a multi-virtual organization, application-driven grid laboratory ({"}GridS{"}) that has sustained for several months the production-level services required by physics experiments of the Large Hadron Collider at CERN (ATLAS and CMS), the Sloan Digital Sky Survey project, the gravitational wave search experiment LIGO, the BTeV experiment at Fermilab, as well as applications in molecular structure analysis and genome analysis, and computer science research projects in such areas as job and data scheduling. The deployed infrastructure has been operating since November 2003 with 27 sites, a peak of 2800 processors, work loads from 10 different applications exceeding 1300 simultaneous jobs, and data transfers among sites of greater than 2 TB/day. We describe the principles that have guided the development of this unique infrastructure and the practical experiences that have resulted from its creation and use. We discuss application requirements for grid services deployment and configuration, monitoring infrastructure, application performance, metrics, and operational experiences. We also summarize lessons learned.",

author = "I. Foster and J. Gieraltowski and S. Gose and N. Maltsev and E. May and A. Rodriguez and D. Sulakhe and A. Vaniachine and J. Shank and S. Youssef and D. Adams and R. Baker and W. Deng and J. Smith and D. Yu and I. Legrand and S. Singh and C. Steenberg and Y. Xia and A. Afaq and E. Berman and J. Annis and Bauerdick, \{L. A.T.\} and M. Ernst and I. Fisk and L. Giacchetti and G. Graham and A. Heavey and J. Kaiser and N. Kuropatkin and R. Pordes and V. Sekhri and J. Weigand and Y. Wu and K. Baker and L. Sorrillo and J. Huth and M. Allen and L. Grundhoefer and J. Hicks and F. Luehring and S. Peck and R. Quick and S. Simms and G. Fekete and \{van den Berg\}, J. and K. Cho and K. Kwon and D. Son and H. Park and S. Canon and K. Jackson and Konerding, \{D. E.\} and J. Lee and D. Olson and I. Sakrejda and B. Tierney and M. Green and R. Miller and J. Letts and T. Martin and D. Bury and C. Dumitrescu and D. Engh and R. Gardner and M. Mambelli and Y. Smirnov and J. Voeckler and M. Wilde and Y. Zhao and X. Zhao and P. Avery and R. Cavanaugh and B. Kim and C. Prescott and J. Rodriguez and A. Zahn and S. McKee and C. Jordan and J. Prewett and T. Thomas and H. Severini and B. Clifford and E. Deelman and L. Flon and C. Kesselman and G. Mehta and N. Olomu and K. Vahi and K. De and P. McGuigan and M. Sosebee and D. Bradley and P. Couvares and \{de Smet\}, A. and C. Kireyev and E. Paulson and A. Roy and S. Koranda and B. Moe and B. Brown and P. Sheldon",

year = "2004",

language = "English",

pages = "236--245",

journal = "IEEE International Symposium on High Performance Distributed Computing, Proceedings",

issn = "1082-8907",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

note = "Proceedings - 13th IEEE International Symposium on High Performance Distributed Computing ; Conference date: 04-06-2004 Through 06-06-2004",

}

Foster, I, Gieraltowski, J, Gose, S, Maltsev, N, May, E, Rodriguez, A, Sulakhe, D, Vaniachine, A, Shank, J, Youssef, S, Adams, D, Baker, R, Deng, W, Smith, J, Yu, D, Legrand, I, Singh, S, Steenberg, C, Xia, Y, Afaq, A, Berman, E, Annis, J, Bauerdick, LAT, Ernst, M, Fisk, I, Giacchetti, L, Graham, G, Heavey, A, Kaiser, J, Kuropatkin, N, Pordes, R, Sekhri, V, Weigand, J, Wu, Y, Baker, K, Sorrillo, L, Huth, J, Allen, M, Grundhoefer, L, Hicks, J, Luehring, F, Peck, S, Quick, R, Simms, S, Fekete, G, van den Berg, J, Cho, K, Kwon, K, Son, D, Park, H, Canon, S, Jackson, K, Konerding, DE, Lee, J, Olson, D, Sakrejda, I, Tierney, B, Green, M, Miller, R, Letts, J, Martin, T, Bury, D, Dumitrescu, C, Engh, D, Gardner, R, Mambelli, M, Smirnov, Y, Voeckler, J, Wilde, M, Zhao, Y, Zhao, X, Avery, P, Cavanaugh, R, Kim, B, Prescott, C, Rodriguez, J, Zahn, A, McKee, S, Jordan, C, Prewett, J, Thomas, T, Severini, H, Clifford, B, Deelman, E, Flon, L, Kesselman, C, Mehta, G, Olomu, N, Vahi, K, De, K, McGuigan, P, Sosebee, M, Bradley, D, Couvares, P, de Smet, A, Kireyev, C, Paulson, E, Roy, A, Koranda, S, Moe, B, Brown, B & Sheldon, P 2004, 'The Grid2003 Production Grid: Principles and Practice', IEEE International Symposium on High Performance Distributed Computing, Proceedings, pp. 236-245.

TY - JOUR

T1 - The Grid2003 Production Grid

T2 - Proceedings - 13th IEEE International Symposium on High Performance Distributed Computing

AU - Foster, I.

AU - Gieraltowski, J.

AU - Gose, S.

AU - Maltsev, N.

AU - May, E.

AU - Rodriguez, A.

AU - Sulakhe, D.

AU - Vaniachine, A.

AU - Shank, J.

AU - Youssef, S.

AU - Adams, D.

AU - Baker, R.

AU - Deng, W.

AU - Smith, J.

AU - Yu, D.

AU - Legrand, I.

AU - Singh, S.

AU - Steenberg, C.

AU - Xia, Y.

AU - Afaq, A.

AU - Berman, E.

AU - Annis, J.

AU - Bauerdick, L. A.T.

AU - Ernst, M.

AU - Fisk, I.

AU - Giacchetti, L.

AU - Graham, G.

AU - Heavey, A.

AU - Kaiser, J.

AU - Kuropatkin, N.

AU - Pordes, R.

AU - Sekhri, V.

AU - Weigand, J.

AU - Wu, Y.

AU - Baker, K.

AU - Sorrillo, L.

AU - Huth, J.

AU - Allen, M.

AU - Grundhoefer, L.

AU - Hicks, J.

AU - Luehring, F.

AU - Peck, S.

AU - Quick, R.

AU - Simms, S.

AU - Fekete, G.

AU - van den Berg, J.

AU - Cho, K.

AU - Kwon, K.

AU - Son, D.

AU - Park, H.

AU - Canon, S.

AU - Jackson, K.

AU - Konerding, D. E.

AU - Lee, J.

AU - Olson, D.

AU - Sakrejda, I.

AU - Tierney, B.

AU - Green, M.

AU - Miller, R.

AU - Letts, J.

AU - Martin, T.

AU - Bury, D.

AU - Dumitrescu, C.

AU - Engh, D.

AU - Gardner, R.

AU - Mambelli, M.

AU - Smirnov, Y.

AU - Voeckler, J.

AU - Wilde, M.

AU - Zhao, Y.

AU - Zhao, X.

AU - Avery, P.

AU - Cavanaugh, R.

AU - Kim, B.

AU - Prescott, C.

AU - Rodriguez, J.

AU - Zahn, A.

AU - McKee, S.

AU - Jordan, C.

AU - Prewett, J.

AU - Thomas, T.

AU - Severini, H.

AU - Clifford, B.

AU - Deelman, E.

AU - Flon, L.

AU - Kesselman, C.

AU - Mehta, G.

AU - Olomu, N.

AU - Vahi, K.

AU - De, K.

AU - McGuigan, P.

AU - Sosebee, M.

AU - Bradley, D.

AU - Couvares, P.

AU - de Smet, A.

AU - Kireyev, C.

AU - Paulson, E.

AU - Roy, A.

AU - Koranda, S.

AU - Moe, B.

AU - Brown, B.

AU - Sheldon, P.

PY - 2004

Y1 - 2004

N2 - The Grid2003 Project has deployed a multi-virtual organization, application-driven grid laboratory ("GridS") that has sustained for several months the production-level services required by physics experiments of the Large Hadron Collider at CERN (ATLAS and CMS), the Sloan Digital Sky Survey project, the gravitational wave search experiment LIGO, the BTeV experiment at Fermilab, as well as applications in molecular structure analysis and genome analysis, and computer science research projects in such areas as job and data scheduling. The deployed infrastructure has been operating since November 2003 with 27 sites, a peak of 2800 processors, work loads from 10 different applications exceeding 1300 simultaneous jobs, and data transfers among sites of greater than 2 TB/day. We describe the principles that have guided the development of this unique infrastructure and the practical experiences that have resulted from its creation and use. We discuss application requirements for grid services deployment and configuration, monitoring infrastructure, application performance, metrics, and operational experiences. We also summarize lessons learned.

AB - The Grid2003 Project has deployed a multi-virtual organization, application-driven grid laboratory ("GridS") that has sustained for several months the production-level services required by physics experiments of the Large Hadron Collider at CERN (ATLAS and CMS), the Sloan Digital Sky Survey project, the gravitational wave search experiment LIGO, the BTeV experiment at Fermilab, as well as applications in molecular structure analysis and genome analysis, and computer science research projects in such areas as job and data scheduling. The deployed infrastructure has been operating since November 2003 with 27 sites, a peak of 2800 processors, work loads from 10 different applications exceeding 1300 simultaneous jobs, and data transfers among sites of greater than 2 TB/day. We describe the principles that have guided the development of this unique infrastructure and the practical experiences that have resulted from its creation and use. We discuss application requirements for grid services deployment and configuration, monitoring infrastructure, application performance, metrics, and operational experiences. We also summarize lessons learned.

UR - https://www.scopus.com/pages/publications/4944258377

M3 - Conference article

AN - SCOPUS:4944258377

SN - 1082-8907

SP - 236

EP - 245

JO - IEEE International Symposium on High Performance Distributed Computing, Proceedings

JF - IEEE International Symposium on High Performance Distributed Computing, Proceedings

Y2 - 4 June 2004 through 6 June 2004

ER -

The Grid2003 Production Grid: Principles and Practice

Abstract

Other files and links

Fingerprint

Cite this