Skip to main navigation Skip to search Skip to main content

A framework for self-optimizing, fault-tolerant, high performance bulk data transfers in a heterogeneous grid environment

  • University of Wisconsin-Madison

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

6 Scopus citations

Abstract

The drastic increase in the data requirements of scientific applications combined with an increasing trend towards collaborative research has resulted in the need to transfer large amounts of data among the participating sites. The general approach to transferring such large amounts of data has been to either dump data to tapes and mail them or employ scripts with an operator at each site to baby-sit the transfers to deal with failures. We introduce a framework which automates the whole process of data movement between different sites. The framework does not require any human intervention and it can recover automatically from various kinds of storage system, network, and software failures, guaranteeing completion of the transfers. The framework has sophisticated monitoring and tuning capability that increases the performance of the data transfers on the fly. The framework also generates on-the-fly visualization of the transfers making identification of problems and bottlenecks in the system simple.

Original languageEnglish
Title of host publicationProceedings - 2nd International Symposium on Parallel and Distributed Computing, ISPDC 2003
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages137-144
Number of pages8
ISBN (Electronic)0769520693, 9780769520698
DOIs
StatePublished - 2003
Event2nd International Symposium on Parallel and Distributed Computing, ISPDC 2003 - Ljubljana, Slovenia
Duration: Oct 13 2003Oct 14 2003

Publication series

NameProceedings - 2nd International Symposium on Parallel and Distributed Computing, ISPDC 2003

Conference

Conference2nd International Symposium on Parallel and Distributed Computing, ISPDC 2003
Country/TerritorySlovenia
CityLjubljana
Period10/13/0310/14/03

Fingerprint

Dive into the research topics of 'A framework for self-optimizing, fault-tolerant, high performance bulk data transfers in a heterogeneous grid environment'. Together they form a unique fingerprint.

Cite this