We present a transparent, system-level checkpointing solution for master-worker parallelism that automatically adapts, upon restart, to the number of processor nodes available. This is important, since nodes in a cluster fail. It also allows one to adapt to using multiple cluster partitions and multiple resources from the Computational Grid, as they become available. Checkpointing a master-worker computation has the additional advantage of needing to checkpoint only the master process. This is both fast and more economical of disk space. This has been demonstrated by checkpointing Geant4, a million line C++ program. Our solution has been implemented in the context of TOP-C (Task Oriented Parallel C/C++), a free, open-source parallel package, although it can easily be ported to additional master-worker packages.
checkpointing, cluster computing, Geant4, C++, TOP-C
Personal use of this material is permitted. Permission from IEEE must be obtained for all other users, including reprinting/ republishing this material for advertising or promotional purposes, creating new collective works for resale or redistribution to servers or lists, or reuse of any copyrighted components of this work in other works.
Cooperman, Gene D.; Ansel, Jason; and Ma, Xiaoqin, "Transparent adaptive library-based checkpointing for master-worker style parallelism" (2006). Computer and Information Science Faculty Publications. Paper 15. http://hdl.handle.net/2047/d20000618
Click button above to open, or right-click to save.