Information Technology seminars
Rapid and reliable specification, execution,and management of large-scale science and engineering workflows
- Date and time:
- 03/11/2009, 11:00-12:00
- Location:
- Building: H, Room: 7.81, Caulfield campus
- Presenters:
- Ben Clifford, University of Chicago Computation Institute, up to July 2009 (Visiting Researcher)
- Abstract:
- Swift is a system for the rapid and reliable specification, execution,and management of large-scale science and engineering workflows. It supports applications that execute many tasks coupled by disk-resident datasets - as is common, for example, when analyzing large quantities of data or performing parameter studies or ensemble simulations. Swift comprises a language, SwiftScript, to specify complex loosely coupled parallel
computations, and an execution engine that can dispatch many (> 100,000)tasks to 1000s of processors, on desktop machines, parallel computers,campus grids, and multi-site grids Swift users span the physical sciences, biological sciences, social sciences, humanities, computer science, and education.
This talk will give some of the motivations behind Swift, an overview of the language and of the execution, some performance numbers, and related projects.
More information
- Speaker biographies:
- At UCCI Ben worked primarily on Swift, a programming language and execution environment for coarse grained distributed parallel data-centric computation.
Development included the integration of many different components: at the hardware level, a range of architectures from laptops to IBM Blue Gene supercomputers and large scale grids of clusters; at the middleware level, a range of execution and data transfer systems including the Globus Toolkit and PBS; at the application layer, interfacing existing scientific application codes to the programming model provided by Swift; and at the social level, co-ordinating the needs of various application and infrastructure groups each with their own vested interests.
He developed a small suite of performance analysis tools to aid in characterisation and debugging of the use of Swift for specific applications and environments. This was especially useful as the application space changed from early use with hundreds of minute long tasks to a hundred thousand tasks of sub-second duration composed of many smaller subtasks often of millisecond duration.
In addition to production-level development, he prototyped the use of Swift in new environments, including the South African National Grid (SAGrid) using gLite middleware, and on campus workstation pools using Condor; he also developed a prototype to record provenance of output data produced by Swift and contributed to the Open Provenance Model (OPM).Ben was also involved in education, outreach and training activities for Globus and the Open Science Grid.
In the summer of 2008, Ben mentored a Google Summer of Code student, Milena Nikolic, in a project to add stronger compile time type checking to Swift.
Prior to UCCI, Ben worked with USC/Information Sciences Institute (ISI), Center for Grid Technologies.
- Enquiries:
- Rob Gray
- Research group website:
-
http://www.infotech.monash.edu.au/research/centres/dsse/
