Adaptive Scheduling in Virtualized Heterogeneous Clusters
Muhammad Atif (School of Computer Science, CECS ANU)
CS HDR MONITORING CompSys Research GroupDATE: 2010-04-08
TIME: 14:30:00 - 15:00:00
LOCATION: Ian Ross Seminar Room
CONTACT: JavaScript must be enabled to display this email address.
ABSTRACT:
Cluster computing has recently seen an evolution from single processor systems to multi core SMP systems. This has resulted in lower cost/performance ratio for the commodity of the shelf clusters (COTS).
This trend has coincided with the resurgence of virtualization technology. The virtualization technology is receiving widespread adoption mainly due to the potential benefits of server consolidation and isolation, flexibility, security and fault tolerance. Virtualization also offers other benefits, which include development/testing of applications, live migration and load balancing.
We believe that one can leverage the virtualization environment to achieve reduced job turn around times, especially in the case of COTS. These clusters are highly heterogeneous in nature which is due to the presence of different CPU architectures, available memory and the communication interfaces. Different CPU architectures, memory capacities, communication and I/O interfaces of the participating compute nodes present many challenges to the job schedulers and often result in under or over utilization of the compute resources.
We have investigated resource scheduling in the compute clusters with the perspective of dynamic resource remapping. Our approach is to profile each job in the compute farm at run-time, and arrive at the optimal resource map for the each job. We then migrate the jobs to the best suited compute nodes to improve the overall through put of the compute farm. For this, we have developed a novel heterogeneity and virtualization aware profiling framework, which is able to predict the CPU and communication characteristics of the high performance scientific applications.
This talk focuses on the design and implementation of profiling and prediction framework. We also present initial results which are quite promising.
BIO:
PhD student in Computer Science. http://cs.anu.edu.au/user/3926
