Keywords

clusters, inefficiencies, hardware drift

Abstract

While clusters have already proven themselves in the world of high performance computing, some clusters are beginning to exhibit resource inefficiencies due to increasing hardware diversity. Much of the success of clusters lies in the use of commodity components built to meet various hardware standards. These standards have allowed a great level of hardware backwards compatibility that is now resulting in a condition referred to as hardware 'drift' or heterogeneity. The hardware heterogeneity introduces problems when diverse compute nodes are allocated to a parallel job, as most parallel jobs are not self-balancing. This paper presents a new method that allows the batch scheduling system to intelligently select the best resource set for a parallel job in order to minimize the adverse effects of hardware drift and increase overall performance of the cluster. The performance improvements of this technique are evaluated in terms of parallel job efficiency and scheduling resource utilization and overall system performance. Using the emulation capabilities of the Maui Scheduler, this paper evaluates a number of variations of the resource set allocation algorithm on true cluster throughput and utilization using a recorded trace workload from a production cluster.

Original Publication Citation

Improving Cluster Utilization Through Set-Based Allocation Policies, David B. Jackson, Brian Haymore, Julio Facelli, Quinn O. Snell. Proceedings of the International Conference on Parallel Processing Workshops, Valencia, Spain, September 21.

Document Type

Peer-Reviewed Article

Publication Date

2001-09-01

Permanent URL

http://hdl.lib.byu.edu/1877/2577

Publisher

IEEE

Language

English

College

Physical and Mathematical Sciences

Department

Computer Science

Download

Share

COinS