Jeanette M. Sperhac

Analysis of XDMoD/SUPReMM Data Using Machine Learning Techniques

Machine learning techniques were applied to job accounting and performance data for application classification. Job data were accumulated using the XDMoD monitoring technology named SUPReMM, they consist of job accounting information, application information from Lariat/XALT, and job performance data from TACC_Stats. The results clearly demonstrate that community applications have characteristic signatures which can be exploited for job classification. We conclude that machine learning can assist in classifying jobs of unknown application, in characterizing the job mixture, and in harnessing the variation in node and time dependence for further analysis.

A Quantitative Analysis of Node Sharing on HPC Clusters Using XDMoD Application Kernels

In this investigation, we study how application performance is affected when jobs are permitted to share compute nodes. A series of application kernels consisting of a diverse set of benchmark calculations were run in both exclusive and node-sharing modes on the Center for Computational Researchs high-performance computing (HPC) cluster. Very little increase in runtime was observed due to job contention among application kernel jobs run on shared nodes. The small differences in runtime were quantitatively modeled in order to characterize the resource contention and attempt to determine the circumstances under which it would or would not be important. A machine learning regression model applied to the runtime data successfully fitted the small differences between the exclusive and shared node runtime data; it also provided insight into the contention for node resources that occurs when jobs are allowed to share nodes. Analysis of a representative job mix shows that runtime of shared jobs is affected primarily by the memory subsystem, in particular by the reduction in the effective cache size due to sharing; this leads to higher utilization of DRAM. Insights such as these are crucial when formulating policies proposing node sharing as a mechanism for improving HPC utilization.

A Multifaceted Approach Towards Education in Data Analytics

This work presents a novel multifaceted approach for facilitating education in data analytics. This novel approach is necessary as this new and growing discipline warrants understanding within diverse organizational arenas while recognizing that students are likely non-traditional, usually already employed in various fields and having different level of preparation. Elements of the approach are applied at the State University of New York (SUNY) – one of the largest university systems in the world.

TAS view of XSEDE users and usage

The Technology Audit Service has developed, XDMoD, a resource management tool. This paper utilizes XDMoD and the XDMoD data warehouse that it draws from to provide a broad overview of several aspects of XSEDE users and their usage. Some important trends include: 1) in spite of a large yearly turnover, there is a core of users persisting over many years, 2) user job submission has changed from primarily faculty members to students and postdocs, 3) increases in usage in Molecular Biosciences and Materials Research has outstripped that of other fields of science, 4) the distribution of user external funding is bimodal with one group having a large ratio of external funding to internal XSEDE funding (ie, CPU cycles) and a second group having a small ratio of external to internal (CPU cycle) funding, 5) user job efficiency is also bimodal with a group of presumably new users running mainly small inefficient jobs and another group of users running larger more efficient jobs, 6) finally, based on an analysis of citations of published papers, the scientific impact of XSEDE coupled with the service providers is demonstrated in the statistically significant advantage it provides to the research of its users.