Big data applications such as Apache Hadoop and Apache Spark are supervised by resource managers. This project explores the use of a machine learning based workload classifier called KERMIT to assist resource managers in predicting workload change in distributed clusters and evaluating which metrics are best suited to identifying true change. The results of this project are an important contribution towards implementing classifiers best suited to specific types of classification, selecting an ideal width for statistical inference and for further research into selecting which measures are best suited for which types of data.