Unit Test Smells and Accuracy of Software Engineering Student Test Suites

Abstract

With an increasing emphasis on unit testing in computer science curricula, we examined students work on testing assignments to investigate their adoption of test smells---practices that indicate potential problems in unit tests. We discovered three common causes of test smells in students unit tests: multiple member function calls, multiple assertions, and conditional logic. We also explored how each might be associated with test inaccuracies. In a quasi-experimental study, we evaluated the quality of students unit tests by evaluating test accuracy---tests reliability in distinguishing between a corpus of acceptable production code and a separate corpus containing faults. Correlational and comparative analyses revealed that unit tests with calls to multiple member functions and/or conditional logic were associated with worse test accuracy. However, no relationship was found between test accuracy and whether or not unit tests contained multiple assertions.