Monday, September 14, 2015

Programming a test harness

I would like to build a test harness around programs. Automated tests should increase my confidence in the reproducibility of their outcome.
"Whenever you are tempted to type something into a print statement or a debugger expression, write it as a test instead." — Martin Fowler. Quoted here.

Where to store test data

While trying to find out where to place test data, this answer thought me to distinguish between unit tests, which are meant to test each function individually on small mock data and integration tests, which would be based on a larger, real dataset.

Testthat

In a commit called "Don't attach dplyr backends", Hadley Wickham removed direct function calls from loaded packages. Probably to ensure that packages are not loaded directly, he changed function calls to a form of packagename::function().

The author of the testthat R package wrote that autotest
"[...] promotes a workflow where the only way you test your code is through tests. Instead of modify-save-source-check you just modify and save, then watch the automated test output for problems."

Debian Continuous Integration

ci.debian.net

"How often are test suites executed?
The test suite for a source package will be executed:

  • when any package in the dependency chain of its binary packages changes;
  • when the package itself changes;
  • when 1 month is passed since the test suite was run for the last time."

Online Continuous Integration