For my software engineering course, I am creating a very simple Roman to Arabic numeral converter. (If anyone knows what the unicode character for Roman numeral 5000, a V with a line over it, I would appreciate it for accuracy.) It isn’t the converter program that is important. The purpose is to demonstrate black box testing methodologies: testing without knowledge of the internals of a program.

I developed a test plan and wrote the test cases in JUnit, as instructed, before proceeding on to writing the actual converter. I then ran my tests. A couple failed and a hand full had an error. I rewrote the code; the class is small enough to justify it, rather than worry about where my sometimes failing  bug was.

Yet, I still had the same problems. So, I copied the short bits of code into a main() function to use the debugger. The debugger revealed no faults: my system produced the correct output. JUnit had produced a false negative. I didn’t know it could even do that. It did not occur across similar tests with identical assert statements, so I’m not sure what it could be.