Run a JUnit test repeatedly

I like JUnit. It's easy to extend, and easy to use.
After reading abyx's post on Retry Rule, I decided to write a post about running repeating tests with JUnit.


Hope you enjoy this:


Sometimes you wish to run a test many times. For example, it might have some random factor in it. To cover many cases, you'd like to run it a lot of times. What options are there?

First, and the most obvious, is a loop. Run a for loop, and see it fail:

 public class ExampleTest {  
      @Test  
      public void sometimesFail() {  
           for (int i = 0; i < 10; i++) {  
                int rand = new Random().nextInt(3);  
                if (rand % 3 == 0) {  
                     fail();  
                }  
           }  
      }  
 }  

This will generate a single test in the tree. It will fail on the first error and won't give you an assessment of how many times it took till it failed.


Another option is the Parametrized Runner. That would look like this:

 @RunWith(Parameterized.class)  
 public class ExampleTest2 {  
      @Parameters  
      public static Collection<Object[]>
                                 generateParams() {  
           List<Object[]> params =
                   new ArrayList<Object[]>();  
           for (int i = 1; i <= 10; i++) {  
                params.add(new Object[] {i});  
           }  
           return params;  
      }  
        
      public ExampleTest2(int param) {}  
        
      @Test  
      public void sometimesFail() {  
           int rand = new Random().nextInt(3);  
           if (rand % 3 == 0) {  
                fail();  
           }  
      }  
 }  

This is a nice solution, but the test is filled with unnecessary code. Also, all the tests in the class will run for each parameter. What if I only want to repeat a single method?


Here is a nice solution, in my opinion:

 @RunWith(ExtendedRunner.class)  
 public class ExampleTest3 {  
      @Test  
      @Repeat(10)  
      public void sometimesFail() {  
           int rand = new Random().nextInt(3);  
           if (rand % 3 == 0) {  
                fail();  
           }  
      }  
 }  

 Only state the method you want to repeat, and how many times to do this. The rest is done for you.


So, whats behind this code?
First, add an Annotation:

 @Retention(RetentionPolicy.RUNTIME)  
 @Target({ElementType.METHOD})  
 public @interface Repeat {  
      int value();  
 }  

Finally, lets add the Runner. This simply overrides the default JUnit runner (its a bit long, but not too much):

 public class ExtendedRunner extends BlockJUnit4ClassRunner {  
   
      public ExtendedRunner(Class<?> klass) throws InitializationError {  
           super(klass);  
      }  
   
      @Override  
      protected Description describeChild(FrameworkMethod method) {  
           if (method.getAnnotation(Repeat.class) != null &&  
                     method.getAnnotation(Ignore.class) == null) {  
                return describeRepeatTest(method);  
           }  
           return super.describeChild(method);  
      }  
   
      private Description describeRepeatTest(FrameworkMethod method) {  
           int times = method.getAnnotation(Repeat.class).value();  
   
           Description description = Description.createSuiteDescription(  
                     testName(method) + " [" + times + " times]",  
                     method.getAnnotations());  
   
           for (int i = 1; i <= times; i++) {  
                description.addChild(Description.createTestDescription(  
                          getTestClass().getJavaClass(),  
                          "[" + i + "] " + testName(method)));  
           }  
           return description;  
      }  
   
      @Override  
      protected void runChild(final FrameworkMethod method, RunNotifier notifier) {  
           Description description = describeChild(method);  
             
           if (method.getAnnotation(Repeat.class) != null &&  
                     method.getAnnotation(Ignore.class) == null) {  
                runRepeatedly(methodBlock(method), description, notifier);  
           }  
           super.runChild(method, notifier);  
      }  
   
      private void runRepeatedly(Statement statement, Description description,  
                RunNotifier notifier) {  
           for (Description desc : description.getChildren()) {  
                runLeaf(statement, desc, notifier);  
           }  
      }  
        
 }  

As always, here is the source code: http://tinyurl.com/3mu2zbc
Also, you can Search Amazon.com for JUnit books.