Test Driven Development The First Steps by

Ivan A Krizsan Version: June 9, 2009

Copyright 2009 by Ivan A Krizsan. All Rights Reserved.

1

Table of Contents Table of Contents..................................................................................................................................2 Purpose ................................................................................................................................................ 3 Licensing ............................................................................................................................................. 3 Disclaimers .......................................................................................................................................... 3 Prerequisites......................................................................................................................................... 4 Creating User Libraries in Eclipse....................................................................................................... 4 Creating the Project.............................................................................................................................. 6 Developing the Example...................................................................................................................... 8 Background...................................................................................................................................... 8 Packages...........................................................................................................................................8 First Iteration....................................................................................................................................9 Second Iteration............................................................................................................................. 19 Third Iteration................................................................................................................................ 21 TestNG Test Suites............................................................................................................................. 25 Code Coverage................................................................................................................................... 26 Installing EclEmma........................................................................................................................26 Examining Code Coverage............................................................................................................ 27 Further Inspiration.............................................................................................................................. 29

2

Purpose This document contains a first introduction to test-driven development using TestNG and EasyMock. It is aimed at people that are at least slightly familiar with developing Java in Eclipse, but that have no experience with test-driven development.

Licensing This document is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 License.

Disclaimers Though I have done my best to avoid it, this document might contain errors. I cannot be held responsible for any effects caused, directly or indirectly, by the information in this document – you are using it on your own risk. Submitting any suggestions, or similar, the information submitted becomes my property and you give me the right to use the information in whatever way I find suitable, without compensating you in any way. All trademarks in this document are properties of their respective owner and do not imply endorsement of any kind. This document has been written in my spare time and has no connection whatsoever with my employer.

3

Prerequisites This example assumes that you are familiar with basic Java development in Eclipse. Using NetBeans or some other IDE is also possible, but may require slightly more knowledge on your part. Besides an IDE, you will also need to download the following libraries: • TestNG TestNG is a testing framework that enables you to write unit tests. Besides downloading the TestNG libraries, I also strongly suggest installing the Eclipse plugin as described on the TestNG download page. • EasyMock EasyMock is a tool that will help you to create so-called mock objects on the fly, in your unit tests. Don't worry if it is not clear what this means, it will be easier to understand once you have worked through this introduction. After having downloaded and unpacked the libraries, place them in a location of your choice, for instance somewhere where you have other libraries and frameworks.

Creating User Libraries in Eclipse To make life more convenient, we will create user libraries for TestNG and EasyMock in Eclipse. This way, you can conveniently include these libraries in any project without having to copy the JAR files every time. • In Eclipse, open the Preferences. • In the text field in the upper left corner, type “user libraries” without quotes. • Click on the node Java -> Build Path -> User Libraries. • Click the New button in upper right corner and enter “TestNG” as the user library name and click the OK button.

4

• •

With the TestNG user library selected, click the Add JARs button on the right. Navigate to the location of the TestNG libraries and select the JAR file with the name “testng-5.9-jdk15.jar”. If you, by any chance, want to use TestNG with Java 1.4, then you should select the “testng5.9-jdk14.jar” file. The result should look something like this:

•

Repeat the process for EasyMock. The result should look something like this:

As an exercise for the reader, which may be skipped, source-code location and JavaDoc location for the above two libraries can also be assigned.

5

Creating the Project Now we'll create an Eclipse project in which to develop the example. • Select New → Java Project in the File menu. • Enter the name of the project in the Project Name field. I will call my project “TestDrivenExample”. • In the same dialog, select the Java Runtime Environment (JRE). I will use the Java 6 runtime, but the Java 5 runtime should work equally well. • Click the Next button at the bottom of the dialog. • Click the Libraries tab in the new dialog. • Click the Add Library... button to the right in the dialog. • Select the TestNG and EasyMock libraries we created above and click the Finish button.

6

•

The result in the New Java Project dialog should look like this:

•

Click Finish to finish the creation of the project.

7

Developing the Example Now we are ready to start developing our program. This example will implement a service that evaluates simple mathematical expressions in RPN form – a calculator service.

Background An example of an expression written using RPN (Reverse Polish Notation), which is also called postfix notation, form is: 23*5+

The above expression is equal to the following expression written using the more popular infix notation: 2*3+5

The both expressions gives the result 11.

Packages In order to set up the project for development, we create three packages: • com.ivan.services This package will contain interfaces for the services we will develop. • com.ivan.services.impl This package will contain implementations of the interfaces in the above package. • com.ivan.services.test This package will contain tests that tests the services.

8

First Iteration In the first iteration, we will implement support for processing additions in the calculator.

The Calculator Interface We'll start from the top by defining an interface for our calculator service. It is also possible to write the test first and then define interfaces and implement functionality. /* * Calculator.java * Copyright (c) 2009 by Ivan A Krizsan. All rights reserved. */ package com.ivan.services; /** * This interface defines the available functions of the calculator * service that can evaluate RPN (reverse polish notation) expressions. * * @author Ivan A Krizsan */ public interface Calculator { /* Constant(s): */ /** * Evaluates the supplied RPN expression and return the result. * * @param inExpression RPN expression. * @return Value(s) after having performed operations in the * expression as a space-separated string where the rightmost * value is the topmost value on the stack. */ public String evaluate(final String inExpression); }

9

The Test Having an interface, but still no implementation, we write our first test. The test is initially very simple, but will be extended as we refine the calculator. /* * CalculatorTest.java * Copyright (c) 2009 by Ivan A Krizsan. All rights reserved. */ package com.ivan.services.test; import org.testng.Assert; import org.testng.annotations.Test; import com.ivan.services.Calculator; /** * This class contains the tests of the calculator service. * * @author Ivan A Krizsan */ @Test(groups="services") public class CalculatorTest { /* Constant(s): */ private final static String EXPRESSION1 = "1 2 +"; private final static String EXPRESSION1_RESULT = "3"; /* Instance variable(s): */ /** * Tests evaluation of a simple expression containing only * one addition. */ public void testAddition() { Calculator theCalculator = new CalculatorImpl(); String theResult = theCalculator.evaluate(EXPRESSION1); Assert.assertTrue(EXPRESSION1_RESULT.equals(theResult)); } }

Note that I have, in the @Test annotation, assigned a group to the test. We'll use that later when we want to create a suite of tests. In the testAddition method, we use the Assert tool class from TestNG which helps us verify things about our program. In the case above, we verify that the constant containing the correct answer of the evaluation is equal to the result produced by the calculator service. For more information about what methods are available in the Assert class, please consult the TestNG JavaDoc.

10

The Calculator Implementation As of now, the above test does not even compile properly since there is no class CalculatorImpl. Eager to run our test, we'll rectify that immediately by providing a first implementation of the calculator service. /* * CalculatorImpl.java * Copyright (c) 2009 by Ivan A Krizsan. All rights reserved. */ package com.ivan.services.impl; import com.ivan.services.Calculator; /** * This class provides the implementation of the calculator * service. * To evaluate expressions, the calculator uses a number of * separate services; one for addition, one for subtraction etc. * * @author Ivan A Krizsan */ public class CalculatorImpl implements Calculator { /* Constant(s): */ /* Instance variable(s): */ @Override public String evaluate(final String inExpression) { // TODO Auto-generated method stub return null; } }

Of course this won't work, since the evaluate method just returns null, but we'll settle for this implementation for the moment. Test-driven development can be summarized by one single sentence:

Only write code to fix a failing test. So, let's see if the test fails or passes!

Organizing Imports in Eclipse Before being able to run the test, we must organize the imports of the test class so that it becomes aware of the CalculatorImpl class we just implemented. This can be accomplished in three ways: • Right-click in the CalculatorTest.java editor in Eclipse and select Source → Organize Imports.

•

Go to the Source menu and select Organize Imports. 11

•

Use the keyboard shortcut, which can be seen in the Source menu.

Running the Test Finally, we can run the test! With the TestNG plug-in installed in Eclipse, TestNG unit tests will automatically be recognized in the Run As menus. • Right-click the CalculatorTest.java file in the package explorer. • Select Run As → TestNG Test. After a little while, the TestNG view will appear showing a red bar and telling us one test has failed.

Re-Running Tests

There are two very useful little buttons in the above view:

The left one can be used to re-run the last test, including all of the test methods in the test. The right one is used to re-run only the test methods that failed in the last test. Finding the Cause of the Failure

To find out why the test failed, click on a test method that is tagged with a small red box with an X in it. In the lower part of the TestNG view, the failure exception that caused the test method in question to fail appears:

Not surprisingly, we find that the assertion that compared the expected result from the calculator to 12

the actual result the calculator produced failed, since the two results were not equal. Double-clicking on a line in the Failure Exception view takes you to that line of code, if the sourcecode in question is available.

Fixing the Calculator Implementation Perhaps one would implement all the functionality in the calculator implementation class, but for the sake of learning about test-driven development, we will delegate the responsibility for all calculations to other services. Since the first test contains an addition, we'll first use an addition service described by the following interface: /* * Addition.java * Copyright (c) 2009 by Ivan A Krizsan. All rights reserved. */ package com.ivan.services; /** * This interface defines the properties of a service that * adds numbers. * * @author Ivan A Krizsan */ public interface Addition { /* Constant(s): */ /** * Adds the two supplied numbers and returns the sum. * Does not perform any boundary checking. * * @param inNum1 First number to add. * @param inNum2 Second number to add. * @return Sum of supplied numbers. */ public long add(final long inNum1, final long inNum2); }

Now we can modify the calculator implementation so that it uses an addition service to add numbers. The new version of the CalculatorImpl class looks like this: /* * CalculatorImpl.java * Copyright (c) 2009 by Ivan A Krizsan. All rights reserved. */ package com.ivan.services.impl; import java.util.Stack; import com.ivan.services.Addition; import com.ivan.services.Calculator; /** * This class provides the implementation of the calculator * service. * To evaluate expressions, the calculator uses a number of * separate services; one for addition, one for subtraction etc. * * @author Ivan A Krizsan */ public class CalculatorImpl implements Calculator { /* Constant(s): */ private final static String NUMBERS_REGEXP = "[\\-]*[0-9]+"; private final static String OPERATORS_REGEXP = "[\\+\\-*/]"; /* Instance variable(s): */ private Stack mNumbersStack = new Stack(); private Addition mAdditionService; @Override public String evaluate(final String inExpression)

13

{ String theParts[]; /* Split the expression into parts; numbers and operands. */ theParts = inExpression.split(" "); for (int i = 0; i < theParts.length; i++) { theParts[i] = theParts[i].trim(); } for (int i = 0; i < theParts.length; i++) { if (theParts[i].matches(NUMBERS_REGEXP)) { /* * The part is a number. Parse it and put it on the * stack. */ Long theNumber = Long.parseLong(theParts[i]); mNumbersStack.push(theNumber); } else if (theParts[i].matches(OPERATORS_REGEXP)) { /* The part is an operator. */ char theOperator = theParts[i].charAt(0); /* Get the numbers the operator is to operate on. */ Long theNumber1 = mNumbersStack.pop(); Long theNumber2 = mNumbersStack.pop(); Long theResult; /* Let the appropriate service perform the operation for us. */ switch (theOperator) { case '+': theResult = mAdditionService.add(theNumber1, theNumber2); break; default: throw new IllegalArgumentException( "Unrecognized operand: " + theParts[i]); } mNumbersStack.push(theResult); } else { throw new IllegalArgumentException("Illegal token in expression: " + theParts[i]); } } /* * Create the result string, which consists of the space-separated * values on the stack. */ String theResultString = ""; while (!mNumbersStack.isEmpty()) { theResultString += mNumbersStack.pop().toString(); if (!mNumbersStack.isEmpty()) { theResultString += " "; } } return theResultString; } /** * Sets the addition service to be used by the calculator. * * @param inAdditionService Service performing additions * for the calculator. */ public void setAdditionService(final Addition inAdditionService) { mAdditionService = inAdditionService; }

14

}

15

Re-Running the Test With the new implementation of the calculator in place, we can now re-run the test. Surprisingly(?), the test fails again. If we look at the Failure Exception section of the TestNG view, we can see that the test fails for a different reason:

A NullPointerException occurred on line 65 in the CalculatorImpl class. Looking at the CalculatorImpl class, we see the following:

Ah, the reason for the NullPointerException is that we do not have an addition service helping the calculator to do additions! As you may have noticed, we never created an addition service in the calculator service. Instead we implemented a method called setAdditionService, a setter-method, to allow the addition service to be injected into the calculator service. The reason for doing it this way is that it makes the ties, the coupling, between the different service less strong and it is easier to test the code, as we will see in the next section.

Mock Objects When testing a class like the CalculatorImpl class, we want to test it in isolation from the rest of the system, in this case the addition service. One approach is to write a fake addition service, a stub, that always returns some fixed value. Another approach is to use mock objects. A mock addition service looks, to the calculator service, like a genuine addition service. Additionally, the mock object is able to record and verify calls that were made on the mock object. This enables us to verify the interaction between the object under test and other parts of the program it depends on. Mock objects are generated on the fly, so we don't need to write them, just write the code that creates them. Enough theory, let's update the test to inject a mock addition service.

16

The new version of the CalculatorTest class now looks like this: /* * CalculatorTest.java * Copyright (c) 2009 by Ivan A Krizsan. All rights reserved. */ package com.ivan.services.test; import import import import import import

org.easymock.Capture; org.easymock.CaptureType; org.easymock.EasyMock; org.easymock.IMocksControl; org.testng.Assert; org.testng.annotations.Test;

import com.ivan.services.Addition; import com.ivan.services.Calculator; import com.ivan.services.impl.CalculatorImpl; /** * This class contains the tests of the calculator service. * * @author Ivan A Krizsan */ @Test(groups = "services") public class CalculatorTest { /* Constant(s): */ private final static long NUM1 = 1; private final static long NUM2 = 2; private final static String EXPRESSION1 = "" + NUM1 + " " + NUM2 + " +"; private final static long EXPRESSION1_RESULT = 3; /* Instance variable(s): */ /** * Tests evaluation of a simple expression containing only one * addition. */ public void testAddition() { Capture theParamCapt = new Capture(CaptureType.ALL); /* Create a mock addition service and set expectations. */ IMocksControl theMockControl = EasyMock.createNiceControl(); Addition theMockAdditionService = theMockControl.createMock(Addition.class); /* * When the addition service is called, it will capture the * parameters and always return 3. */ EasyMock.expect( theMockAdditionService.add(EasyMock.capture(theParamCapt), EasyMock .capture(theParamCapt))).andReturn(EXPRESSION1_RESULT); /* Tell EasyMock we are done specifying expectations. */ theMockControl.replay(); /* Create and setup the object under test, the calculator service. */ CalculatorImpl theCalculator = new CalculatorImpl(); theCalculator.setAdditionService(theMockAdditionService); /* Invoke the calculator service. */ String theResult = theCalculator.evaluate(EXPRESSION1); /* Verify the result returned by the calculator service. */ Assert.assertEquals(theResult, "" + EXPRESSION1_RESULT); /* * Verify the parameters sent to the addition service. * This is rather unnecessary and make the test too * inflexible, but I wanted to show how to capture * parameters in method calls with EasyMock. */ long theParamSum = 0; for (Long theParam : theParamCapt.getValues())

17

{ theParamSum += theParam; } Assert.assertEquals(theParamSum, EXPRESSION1_RESULT); /* Verify the way the calculator service used the addition service. */ theMockControl.verify(); } }

When we now re-run the test, it now passes!

The green bar is the goal we strive for when doing test-driven development.

18

Second Iteration Tests should not only be written for successful scenarios, so-called happy-path scenarios, but also to provoke failures in the software being developed. The reasons for this is that the software being developed may, for instance, receive bad input from a user and should, despite this, behave in a predefined way. For instance, the person writing a program that uses our calculator service will probably want to know what exceptions to expect if there are problems with the input. To test the calculator service with bad input, we add the following code to the CalculatorTest class: ... /* Constant(s): */ ... private final static String EMPTY_EXPRESSION = ""; private final static String BAD_EXPRESSION1 = "1 +"; private final static String BAD_EXPRESSION2 = "test"; private final static String BAD_EXPRESSION3 = "1 b +"; ... /** * Tests evaluation of bad expressions. */ public void testBadExpression() { /* * Create and setup the object under test, the calculator service. * We do not need a mock addition service, since none of the * expressions should cause the addition service to be invoked. */ CalculatorImpl theCalculator = new CalculatorImpl(); /* * Try evaluating the bad expressions. * Note how the test also has been refactored to avoid * repetition of code. */ evaluateBadExpression(theCalculator, EMPTY_EXPRESSION); evaluateBadExpression(theCalculator, BAD_EXPRESSION1); evaluateBadExpression(theCalculator, BAD_EXPRESSION2); evaluateBadExpression(theCalculator, BAD_EXPRESSION3); } private void evaluateBadExpression(final CalculatorImpl inCalculator, final String inBadExpression) { boolean theExceptionFlag = false; try { inCalculator.evaluate(inBadExpression); /* * The evaluation of bad expressions should always cause * exception to be thrown, so the assertion below should * never be reached. * Note that we can add a message to the assertion that * will be shown if the assertion fail. */ Assert.assertTrue(false, "Bad expressions should cause exception"); } catch (final IllegalArgumentException theException) { theExceptionFlag = true; } Assert.assertTrue(theExceptionFlag); }

Running the modified test, we can see that it fails in the new test-method with a EmptyStackException when evaluating BAD_EXPRESSION1. Exercise for the reader: Make sure you understand how to determine which expression causes the test to fail.

19

Modify the evaluateBadExpression method in the test to look like this: ... private void evaluateBadExpression(final CalculatorImpl inCalculator, final String inBadExpression) { boolean theExceptionFlag = false; try { inCalculator.evaluate(inBadExpression); /* * The evaluation of bad expressions should always cause * exception to be thrown, so the assertion below should * never be reached. * Note that we can add a message to the assertion that * will be shown if the assertion fail. */ Assert.assertTrue(false, "Bad expressions should cause exception"); } catch (final IllegalArgumentException theException) { theExceptionFlag = true; } catch (final EmptyStackException theException) { theExceptionFlag = true; } Assert.assertTrue(theExceptionFlag); } ...

If we now re-run the test, it passes with a green bar! We can now modify the comment in the Calculator interface to document the behaviour of the calculator when being supplied bad expressions. The new version of the interface looks like this: /* * Calculator.java * Copyright (c) 2009 by Ivan A Krizsan. All rights reserved. */ package com.ivan.services; import java.util.EmptyStackException; /** * This interface defines the available functions of the calculator * service that can evaluate RPN (reverse polish notation) expressions. * * @author Ivan A Krizsan */ public interface Calculator { /* Constant(s): */ /** * Evaluates the supplied RPN expression and return the result. * Allowed characters in expressions are digits 0-9 forming * integer numbers and operators +, -, * and /. * * @param inExpression RPN expression. * @return Value(s) after having performed operations in the * expression as a space-separated string where the rightmost * value is the topmost value on the stack. * @throws IllegalArgumentException If expression contains * illegal characters. * @throws EmptyStackException If not enough numbers available * on the stack for an operator. For instance, + needs two numbers * in order to be able to produce a sum. */ public String evaluate(final String inExpression) throws IllegalArgumentException, EmptyStackException; }

Alternatively, one or more custom exceptions can be introduced, if you want to isolate clients of the service from the above unchecked exceptions. 20

Third Iteration In this iteration, we will implement the addition service. Since the implementation is trivial, we will take the opportunity to look at some features of TestNG, namely: • Setup and cleanup methods. By annotating a method in a test class, you can cause it to be run before the test, before each method in a test or before an entire test suite. This can be useful to initialize resource etc. that the test depends on. There are other annotations that cause a method in the test class to execute after the test, after each method in a test or after an entire test suite. • Data providers. Data providers can provide parameters to test methods. The test method will be executed for each entry in an array of data returned by the data provider. • Execution of a test method multiple times and in parallel. Since we already have implemented the interface of the addition service, Addition, we can go straight on to implementing the test of the addition service implementation. /* * AdditionTest.java * Copyright (c) 2009 by Ivan A Krizsan. All rights reserved. */ package com.ivan.services.test; import java.util.Random; import import import import import

org.testng.Assert; org.testng.annotations.AfterTest; org.testng.annotations.BeforeTest; org.testng.annotations.DataProvider; org.testng.annotations.Test;

import com.ivan.services.Addition; import com.ivan.services.impl.AdditionImpl; /** * This class contains the tests of the addition service. * * @author Ivan A Krizsan */ @Test(groups = "services") public class AdditionTest { /* Constant(s): */ /* Instance variable(s): */ private Addition mAdditionService; private Random mRandom; /* Data provider for addition test. */ @DataProvider(name = "additionTestDataProvider") public Long[][] createAdditionTestData() { /* * Note how auto-boxing helps us to create Long objects from * the supplied numbers. */ return new Long[][] { { 1L, 2L, 3L }, { -1L, -2L, -3L }, { 0L, 0L, 0L },

21

{ Long.MAX_VALUE, Long.MIN_VALUE, -1L }, { /* * These test values shows how the number "wraps" when * exceeding the max value a long can hold. It can be * questioned if this is correct behaviour. */ Long.MAX_VALUE, 1L, Long.MIN_VALUE } }; } /** * Performs initialization of things the test depends on prior to * the test. */ @BeforeTest public void setup() { mAdditionService = new AdditionImpl(); mRandom = new Random(System.currentTimeMillis()); } /** * Performs cleanup after the test. */ @AfterTest public void cleanup() { mAdditionService = null; mRandom = null; } /** * Tests addition of numbers provided by data provider named in * the @Test annotation. * This test shows how to use a data provider providing the parameters * of the test method and how to perform multi-threaded testing. * This test is executed 10 times by 5 threads executing in parallel. * * @param inNum1 First number to add. * @param inNum2 Second number to add. * @param inCorrectSum Correct sum of two above numbers. */ @Test(dataProvider = "additionTestDataProvider", threadPoolSize = 5, invocationCount = 10, sequential=false) public void testAddition(final long inNum1, final long inNum2, final long inCorrectSum) { Assert.assertEquals(mAdditionService.add(inNum1, inNum2), inCorrectSum); /* * Insert a random delay in an attempt to make the order * in which the tests are executed less sequential. */ try { Thread.sleep(Math.abs(mRandom.nextLong()) % 20L); } catch (final InterruptedException theException) { // Ignore exceptions. } } }

22

With the test in place, we can implement the addition service: /* * AdditionImpl.java * Copyright (c) 2009 by Ivan A Krizsan. All rights reserved. */ package com.ivan.services.impl; import com.ivan.services.Addition; /** * This class provides the implementation of the addition * service. * * @author Ivan A Krizsan */ public class AdditionImpl implements Addition { /* Constant(s): */ /* Instance variable(s): */ @Override public long add(final long inNum1, final long inNum2) { return inNum1 + inNum2; } }

Running the addition test, we see it passes with a green bar. We can also see that the testAddition method is invoked 50 times. The TestNG view will list all invocation, as well the data from the data provider that was supplied as parameters to the test method when it was invoked.

23

This concludes the development of the calculator service done in this document. The following are left as exercises for the reader: • Additional tests for the calculator service. Subtraction, division, multiplication and compound expressions. • Test, interface and service implementation for the subtraction service. • Test, interface and service implementation for the division service. • Test, interface and service implementation for the multiplication service.

24

TestNG Test Suites When annotating the test classes, we supplied a group name in the annotation: ... @Test(groups="services") ...

Tests belonging in one or more groups can be further grouped into a test suite. A test suite can then be launched, causing all the tests in the suite to be run. Test suites can be very useful if the program you develop is larger. Perhaps you are working on developing services. It is likely that you will run all the tests in the services group, but you will not be interested tests in other group for the moment. A test suite enable you to run only the tests in the services group and thereby helping you to focus on the task at hand and avoid unnecessary delays waiting for other tests to finish. Let's write a test suite that executes all the tests in the services group in the example project: • Create a package named test in source of the Eclipse project. • In the new package, create a file named services_testsuite.xml. • Enter the following as the content of the newly created file:

• •

Save the file. Right-click on the services_testsuite.xml file in the Package Explorer view in Eclipse and select Run As → TestNG Suite. In the TestNG view we can see that tests in the classes AdditionTest and CalculatorTest both have been run.

25

Code Coverage In this final section of this tutorial, we are going to take a look at a tool that comes in very handy when doing test-driven development. The tool is named EclEmma and is used to determine what parts of your code has been executed and what parts have not been executed. This is very useful when writing tests, since you want the tests to test all parts of your code.

Installing EclEmma If you haven't already installed the Eclipse plug-in EclEmma, then, in Eclipse, follow the procedure below: • Select the Help → Software Updates menu item. • Click the Add Site button. • Paste the following URL in the Location field: http://www.eclemma.org/ • Make sure the checkbox next to the EclEmma update site URL is checked. • Click the Install button and follow the instructions. • Restart Eclipse, if necessary. For older versions of Eclipse and alternative installation methods, please see http://www.eclemma.org/installation.html. If the EclEmma plug-in was successfully installed, you should see a new button, circled in red in the picture below, in the shortcuts menu bar:

This is the Coverage As launch button. In the Run menu, there will be three new menu items at the top of the menu:

These three menu items are alternatives to the button in the toolbar.

26

Examining Code Coverage Now we are ready to examine the code coverage of our tests. • In the Run menu, select Coverage History → src.test.services_testsuite.xml

•

When all the tests have finished running, the Coverage view will display information on how large a percent of each package, each class and each method in the project that were executed. With all nodes expanded, the Coverage view looks something like this:

•

We can see that roughly 86% of the code in the CalculatorImpl class was executed during testing. This is something we want to investigate further... Double-click on the CalculatorImpl.java node in the Coverage view.

•

27

•

Scroll down in the CalculatorImpl.java file until you see the two blocks of code coloured in red as in the figure below.

•

The red colour means that a code block has not been executed, the green colour means that the marked code has been executed. Note! EclEmma does not always mark blocks of code with the correct colour, as an example see the method evaluateBadExpression in the class CalculatorTest. The first row that assigns false to the variable theExceptionFlag is marked in red, despite the fact that the method must have been invoked. From the red code in the CalculatorImpl class, we can draw the following conclusions: - Our tests never submits an expression to the calculator with an illegal operand. - Our tests never submit an expression to the calculator which causes more than one value to be left on the stack. Having drawn the above conclusions, we may want to implement further tests to exercise the above cases.

•

•

A question that tend to arise in a situation like this is: Is 100% code coverage a goal to strive for? I feel that it is important to balance the effort of writing tests with the return of investment it gives. For instance, I never write tests for methods that are trivial, like getters and setters, but it is a matter of personal taste and I won't take a stand in this question.

28

Further Inspiration I am standing on the shoulders of giants when writing this document. The following links may be sources of further information and inspiration concerning test-driven development: • http://testng.org TestNG unit testing framework for Java. • http://www.junit.org/ JUnit testing framework for Java. • http://www.eclemma.org/ Code coverage tool for Java. • http://en.wikipedia.org/wiki/Test-driven_development The Wikipedia article on test-driven development. • http://easymock.org/ Mock object generation library for Java. • http://code.google.com/p/powermock/ PowerMock further extends, among others, EasyMock, with further capabilities aimed at making it easier to write tests. • http://code.google.com/p/hamcrest/ Hamcrest provides a library of matchers that are very useful when writing assertions for unit tests. • http://www.dbunit.org/ DBUnit is a JUnit extension aimed at making it easier to write unit tests that depend on a database being in a known state. • http://httpunit.sourceforge.net/ HTTPUnit makes it possible to automate the testing of web applications that provide a user interface in a web browser. • http://www.soapui.org/ Testing tool for web services that is able to generate unit tests. • http://www.jboss.org/jsfunit/ JSFUnit is a test framework for testing web applications with a JavaServer Faces user interface. My apologies to those I have forgotten to include in this list.

29