Sunday, 11 November 2012

Testing legacy code with Golden Master

As a warm up for SCNA, the Chicago Software Craftsmanship Community ran a hands-on coding session where developers, working in pairs, should test and refactor some legacy code. For that they used the Gilded Rose kata. You can find links to versions in java, C# and ruby here and for clojure here.

We ran the same session for the London Software Craftsmanship Community (LSCC) early this year and back then I decided to write my tests BDD-style (I used JBehave for that). You can check my solution here.

This time, instead of writing unit tests or BDD / Spec By Example to test every branch of that horrible code, I decided to solve it using a test style called Golden Master.

The Golden Master approach

Before making any change to the production code, do the following:
  1. Create X number of random inputs, always using the same random seed, so you can generate always the same set over and over again. You will probably want a few thousand random inputs.
  2. Bombard the class or system under test with these random inputs.
  3. Capture the outputs for each individual random input
When you run it for the first time, record the outputs in a file (or database, etc). From then on, you can start changing your code, run the test and compare the execution output with the original output data you recorded. If they match, keep refactoring, otherwise, revert back your change and you should be back to green.

Approval Tests

An easy way to do Golden Master testing in Java (also available to C# and Ruby) is to use Approval Tests. It does all the file handling for you, storing and comparing it. Here is an example:

For those not familiar with the kata, after passing a list of items to the GildedRose class, it will iterate through them and according to many different rules, it will change their "sellIn" and "quality" attributes.

I've made a small change in the Item class, adding a automatically generated toString() method to it:
The first time the test method is executed, the line:


will generate a text file, in the same folder where the test class is, called: GildedRoseTest.should_generate_update_quality_output.received.txt. That mean, ..received.txt

ApprovalTests then will display the following message in the console:

To approve run : mv /Users/sandromancuso/development/projects/java/gildedrose_goldemaster/./src/test/java/org/craftedsw/gildedrose/GildedRoseTest.should_generate_update_quality_output.received.txt /Users/sandromancuso/development/projects/java/gildedrose_goldemaster/./src/test/java/org/craftedsw/gildedrose/GildedRoseTest.should_generate_update_quality_output.approved.txt

Basically, after inspecting the file, if we are happy, we just need to change the .received with .approved to approve the output. Once this is done, every time we run the test, ApprovalTests will compare the output with the approved file.

Here is an example of how the file looks like:

Item [name=Aged Brie, sellIn=-23, quality=-44]
Item [name=Elixir of the Mongoose, sellIn=-9, quality=45]
Item [name=Conjured Mana Cake, sellIn=-28, quality=1]
Item [name=Aged Brie, sellIn=10, quality=-2]
Item [name=+5 Dexterity Vest, sellIn=31, quality=5]

Now you are ready to rip the GildedRose horrible code apart. Just make sure you run the tests every time you make a change. :)

If you are using Eclipse or IntelliJ, you can also use Infinitest. It automatically runs your tests every time you save a production or test class. It is smart enough to run just the relevant tests and not the entire test suite.  In Eclipse, it displays a bar at the bottom-left corner that can be red, green or yellow (in case there are compilation errors and the tests can't be run).

With this, approach, refactoring legacy code becomes a piece of cake. You make a change, save it, look at the bar at the bottom of the screen. If it is green, keep refactoring, if it is red, just hit CTRL-Z and you are back in the green. Wonderful. :)


Thanks to Robert Taylor and Balint Pato for showing me this approach for the first time in one of the LSCC meetings early this year. It was fun to finally do it myself.


Robert Taylor said...

Ctrl-Z or git reset --hard. You're committing regularly, right? ;)

Sandro Mancuso said...

Of course I was. I just did not mention. :)

Not THE Harry Harrison said...

When you have to fix a bug in the output, then you have to recapture the golden output data, yes?

Sandro Mancuso said...

Yes. If the behaviour of the code changes, you also need to regenerate them. Ideally you use golden master as a safety net to refactor the code and write unit tests. Once the code is clean and unit tested you could probably ignore the golden master.

sanlaville said...

Thanks for sharing.

It's also possible to use Approvals.verifyAll("Item", items);

In this case, you don't need the method getStringRepresentationFor((List items) anymore.

Post a Comment