Test Data Setup: Staying clean, DRY, and sane

There are many good reasons to avoid hitting a database in your tests. I agree with all of them, and I try my best to avoid doing it.

However, some tests do need to hit the database. Even the most dependency-injected and mock-infested system should hit the database when testing the data access layer... after all, what good is a test suite that doesn't test any of your actual data access logic? And if you're smart and follow the testing pyramid then you'll have some integration and acceptance tests that need a database as well.

In "Rules for Effective Data Tests" I mentioned some strategies for setting up those data tests. This post expands on those ideas and shows how to keep your setup code clean, DRY and maintainable.

What's so difficult about setting up a data test?

First, a definition. When I say "data setup" I'm talking about anything you do in the body of a test [or a setup method] to create the database records needed for a given test to execute.

While similar to the setup of a "true" unit test, interacting with a Real Life Database™ makes things a little more interesting. Some of the challenges we have to overcome are:

• Test residue: Unless we delete it, data created by each test remains in the database when the test exits. At best this just wastes space; at worst, it starts to interfere with other tests. (See here for a common solution to this problem) 
• Database constraints: Foreign key constraints are a real pain. When setting up test data you need to create the entire data graph to satisfy the database constraints, regardless of if those relationships are actually relevant to the test.  
• Verbosity: Because of the foreign key issues mentioned above, setting up data tests requires more code than setting up a unit test. This makes tests harder to write, harder to maintain, and harder to keep DRY. 
• False negatives: The more complex the setup, the greater the change that tests will fail not because your application logic is wrong, but because you screwed up the setup. 
• Painful to debug: Debugging a data test is more difficult and time consuming than a unit test. Not only does the test take longer to run, but debugging it often means poking around in both the application debugger and a database tool.

Characteristics of good setup code

The primary contributor to the quality and maintainability of your data tests is the setup code; the easier it is for someone to understand the specific scenario you are creating, the better equipped they are to maintain that test.

Conversely, the harder the scenario is to understand and maintain, the less value that test will provide over time. Tests that contain an unintelligible jumble of setup code have a very real risk of being deleted (rather than fixed) if they ever break due to new code changes.

So what is "good" setup code? It should be: 

• Highly expressive (high signal-to-noise ratio). Readers should be able to very quickly understand the scenario(s) you are creating without mentally parsing code. 
• Highly reusable through the use of default values. If I just need to create a Person, let me call "CreatePerson()" and fill in the details for me. 
• Easily customizable to each test's needs. Since the customized data are usually very relevant to the test at hand, it should be easy for a reader to spot them.  
• Maintainable; databases change, and its not uncommon to add a new required field. The fewer changes you need to make to existing test code to support these changes the better.

Data Helpers: the Object Mother pattern for DB entities

The Object Mother pattern describes a special kind of factory class that encapsulates the instantiation of an object (or group of objects) in a specific state, usually mirroring a common scenario in the underlying domain. For instance, you might have an Object Mother that creates an Order object, adds some Order Items and marks it as Shipped. The goal is to turn a complex initialization process into a one-liner so that it is easier to read and maintain.

Data Scenarios: a bunch of Object Mothers working together

Data Helpers are great when you need to create test data, especially if you want to specify a few key properties and use defaults for the rest.

But what if you want to create multiple related entities, or you want to reuse a specific setup in multiple tests? For instance, you need to create a Customer, with completed Orders in the past, and an in progress Order that's ready for checkout. In these cases, I create a special type of Data Helper that I call a "Data Scenario". 

Scenario objects have these characteristics:

• Create a large or complex set of data: Just like Data Helpers reduce individual object setup to a one-liner, Scenarios reduce multiple object setup to a one-liner.
• Model real-world scenarios: The whole point of a Scenario is to encapsulate realistic data patterns that might exist in production.
• Expose a smaller set of configurable defaults: Scenarios tend to expose fewer arguments than Data Helpers because they are better suited to creating general purpose groups of data rather than highly-specific records.
• Are often used in fixture-level setup: A common pattern is for a group of tests to share a Scenario object that is created in the test fixture's setup routine, and then provide test-specific adjustments to the Scenario via inline Data Helper calls. 
• Are instantiated, not static: Scenario objects are NOT static methods of a helper class. Instead, they are objects that get instantiated and perform their data manipulations in the constructor. This allows Scenarios to be created, manipulated and passed around as needed.
• Expose pointers to the interesting data: A Scenario object should contain public properties containing references to the entities it creates (or at least their IDs). This allows test code to further manipulate the Scenario data or to make assertions against it.