In a previous post, I explored an idea of using Microsoft's built-in T4 code-gen framework to generate BDD-style tests from a simple text specification. Since then, I've improved the T4 some and also reevaluated the idea some. First, let's look at the updated code-gen. My goals from the last post were:
- A very simple, 'traditional' class where the test writers 'fill in the blanks'.
- intellisense to help discover the methods to implement or override
- throw a 'Not Implemented Exception' for any feature not yet implemented.
- the nice HTML report MSpec generates.
To that, I also decided it would be nice to update the tests whenever the spec was updated. I ended up dropping the HTML report, but adding this feature. There are two ways I can think of to do this. One would be to maintain a "generated" class, which the tests would inherit and override test methods from. This, though, has the downside of requiring more typing from the developers. Since NUnit doesn't run [Test]s from a base class, developers would have to essentially re-write the entire base class, overriding each test method and marking them with the test attribute, defeating the purpose of code generating altogether. The second approach is to create a generated class, and update it with test methods as needed. This is tricky, but doable, and the approach I ended up taking. Let's look at some code.
The spec, as last time:
Since T4 now needs to read the spec and either a) create a new file or b) update an existing one, it becomes more complex. I broke it up into a couple of methods for generating the code:
This code reads the .spec and generates unit tests, but works differently in that it renders the output to a file instead of relying on the Visual Studio tooling to output the file. If the file exists, it checks that it isn't read-only and looks for a comment "//Additional specification methods go here". If it finds it, it inserts any new test methods in that space. If the method already exists, then it is not overwritten. Because of the way T4 works, these methods have to be called from a second tt file:
The effect is that the code gen works to keep the test class up-to-date with the spec, but doesn't overwrite any of the developer's code. The end result looks like this:As you can see, developers have only to "fill in the blanks" for each TODO (Thanks Alex for the suggestion!).
All that said, I've started to rethink the benefit of this some. It's cool, but I'm not sure how much it actually saves over just writing the class from scratch. Especially with some code templates and ReSharper, it just wouldn't take much more effort to write the tests than it does to write the .spec and then generate the code. Then again, the .spec is something you could type up in a meeting with the product owners, so you may as well generate some code from it. So, I'm still on the fence about the actual usefulness of this, but it's been worth it for the T4 learning experience alone. At this point, I'm going to wait for a project it makes sense to try this on before going much further with it. But do let me know if you have any questions or suggestions for this, or if you do something similar for your organization!