Ruby UnitTesting

Taming Asynchronous Testing

Ok, the title’s a bit overblown, but I encountered a fun bit of Ruby in working with threaded code.

My class’s primary responsibility was starting/stopping a loop on another polling thread. The basic shape of things ended up something like this:

  def start do
      … interesting work here

… which left a dilemma around testing. We forged down a path of setting the loop with timeouts and known counts, well knowing the perils. While these ran consistently enough locally, the time wasn’t always “enough” when we ran on our CI servers. We didn’t want to add tons of waiting in the unit tests, but cutting it too fine led to spurious failures on fussy VM’s.

I set out to revise the tests to be more deterministic. One prong of my attack was to provide functional/acceptance tests which could happily run with significant time buffers that would be cripplingly long in a unit test.

Unit tests, though, we remained a question. I toyed with passing in a stub to wrap the thread interactions, but because of the interactions that stub actually had to be plumbed through two separate classes to get where it was going. Ugly.

Could I use Ruby’s much vaunted openness to solve this? Of course! Instead of passing some other dependency, what if the Thread code was just replaced to work synchronously?

The statically-typed, C# side of my brain lurched at the thought, but it worked great. I created a FakeThread class that simply runs the block it’s given during initialize(). Other thread calls needed basic implementation in the FakeThread, but that was simple to accomplish.

With the fake class in hand, we needed to swap it in. However the naive approach:

  def setup
    Thread = FakeThread

… causes Ruby to gripe mightily:

  warning: already initialized constant Thread

But luckily we can “fix” that by removing the Thread constant from the module, then redefining it with our own implementation:

  def setup
    Object.send(:remove_const, “Thread”) if Object.const_defined?(“Thread”)
    Object.const_set(“Thread”, FakeThread)


Gnarly? Perhaps. I’d never consider this in production code. But the simplicity of splicing in the right dependencies into a test? It’s exactly what drew me to Ruby in the first place.

My coworker, when looking over the result, and hearing my bashfulness at having overridden a core class as Thread, concluded “Today, you are a Ruby programmer.”


Unit testing isn’t about bugs

For those already living the TDD lifestyle this will be old news. But a couple weeks back I overheard one of my coworkers say, “Unit testing doesn’t find bugs because they’re only checking what you already expect. That’s not where the bugs are”

As sold as I am on unit tests, I agree. No amount of automated dev testing can replace good QA. But this comment also misses the major wins for unit testing.
Unit tests force smaller classes and methods
The largest impact to my coding from TDD is that my classes and methods have shrunk. It’s well known that short and simple code is best, but it takes discipline to keep your code that way.
Unit tests backs you up on keeping things small. Tests get hard to write when things are large and complicated. You’ll start to feel an itch (or worse) when that method does too much, or that class requires onerous initialization.
Yes, you still have to recognize the smell and do something about it, but the unit tests gives some nice pressure in the right direction.
Unit tests highlight bad naming
A good test reads like a description of what a method does. Are there required actions in the midst of your test that don’t fit with the flow? Maybe you’ve got a bad name there.
Again, bad names crop up everywhere, but good, striped down unit test are often the first spot to highlight a name that needs work.
Unit tests give you confidence
With tests in place, I’m more likely to rename, extract methods, and otherwise clean things up. The more thorough the tests (and the simpler the code in question) the more comfortable I am with making changes.
Until you’ve got unit tests in place and make a large-scale change without breaking anything significant, I don’t think you can entirely understand how valuable that confidence is.
Unit tests do find bugs… later
The tests you write won’t find the bug you just checked in and didn’t write a test for. But it will save you six months from now when you violate that subtle condition in a hurry to fix some other bug. They’ll ensure over time that some aspects of the code haven’t changed, no matter what chaos is swirling around them.
So there it is. Even if unit tests don’t point out all your bugs straight off, they are still worth the effort.

Spreading Straw

Last weekend, I did one of my “favorite” chores–cleaning up after our two retired greyhounds in the backyard. What does this have to do with programming? Well, shovel in hand, for some reason I started thinking about unit testing. An analogy bubbled and percolated, refusing to let me focus on spotting dog crap.

Our backyard isn’t large, and greyhounds like to run. They’ve devestated any semblence of grass back there, and during Oregon’s wet season (read “nine months of the year”), it’s muddy. Real muddy. To combat the mess, we get bails of straw from the local feed store and spread it all over. Ugly to look at perhaps, but it keeps life an order of magnitude cleaner.
How does this relate to testing? Well, a lot of people talk about getting good code coverage. But even if you get 100% coverage of every path, that doesn’t necessary mean you’ve tested everything.
Imagine each individual bit of straw is a test, while the mud is the codebase. I guess the dogs would be developers then, churning it up, while the rain would be product showering you with requests that threaten to ruin all your beautiful architecture. Hold on, went too far there. Back up to the straw and mud, that’s the important bit.
If I spread my entire yard with straw exactly one piece deep, how well do you think that’d work? We’d have a pit in seconds once the greys got going. Not all parts of the yard are equal. The pit at the corner where water comes down the sideyard and the dogs make a hard turn? That needs some extra covering. How about the main track the dogs have dug down below the walnut tree? The whole length gets more.
I see unit testing the same way. Yes, you should shoot to covering everything, but if you stop at “Tests hit every line of code,” you could be missing something. Where are the more complicated sections? Where should your tests be communicating more of the code’s purpose? Where have your bugs been coming in frequently?
Maybe those spots need a little extra straw.