One time the unit tests failed me

Since my initiation into the world of unit testing about a year or so ago, I’ve been a huge fan of unit testing. I write unit tests for almost everything now, rather than doing repeated manual testing, which is sometimes more tedious (and certainly gets boring after a while).

But just now, the unit tests failed me. I’ll explain the case in a moment, but let me just say that, as great as unit tests are, they aren’t the only kind of testing you need. Doing manual beta testing is always needed for good measure.

So here’s what happened. I had written a custom module for my WordPoints plugin, with unit tests, of course. And the unit tests were passing fine.

That was last week. But before I tried it out on a live site, I wanted to test it manually on one of my local setups. And guess what. It didn’t work.

I eventually figured out that the issue was a bug in WordPoints, that was caused by WordPress having a 64 character limit for option names. I knew about this limit, but it hadn’t bitten me yet. I’d actually forgotten whether I’d handled the limit properly or not. But that is by the by.

What struck me after I figured out what was going on was this: “What about the unit tests I wrote, they passed fine!”

Yep, they did. And here’s why. For the unit tests, WordPress’s testcase sets up a database transaction before each test. Of course it doesn’t actually commit the data to the database, instead it rolls back the transaction after the test is done. This way, the database stays clean between tests, so they don’t leak into each other.

And this is why my tests didn’t fail. The data wasn’t actually getting committed to the database, so saving the option value wasn’t failing like it normally would. I assume that the database checks that the table and column names are all good, but doesn’t realize that you’re trying to write 65 characters to a VARCHAR(64) field until the data gets committed (which in this case it doesn’t).

In the end, I really think the issue here is that WordPress should fail, or tell us that we’re _doing_it_wrong(), when we try to use option names that are too long. But until then, we need to be careful, when using dynamic option names, that we check they aren’t too long. Especially since unit tests won’t be enough to catch the issue.

Leave a Reply

Your email address will not be published. Required fields are marked *