Tag Archives: Plugin Development

One time the unit tests failed me

Since my initiation into the world of unit testing about a year or so ago, I’ve been a huge fan of unit testing. I write unit tests for almost everything now, rather than doing repeated manual testing, which is sometimes more tedious (and certainly gets boring after a while).

But just now, the unit tests failed me. I’ll explain the case in a moment, but let me just say that, as great as unit tests are, they aren’t the only kind of testing you need. Doing manual beta testing is always needed for good measure.

So here’s what happened. I had written a custom module for my WordPoints plugin, with unit tests, of course. And the unit tests were passing fine.

That was last week. But before I tried it out on a live site, I wanted to test it manually on one of my local setups. And guess what. It didn’t work.

I eventually figured out that the issue was a bug in WordPoints, that was caused by WordPress having a 64 character limit for option names. I knew about this limit, but it hadn’t bitten me yet. I’d actually forgotten whether I’d handled the limit properly or not. But that is by the by.

What struck me after I figured out what was going on was this: “What about the unit tests I wrote, they passed fine!”

Yep, they did. And here’s why. For the unit tests, WordPress’s testcase sets up a database transaction before each test. Of course it doesn’t actually commit the data to the database, instead it rolls back the transaction after the test is done. This way, the database stays clean between tests, so they don’t leak into each other.

And this is why my tests didn’t fail. The data wasn’t actually getting committed to the database, so saving the option value wasn’t failing like it normally would. I assume that the database checks that the table and column names are all good, but doesn’t realize that you’re trying to write 65 characters to a VARCHAR(64) field until the data gets committed (which in this case it doesn’t).

In the end, I really think the issue here is that WordPress should fail, or tell us that we’re _doing_it_wrong(), when we try to use option names that are too long. But until then, we need to be careful, when using dynamic option names, that we check they aren’t too long. Especially since unit tests won’t be enough to catch the issue.

Plugin back-compat with old WP versions

One of the great things about the WordPress project, is that it is committed to retaining backward-compatibility. That makes life easier for plugin developers, because we don’t have to constantly worry that the next update to WordPress will break our plugins. It won’t. And in rare cases where it could, we are given full warning ahead of time.

But sometimes things do get deprecated, and a new, better way of doing things is introduced. When this happens, we often need to update to the new way of doing things. Sometimes this means our plugins won’t work with older versions of WordPress out of the box, unless we write special code for backward compatibility.

This post is a plea to WordPress developers not to do this.

Actually, I’m all for maintaining backward compatibility with older versions of WordPress, as long as it doesn’t keep you from moving forward. I’ve actually done it myself. But before writing back-compat code, think a little bit about the out-dated version of WordPress that you are supporting. Should people actually be using that version of WordPress? Sure, some folks don’t need to have every new feature that WordPress comes out with, they’re doing just fine with the version they have. But it isn’t about the features. It isn’t even about the bugs. It’s not even about the fact that the admin panel feels like something brought through the flood on the ark. No. It’s about the vulnerabilities.

If you haven’t actually faced this yet, let me disillusion you. All web software has security vulnerabilities. Every. Single. One.

The old versions of WordPress have vulnerabilities, and the new versions of WordPress have vulnerabilities. In fact, there are vulnerabilities in WordPress trunk, that as you read this, are just sitting there waiting to be discovered. The difference is that we know what the vulnerabilities are in the old versions of WordPress. And we know something else. They aren’t fixed. They are still there on every site that is running that old WordPress version. That old, insecure version of WordPress.

Now let me ask you, is that something that you, as a WordPress developer, want to promote? Using vulnerable software?

But what can you do about it? You can decide not to waste your precious time writing code whose only purpose is to support these outdated WordPress versions.

I’m not asking much. I’m not asking you to write code to purposely break your plugin’s compatibility with old WP versions. If it happens to run fine on vulnerable versions of WordPress, that’s fine with me.

I’m actually not asking you to do anything, for starters. Just stop doing something that maybe you’re already doing: writing back-compat code for insecure WordPress versions. And if you do want to do something, here’s an idea: remove the back-compat code you’ve already written.

Maintaining backward compatibility is always a tough decision. It takes extra work, work that could be used to fix bugs or design cool new features. So why do it when you have such a good excuse reason not to? Writing back-compat code for insecure versions of WP is encouraging users to continue using those versions instead of updating to newer versions, where all known vulnerabilities have been fixed.

There was a time in the not-to-distant past, when only the latest version of WordPress received security updates. That time is no more. With the advent of automatic updates, making security patches available to users of older versions became much easier. Now all versions of WordPress back to 3.7, when automatic updates were introduced, are maintained. So you can provide backward compatibility to 3.7 in your code in good conscience. But please, don’t go any further than that.

Telling your users why you don’t support older versions is also a good idea. They will thank you. Or they’ll call you names, and wonder why they keep getting hacked. Either way, you know that you are doing something to make the web more secure.

WP Ajax Plugin Unit Testing

The other day I was refactoring some code for an Ajax callback in one of my plugins. I was dreading having to manually test that the changes I was making didn’t break anything, when I had an idea. “Why don’t I have unit tests for this?”, I though to myself. Before I started trying to mock Ajax requests myself, I thought I’d ask Google what was already out there. As it turns out, the WordPress unit test framework already provides a testcase specifically for testing Ajax callbacks. Who knew? Obviously, the developers did, but I hadn’t thought much about it before, except that each time you run the WP unit tests, you get a message that it the Ajax unit tests aren’t being run.

While I found this testcase in the WordPress PHPUnit test source code, what I didn’t find was any tutorials on how it works. So, I just used WP’s AJAX unit tests as an example. Its pretty simple actually.

The WP_Ajax_UnitTestCase

If you’re familiar with plugin unit testing, you know that your testcases need to extend the WP_UnitTestCase class to leverage the full power of the WordPress testing framework. Well, for Ajax tests, its the same thing, only you need to extend a child of WP_UnitTestCase: WP_Ajax_UnitTestCase. This testcase provides a few methods that make the Ajax unit tests simpler.

The most important method is of course the one that actually mocks the Ajax request: _handleAjax(). It takes just one parameter; the name of the Ajax action:

$this->_handleAjax( 'my_ajax_action' );

Setting up for the test

Of course, before you run the Ajax handler, you will need to set up for the test. The first thing you need to do is set up the $_POST data. You need to mock up the $_POSTed data, and then later we’ll check that we get the expected result.

$_POST['_nonce'] = wp_create_nonce( 'my_nonce' );
$_POST['other_data'] = 'something';

Setting the expected exception

The test is performed by running the 'wp_ajax_*' action, and thus invoking the callback. (No request is actually made.) When the Ajax callback function runs, it should finish by calling die() or exit(). Well, actually, it will need to be using wp_die(), because if you call die() you will kill the tests. Lesson: in WP Ajax callbacks, you should use wp_die(). Fortunately, I already was in the callbacks that I was testing, so I was good to go.

Now, when wp_die() gets called, the testcase catches it and prevents it from actually killing the execution of the script. Instead, it throws an exception. The testcase uses two different exceptions, and which one gets thrown depends on whether there was any output from callback (this is caught with a buffer). If there was output, a WPAjaxDieStopException is thrown. Otherwise, a WPAjaxDieContinueException is thrown. These need to be caught. One way that this is done is using the setExpectedException() method provided by PHPUnit.

If, for example, you are expecting the callback to exit without any output, you would do this:

$this->setExpectedException( 'WPAjaxDieStopException' );
$this->_handleAjax( 'my_ajax_action' );

If you need to run some assertions afterward, you’ll need to catch the exception manually instead. For example, your Ajax callback may be supposed to save some data to the database, and you’ll want to check that the data was actually saved.

try {
	$this->_handleAjax( 'my_ajax_action' );
} catch ( WPAjaxDieStopException $e ) {
	// We expected this, do nothing.
}

$this->assertEquals( 'yes', get_option( 'some_option' ) );

Checking for the Correct Output

If your Ajax function outputs some data, you will probably want to check that the expected data gets output. If you are sending a simple response code with wp_die(), you’ll probably want to check that the expected value was output. The message passed to wp_die() is the message of the exception that gets thrown. So one way to check for this is when you set the expected exception:

$this->setExpectedException( 'WPAjaxDieStopException', '1' );

In this case we are expecting '1' to get output to signify a successful result.

If you need to manually catch your exception so you can run some other assertions like we did above, you’ll need to do it differently:

try {
	$this->_handleAjax( 'my_ajax_action' );
} catch ( WPAjaxDieStopException $e ) {
	// We expected this, do nothing.
}

// Check that the exception was thrown.
$this->assertTrue( isset( $e ) );

// The output should be a 1 for success.
$this->assertEquals( '1', $e->getMessage() );

$this->assertEquals( 'yes', get_option( 'some_option' ) );

But if you are outputting your data directly instead of using wp_die(), you need to do it differently. One example might be if you are using wp_send_json_success() to send a JSON response. In this case, the output is caught by _handleAjax() and stored in $this->_last_response. So you could check for the correct JSON response like this:

try {
	$this->_handleAjax( 'my_ajax_action' );
} catch ( WPAjaxDieStopException $e ) {
	// We expected this, do nothing.
}

$response = json_decode( $this->_last_response );
$this->assertInternalType( 'object', $response );
$this->assertObjectHasAttribute( 'success', $response );
$this->assertTrue( $response->success );

Setting the User Role

Many times an Ajax callback isn’t intended for all users. You may be saving something that only users with administrator permissions should be allowed to save. The Ajax test case class provides a handy helper function to let you set the current user’s role. For example, if your Ajax callback should only work for administrators, you may want to test that just any subscriber can’t come along and use it. To do this, you would set the user’s role before calling _handleAjax(), like this:

$this->_setRole( 'subscriber' );

And if you want to make sure that the administrators will have the permissions necessary to use it, you would set the role to 'administrator' instead:

$this->_setRole( 'administrator' );

You can also run as a logged-out user:

$this->logout();

Running Your Tests

Based on the examples above, our final testcase might look something like this:

/**
 * Test case for the Ajax callback to update 'some_option'.
 *
 * @group ajax
 */
class My_Some_Option_Ajax_Test extends WP_Ajax_UnitTestCase {

	/**
	 * Test that the callback saves the value for administrators.
	 */
	public function test_some_option_is_saved() {

		$this->_setRole( 'administrator' );

		$_POST['_wpnonce'] = wp_create_nonce( 'my_nonce' );
		$_POST['option_value'] = 'yes';

		try {
			$this->_handleAjax( 'my_ajax_action' );
		} catch ( WPAjaxDieStopException $e ) {
			// We expected this, do nothing.
		}

		// Check that the exception was thrown.
		$this->assertTrue( isset( $e ) );

		// The output should be a 1 for success.
		$this->assertEquals( '1', $e->getMessage() );

		$this->assertEquals( 'yes', get_option( 'some_option' ) );
	}

	/**
	 * Test that it doesn't work for subscribers.
	 */
	public function test_requires_admin_permissions() {

		$this->_setRole( 'subscriber' );

		$_POST['_wpnonce'] = wp_create_nonce( 'my_nonce' );
		$_POST['option_value'] = 'yes';

		try {
			$this->_handleAjax( 'my_ajax_action' );
		} catch ( WPAjaxDieStopException $e ) {
			// We expected this, do nothing.
		}

		// Check that the exception was thrown.
		$this->assertTrue( isset( $e ) );

		// The output should be a -1 for failure.
		$this->assertEquals( '-1', $e->getMessage() );

		// The option should not have been saved.
		$this->assertFalse( get_option( 'some_option' ) );
	}
}

And our Ajax callback might look like this:

/**
 * Ajax callback to save the 'some_option' option.
 */
function my_some_option_ajax_callback() {

	check_ajax_referer( 'my_nonce' );

	if ( ! isset( $_POST['option_value'] ) || ! current_user_can( 'manage_options' ) ) {
		wp_die( '-1' );
	}

	update_option( 'some_option', $_POST['option_value'] );

	wp_die( '1' );
}
add_action( 'wp_ajax_my_ajax_action', 'my_some_option_ajax_callback' );

Notice that I added @group ajax to the testcase docblock? That isn’t necessary, but you will probably want to do it, because it lets you easily run just your Ajax tests. Also, you will probably want to exclude the Ajax group from running by default, the same way WordPress does. The reason is because the tests can be very slow. The Ajax tests are so slow because all of the default Ajax callbacks in WordPress core get hooked up before each test is run. There are quite a few, and this takes some time. I don’t think this feature is actually necessary, and maybe it will be removed in future, especially if the test suite starts preserving hook state between tests. It also calls 'admin_init' every time, which is also slowing things down.

So that’s it. There really isn’t much to the Ajax unit tests, but it saved me from breaking the code I was refactoring. And hey, after all, that’s what unit tests are for.

Unit Testing WordPress Plugin Uninstallation

If you’re a WordPress plugin developer, there are two things that you should probably be doing. First, you should be writing your plugin with an uninstall routine to clean everything up when a user decides to remove it. Second, you should be unit testing your plugin, to make sure that everything is working. Actually, we could go a step further: You should be unit testing your plugin’s uninstallation script. It kind of follows naturally doesn’t it?

I thought so, so I created a set of tools to make that easier: the WP Plugin Uninstall Tester

From the project README:

The purpose of this testcase is to allow you to make plugin uninstall testing as realistic as possible. WordPress uninstalls plugins when they aren’t active, and these tools allow you simulate that. The installation is performed remotely, so the plugin is not loaded when the tests are being run.

I created these tools after finding that there was a fatal error in one of my plugin’s uninstall scripts. Not that I didn’t have unit tests for uninstallation. I did. But the uninstall tests were being run with the plugin already loaded. So I never realized that I was calling one of the plugin’s functions that wouldn’t normally be available. That’s when I decided to create these testing tools, so my uninstall tests would fail if I wasn’t including all required dependencies in my plugin’s uninstall script.

In addition to providing a realistic uninstall testing environment, it also provides some assertions to help you make sure that your plugin entirely cleaned up the database.

That basically sums up the project. I encourage you to add it to your plugin’s test suite. And it’s hosted on GitHub, so you’re welcome to help make it even better.

Don’t do_shortcode()

If you do much WordPress development, sooner or later you’ll find that you want to call a shortcode programmatically. It’s easy to do this:

do_shortcode( '[shortcode]' );

But that is a very bad idea. The do_shortcode() function is expensive, and is actually a roundabout way of accomplishing this. We know exactly what shortcode we want to call, and what we want the attributes and content to be. But do_shortcode() doesn’t know that. It will use an intensive REGEX to evaluate that simple little string, searching for the shortcode. Instead, since we already know what the shortcode is, we should just call the function that handles that shortcode. Easy enough right?

For a function like this, it is simple:

function my_shortcode() {
	return 'my shortcode';
}
add_shortcode( 'shortcode', 'my_shortcode' );

We can just call the function directly: my_shortcode().

This is simple enough, but it does have a drawback. If this function is part of a project outside your control, there is the possibility that the function might get renamed. Also, there are cases where this approach is not even possible, and I encountered one of these this week.

The shortcode function was a class instance method, and there was no way to access the instance. Kind of like this:

class my_shortcode_class {

	public function __construct() {
		add_shortcode( 'shortcode', array( $this, 'my_shortcode' ) );
	}

	public function my_shortcode() {
		return 'my shortcode';
	}
}

new my_shortcode_class();

Now how are you going to call that programmatically, without do_shortcode()? I couldn’t just create a new instance of the class, because there was other stuff in the constructor that should only run once.

Well, here’s how I did it. All of the shortcode functions are stored in the global $shortcode_tags array. So, I just pulled the function from that based on the shortcode tag, and called it using call_user_func(). Here is a simple wrapper for this, which supports passing attributes and content:

/**
 * Call a shortcode function by tag name.
 *
 * @author J.D. Grimes
 * @link http://codesymphony.co/dont-do_shortcode/
 *
 * @param string $tag     The shortcode whose function to call.
 * @param array  $atts    The attributes to pass to the shortcode function. Optional.
 * @param array  $content The shortcode's content. Default is null (none).
 *
 * @return string|bool False on failure, the result of the shortcode on success.
 */
function do_shortcode_func( $tag, array $atts = array(), $content = null ) {

	global $shortcode_tags;

	if ( ! isset( $shortcode_tags[ $tag ] ) )
		return false;

	return call_user_func( $shortcode_tags[ $tag ], $atts, $content, $tag );
}

Examples:

$result = do_shortcode_func( 'shorcode' );

$result = do_shortcode_func( 'shortcode', array( 'attr' => 'value' ), 'shortcode content' );

So next time you want to do_shortcode(), you can use this wrapper function, or implement something similar within your code that needs to call a shortcode like this.