Cucumber and delayed_job: Together in Less-Than-Perfect Harmony

I love Cucumber. There, I said it. I know I’m not in the minority of Rubyists when I say that either. Cucumber helps me stay focused on the task at hand from the perspective of the people who matter the most, that is to say, the end-user. Cucumber allows me to draft my tests in a more native vernacular, it also lets me keep my tests organized and makes my steps cleaner and more reusable in the future. Thinking in terms of the user also comes with the benefit of ensuring we have tests in place that actually walk through the application from the perspective of the system’s behaviour in response to user actions. However, sometimes we have to set up tests that don’t follow the path of a user. Sometimes we follow the path of the administrator, who is, arguably, a user as well, but we follow his role in a different way than we may treat the role of a non-technical user. In my Cucumber tests, I tend to organize my scenarios by the roles that each feature chiefly applies to. For example, I may have files named:

• visitor_signs_up.feature
• user_signs_in.feature
• administrator_nerfs_trolls.feature

And so on in that fashion. However, sometimes we run into an instance where our user is the system itself. These tests are typically related to automated tasks on the server side such as automated maintenance, cron jobs, and background processes run with delayed_job or Resque. In these cases, the game changes a bit. We are no longer following a terse path through the application from the front-end. Now we are attempting to simulate a very different type of behaviour.

On a recent project, I found myself in need of a recurring task to occur on the server once a month on the first of every month. My first instinct was to use cron. The rationale behind this decision was for pure simplicity. It was only one task, that only needed to be run once a month. Adding new complexities to this seemed unnecessary. However, as I began looking into how to test this scenario, it soon became very apparent that this wasn’t my ideal solution. Cron exists outside of the Rails stack. There is no way for my test suite to directly interact with my system’s cron, and that meant I would have an integration test that makes assumptions about my server configuration. This would lead to a less portable application that had no assurance of whether or not cron was actually present on the system or configured the right way for that matter. I was directed to tools such as javan’s gem “whenever” for keeping the cron configuration within my Rails app, but this still didn’t solve the problem with the Cucumber tests.

Thus I began looking into background job tasks such as delayed_job and Resque. Resque, built by GitHub, runs on Redis, supports multiple simultaneous queues, and a host of other awesome features. As cool as this sounded, this was definitely far too much complexity for my app. So with that, I turned to delayed_job, the de facto standard for background jobs in Rack-based applications. However, delayed_job does not support recurring tasks natively. Instead, I added another tool called dj_remixes by markbates, which is a plugin for DJ that adds a host of great features including recurring tasks. So, now I had my tools in place, all I had to do was write my test…or so I thought. Originally, I started with a test that looked something like this:

My logic behind this test is pretty straightforward. I’m using Timecop to help stub out the dates on the system to reflect the passing of each month in hopes that on the first of each month, the system would automatically update the list of books to mark the book with the most votes as “featured” each month. Obviously, this test is missing a few key items, but my real goal with this scenario is to ensure that the automated task is performing its job when it’s supposed to. However, I soon ran into a major problem. delayed_job runs in a separate thread than your Rails app. So even though I’m stubbing out the time in my Rails application, these changes are not being seen by delayed_job. To find the solution, I had to take a step back to understand what I was really trying to accomplish with this test. I needed to ensure the following:

• I can enqueue the book worker into the DJ table
• When the job in the DJ table is run, it updates the featured book
• After the job has been run, it creates a new job in the table to be run on the NEXT first of the month.

The last part is the most important of the three as it represents that the system is properly enqueueing jobs to run at the dates I need them to. After a lot of tweaking and refining, I finally arrived at the following scenario:

Admittedly, this isn’t ideal for me either. I don’t like having to specify that jobs are run, those should be automated. I also don’t like the idea of running Rake tasks related to setup in my tests. There’s no real reason for that, it just doesn’t feel right. Feel free to share your criticisms, I’m all ears. Even better, I’m more open to suggestions on how to better this test. However, for the meantime, it represents the behaviour I want in my system, even more importantly, it passes.

Some other options I could have considered include stubbing out delayed_job with a fake object that represented DJ’s basic behaviour, but as Yehuda Katz said, “Don’t mock anything you own.” This is in reference to parts of your system of which you have control. External API’s are not part of your ownership, so stubbing those out is always wise, however, writing tests that prove that controlled software dependencies function in integration with your system is a much more ideal solution in my mind. Another thing that I find bothersome is that with this test I still can’t escape the assumptions made about my server configuration. If there was a better way, I’d jump on it, but as far as I can tell, that’s not an option.

Have you ever had to write a test like this? How have you handled it? How can I make this test better? Share your thoughts and happy coding.