Why when I use Queue Mode for RSpec then my tests fail?

knapsack_pro < 7.0 versions in Queue Mode use RSpec::Core::Runner feature that allows running specs multiple times with different runner options in the same process. Thanks to that we can run a set of tests fetched from Knapsack Pro API work queue. Using RSpec::Core::Runner allows dynamic allocation of your tests across CI nodes without reloading the whole Ruby/Rails application for each set of tests assigned to parallel CI node by Knapsack Pro API Queue.

If you have custom things in your spec files that are not common in how typical RSpec spec looks like then the RSpec won't be able to handle it between each set of tests runs. For instance, if you mutate the global state in Ruby code and RSpec don't know how to clean it up between the next set of tests fetched from Queue is run then you may end up with a mutated global state that affects tests passing or not.

If you need help to resolve failed tests in a way that would allow RSpec to run the tests properly feel free to ask us for help. Please provide the backtrace error you see and what type of tests are failing (is it capybara feature specs, models specs, requests specs, etc?).

You can learn more about recent RSpec team changes that were backported into knapsack_pro to make sure the global RSpec world state is cleaned up between RSpec::Core::Runner runs.

To solve failing tests in Queue Mode you can check:

• you use full namespacing. If you see an error like NameError: uninitialized constant MyModule::ModelName then in some cases a top-level constant would be matched if the code hadn't been loaded for the scoped constant. Try to use full namespacing ::SomeModule::MyModule::ModelName etc.

• you can try to use a binary version of knapsack_pro instead of running it via rake task. This helps if your rake tasks mess up with tests and make knapsack_pro Queue Mode fail. See example:

  
  # Knapsack Pro Queue Mode run via binary
  bundle exec knapsack_pro queue:rspec "--profile 10 --format progress"
  
  

• You can check below questions for common reasons of failing tests in Queue Mode (there are a few popular problems)