FAQ

FAQ for knapsack_pro (Ruby)
Supported test runner: RSpec, Cucumber, Minitest, test-unit, Spinach, Turnip

• Common problems & solutions for knapsack_pro gem usage
• Queue Mode problems & solutions for knapsack_pro gem
• General questions for knapsack_pro gem
• Questions around data usage and security of knapsack_pro gem

# Common problems & solutions for knapsack_pro gem usage

• Why I see API error commit_hash parameter is required?
• Why I see LoadError: cannot load such file -- spec_helper?
• Why my CI build fails when I use Test::Unit even when all tests passed?
• Why Capybara feature tests randomly fail when using CI parallelisation?
• Why knapsack_pro freezes / hangs my CI (for instance Travis)?
• Why tests hitting external API fail?
• Why green test suite for Cucumber 2.99 tests always fails with invalid option: --require?
• Failed request to API due to OpenSSL::SSL::SSLError: SSL_connect returned=1 errno=0 state=error: tlsv1 alert internal error
• When Ruby/RSpec tests fail with error like ActiveRecord::SubclassNotFound: Invalid single-table inheritance type: AuthenticationProviders::AnAuthenticationProvider
• How to solve error cannot load such file -- stripe_mock (LoadError) when running tests with Cucumber?
• Why knapsack_pro hangs / freezes / is stale for tests in Chrome (disallow downloads in Sandboxed iframes)?

# Queue Mode problems & solutions for knapsack_pro gem

• Why when I use Queue Mode for RSpec then my tests fail?
• Why when I use Queue Mode for RSpec then FactoryBot/FactoryGirl tests fail?
• Why when I use Queue Mode for RSpec then my rake tasks are run twice?
• Why when I use Queue Mode for RSpec then I see error superclass mismatch for class?
• Why when I use Queue Mode for RSpec then .rspec config is ignored?
• RSpec is not running some tests
• Why when I use Queue Mode for RSpec and test fails then I see multiple times info about failed test in RSpec result?
• Why when I use Queue Mode for RSpec then I see multiple times the same pending tests?
• Does in Queue Mode the RSpec is initialized many times that causes Rails load over and over again?
• Why my tests are executed twice in Queue Mode? Why CI node runs whole test suite again?
• How to fix capybara-screenshot fail with SystemStackError: stack level too deep when using Queue Mode for RSpec?
• Parallel tests Cucumber and RSpec with Cucumber failures exit CI node early leaving fewer CI nodes to finish RSpec Queue
• Why when I reran the same build (same commit hash, etc) on Codeship then no tests would get executed in Queue Mode?
• Why knapsack_pro hangs / freezes / is stale i.e. for Codeship in Queue Mode?
• How to configure puffing-billy gem with Knapsack Pro Queue Mode?
• If one of the parallel CI nodes starts work very late after other parallel CI nodes already finished work then tests are executed twice on late CI node in Knapsack Pro Queue Mode
• How to fail the CI build if one of the test files duration exceeds a certain limit?

# General questions for knapsack_pro gem

• How to run tests for particular CI node in your development environment
• What happens when Knapsack Pro API is not available/not reachable temporarily?
• How can I change log level?
• How to write knapsack_pro logs to a file?
• How to auto split test files by test cases on parallel jobs (CI nodes)?
• How to split slow RSpec test files by test examples (by individual it)?
• How to manually define a list of slow test files to be split by test cases?
• How to split tests based on test level instead of test file level?
• How to make knapsack_pro works for forked repositories of my project?
• How to use junit formatter?
• How to use JSON formatter for RSpec?
• What is optimal order of test commands?
• What is the order of RSpec hooks in the Queue Mode?
• How to set before(:suite) and after(:suite) RSpec hooks in Queue Mode (Percy.io example)?
• How to call before(:suite) and after(:suite) RSpec hooks only once in Queue Mode?
• What hooks are supported in Queue Mode?
• How to run knapsack_pro with parallel_tests gem?
• parallel_tests with knapsack_pro on parallel CI nodes
• parallel_tests with knapsack_pro on single CI machine
• How to retry failed tests (flaky tests)?
• How to run only a specific set of test files in RSpec (using tags or test file pattern)?
• How to run tests from a specific directory (only system tests or features specs)?
• How can I run tests from multiple directories?
• Dir.glob pattern examples for KNAPSACK_PRO_TEST_FILE_PATTERN and KNAPSACK_PRO_TEST_FILE_EXCLUDE_PATTERN
• How to run only RSpec feature tests or non feature tests?
• How to exclude tests from running them?
• How to run a specific list of test files or only some tests from test file?
• How to run knapsack_pro only on a few parallel CI nodes instead of all?
• How to use CodeClimate with knapsack_pro?
• How to use simplecov in Queue Mode?
• How to stop running tests on the first failed test (fail fast tests in RSpec)?
• How to use spring gem with Knapsack Pro?
• How to use spring gem to run Cucumber tests faster in Queue Mode?
• How to require different Cucumber config files in isolation?
• How to run multiple test suite commands in Heroku CI?
• Why my tests are slower when I run more parallel CI nodes (jobs)?

# Questions around data usage and security of knapsack_pro gem

• What data is sent to your servers?
• How is that data secured?
• Who has access to the data?