Unreliable/Flakey Capybara/AngularJS Integration Tests With Timing Issues

前端 未结 2 1844
挽巷
挽巷 2021-02-10 14:40

How do I make these tests reliably pass?

Currently these tests are flakey.
Sometimes they pass. Sometimes they fail.
Below is the setup, code and output demons

相关标签:
2条回答
  • 2021-02-10 15:23

    The Issue

    Problems arose when the test was using capybara's fill_in DSL. In some test runs the fields would be populated correctly and the form would be submitted. In the other scenario the form would be filled in correctly but the submit button would be hit too quickly. The result here is that a record was created but input fields of name and description were not persisted.

    1. When filling in forms, ensure AngularJS bindings are complete and Capybara waiting methods are used

    AngularJS' ng-if statements needed to be used not to show form fields until they are ready.
    This needed to be done in conjunction with the use of Capybara waiting methods to ensure fill_in field are only submitted once form load has completed.

    index.html.erb or equivalent:

    <div  ng-if="tableParams.data">
      <table id="costings_table ng-table="tableParams" class="table">
        <td id="field1">{{table.field1}}</td>
        <td id="field2">{{table.field2}}</td>
       </table>
    </div>
    

    2. Updated the command line version of PhantomJS to the latest (2.1.1)

    This seemed to enable tests to run without as many Capybara waiting methods in order to achieve reliable tests.

    Updated Test Code
    show_costing_spec.rb

    require "rails_helper"
    
    RSpec.describe "Show a new costing in the listing,", :type => :feature do
    
      before :each do
        admin_sign_in
        create_costing("test1")
      end
    
      it "shows the costing after creation" do
        within "#costings_table" do
          expect(page.find("#code2")).to have_content("2")
          expect(page.find("#name2")).to have_content("test1")
        end
      end
    
      it "shows the details of the new costing after creation" do
        within "#costings_table" do
          click_on "show2"
        end
    
        expect(page.find("#page_title")).to have_content("Costing Details")
        expect(page.find("#code")).to have_content("2")
        expect(page.find("#name")).to have_content("test1") 
        expect(page.find("#description")).to have_content("test description")
      end
    end
    

    rails_helper.rb

    # This file is copied to spec/ when you run 'rails generate rspec:install'
    ENV["RAILS_ENV"] ||= 'test'
    require File.expand_path("../../config/environment", __FILE__)
    
    # Add library functions here so we can test them.
    require File.expand_path(File.dirname(__FILE__) + "/../lib/general")
    
    require 'rspec/rails'
    require 'devise'
    
    RSpec.configure do |config|
      config.before(:suite) do
        # Requires supporting ruby files with custom matchers and macros, etc,
        # in spec/support/ and its subdirectories.
        require File.expand_path(File.dirname(__FILE__) + "/support/blueprints")
        Dir[Rails.root.join("spec/support/**/*.rb")].each {|f| require f}
    
        # Setup Devise before it is used in rails_helper
        config.include Devise::TestHelpers, :type => :controller
        Devise.stretches = 1 # Improves speed.
    end
    
    config.include Capybara::DSL, :type => :feature
      config.mock_with :rspec
    
    # Remove this line if you're not using ActiveRecord or ActiveRecord fixtures
    config.fixture_path = "#{::Rails.root}/spec/fixtures"
    
    # Allow a 'focus' tag so that we can run just a few tests which we are currently working on
    config.filter_run focus: true
    config.run_all_when_everything_filtered = true
    config.filter_run_excluding :slow unless ENV["SLOW_SPECS"]
    
    # Defer Garbage Collection
    config.before(:all) { DeferredGarbageCollection.start }
    config.after(:all)  { DeferredGarbageCollection.reconsider }
    
    # Integration Testing
    require 'capybara/rspec'
    require 'capybara/poltergeist'
    
    Capybara.register_driver :poltergeist_debug do |app|
      Capybara::Poltergeist::Driver.new(app, {:inspector => true, js_errors: false })  
    end
    
    Capybara.javascript_driver = :poltergeist_debug
    Capybara.default_driver = :poltergeist_debug
    
    # Debugging tools
    def debugit
      puts current_url
      require 'pry'
      binding.pry
    end
    
    # If you're not using ActiveRecord, or you'd prefer not to run each of your
    # examples within a transaction, remove the following line or assign false
    # instead of true.
    config.use_transactional_fixtures = false
    
    #Show Deprications As Errors with full backtracing
    config.raise_errors_for_deprecations!
    
    #rest of the file....
    # Final part of Configure Database Cleaner
    
    Capybara.default_max_wait_time = 5
    config.use_transactional_fixtures = false
    
    config.before(:suite) do
      if config.use_transactional_fixtures?
        raise(<<-MSG)
          Delete line `config.use_transactional_fixtures = true` from
          rails_helper.rb (or set it to false) to prevent uncommitted
          transactions being used in JavaScript-dependent specs. During 
          testing, the app-under-test that the browser driver connects to 
          uses a different database connection to the database connection 
          used by the spec. The app's database connection would not be 
          able to access uncommitted transaction data setup over the 
          spec's database connection.
         MSG
      end
      DatabaseCleaner.clean_with(:truncation)
    end  
    
    config.before(:each) do
      DatabaseCleaner.strategy = :transaction
    end
    
    config.before(:each, type: :feature) do
      # :rack_test driver's Rack app under test shares database connection
      # with the specs, so continue to use transaction strategy for speed.
      driver_shares_db_connection_with_specs = Capybara.current_driver == :rack_test
    
        if !driver_shares_db_connection_with_specs
          # Driver is probably for an external browser with an app
          # under test that does *not* share a database connection with the
          # specs, so use truncation strategy.
          DatabaseCleaner.strategy = :truncation
        end
      end
    
      config.before(:each) do
        DatabaseCleaner.start
      end
    
      config.append_after(:each) do
        DatabaseCleaner.clean
      end
    end
    
    
    def admin_sign_in
      visit "/login"
    
      #Create staff member in database
      Staff.make!(:admin)
    
      #Log In
      fill_in "staff_username", with: "adminstaff"
      fill_in "staff_password", with: "password"
      click_button "login"
    
      expect(page).to have_text('Logout')
    end
    
    def create_costing(item)
      @item = item
      visit "/api#/costings"
    
      expect(page).to have_selector("#new_btn")
      click_on "new_btn"
    
      expect(page).to have_text("New Costing")
      within "#form_costing" do
        fill_in "name", with: "#{@item}"
        fill_in "description", with: "test description"
        fill_in "from_date1", with: "15/02/2015" 
        fill_in "cost_hourly_cents1", with: "12.00"
    
        expect(page).to have_selector("#create_btn")
        click_on "create_btn"
      end
      expect(page.find("#page_title")).to have_content("Costings")
    end
    
    0 讨论(0)
  • 2021-02-10 15:29

    The immediate thing that jumps out is that your admin_sign_in doesn't actually wait for the sign_in to complete. This means that your call to create_costing can occur without the session cookie having been set in your browser. The last line in your admin_sign_in method should be something like

    expect(page).to have_text('You are signed in') # whatever message is shown upon sign in
    

    or

    expect(page).to have_current_path('/') # whatever path an admin is redirected to upon signing in
    

    That will make sure the login has actually completed and therefore the session cookies have been set in your browser.

    Also your database cleaner config should use an append_after block rather than after - see https://github.com/DatabaseCleaner/database_cleaner#rspec-with-capybara-example

    0 讨论(0)
提交回复
热议问题