Question

我的基本功能规格在本地传递正常但在CircleCI和Codeship上失败。失败的测试：

require 'spec_helper'

describe 'Authentication' do

  describe "sign in" do
    it "is the landing page for non-authenticated users" do
      user = create(:user)
      visit root_path

      expect( page ).to have_content 'LOG IN' # On sign-in page
      fill_in 'Email', with: user.email
      fill_in "Password", with: user.password
      click_button 'Sign in'

      expect( current_path ).to eq user_path(user)
    end
  end

  describe 'registration' do
    it "allows new users to register" do
      visit root_path

      click_link 'Sign up'
      fill_in 'Email', with: 'myfake@email.com'
      fill_in 'Password', with: 'password'
      fill_in 'Password confirmation', with: 'password'
      fill_in 'First name', with: 'John'
      fill_in 'Last name', with: 'Doe'
      click_button "Sign up"

      expect( current_path ).to include '/users/'
      expect( page ).to have_content "Welcome! You have signed up successfully."
    end
  end
end

测试在第一行上都失败了，他们分别设置了页面的期望值（分别为expect( page ).to have_content "LOG IN"和click_link "Sign up"），错误表明页面HTML完全空白：

expected to find text "LOG IN" in ""

我在CircleCI上保存了截图，他们确实显示了一个完全空白的页面。

这里有趣的地方。我尝试通过running/watching the specs on Circle using a VNC调试问题。当我 a）为测试设置driver: :selenium时， b）在测试页面预期之前，在测试中添加sleep 1或两个，并且< strong> c）在使用VNC通过SSH连接到服务器后手动运行测试，我可以看到测试在Selenium中运行（他们在VNC中打开浏览器）并且它们完全通过。

然而，在VNC之外，两个CI服务器中的测试都会一致地失败。有或没有吨sleep和driver: :selenium。有什么想法可能导致常规CircleCI / Codeship服务器与他们的VCN /我的本地测试环境之间出现这种差异？我和CircleCI的人们取得了联系，但是他们暂时难倒了。

如果相关，我正在运行Ruby 2.2.0，Rails 4.2，Capybara 2.4.4，Capybara-Webkit 1.4.1和Selenium-Webdriver 2.44.0

一些可能相关的文件：

spec_helper.rb

ENV["RAILS_ENV"] = "test"

require File.expand_path("../../config/environment", __FILE__)

require "rspec/rails"
require "shoulda/matchers"
require "webmock/rspec"
require 'vcr'

Dir[Rails.root.join("spec/support/**/*.rb")].each { |file| require file }

module Features
  include Warden::Test::Helpers
  Warden.test_mode!

  def sign_in(user)
    login_as(user, scope: :user)
  end
end

module Controllers
  # Pre-parse controller responses for easy access
  def response_body
    body = JSON.parse(response.body)
    body.is_a?(Hash) ? body.to_sh : body.map(&:to_sh)
  end
end

module Mock
  def disable_webmock(&block)
    WebMock.disable!
    yield
    WebMock.enable!
  end
end

RSpec.configure do |config|
  config.expect_with :rspec do |c|
    c.syntax = :expect
  end

  # Save a screenshot to CircleCI when a feature test fails
  config.after(:each, :type => :feature) do |example|
    if example.exception
      artifact = save_page
      puts "\"#{example.description}\" failed. Page saved to #{artifact}"
    end
  end

  config.include Features, type: :feature
  config.include Controllers, type: :controller
  config.include Mock
  config.include Formulaic::Dsl, type: :feature
  config.infer_spec_type_from_file_location!
  config.infer_base_class_for_anonymous_controllers = false
  config.order = "random"
  config.use_transactional_fixtures = false
end

RSpec::Matchers.define :hash_eq do |expected|
  match do |actual|
    actual.recursive_symbolize_keys == expected.recursive_symbolize_keys
  end
end

VCR.configure do |c|
  c.cassette_library_dir = 'spec/fixtures/vcr_cassettes'
  c.hook_into :webmock
  c.allow_http_connections_when_no_cassette = true
  c.configure_rspec_metadata!
  c.ignore_hosts '127.0.0.1', 'localhost:3000'
end

ActiveRecord::Migration.maintain_test_schema!
Capybara.javascript_driver = :webkit

if ENV['CIRCLE_ARTIFACTS']
  Capybara.save_and_open_page_path = ENV['CIRCLE_ARTIFACTS']
end

WebMock.disable_net_connect!(allow_localhost: true)

database_cleaner.rb

RSpec.configure do |config|
  config.use_transactional_fixtures = false

  config.before(:suite) do
    DatabaseCleaner.strategy = :truncation
  end

  config.before(:each) do
    DatabaseCleaner.start
  end

  config.after(:each) do
    DatabaseCleaner.clean
  end

end

Answer 1

我可以想到几件事要尝试：

如果你想让你的规格无头，请尝试使用poltergeist而不是capybara-webkit。 poltergeist有几个我在这里描述的优点：https://stackoverflow.com/a/24108439/634576
Selenium可能只是让人分心，但是，如果你想让它发挥作用，请确保您的CI环境具有正确版本的Firefox。每个版本的selenium-webdriver似乎都需要一小部分版本的Firefox。 On CircleCI, you can configure circle.yml to install a specific version of Firefox.现在我们使用selenium-webdriver 2.46.2和Firefox 39.0.3，在circle.yml中安装了以下内容：
```
dependencies:
  pre:
    - sudo apt-get update; sudo apt-get install firefox=39.0.3+build2-0ubuntu0.12.04.1
```
我不知道Codeship。

Answer 2

我遇到过关于CircleCI的Selenium测试的不稳定性，直到我遇到这篇文章https://discuss.circleci.com/t/selenium-tests-timing-out/7765/2

添加后：

machine:
  environment:
    DBUS_SESSION_BUS_ADDRESS: /dev/null

到我的circle.yml，我的测试变得稳定。

功能规格仅在CircleCI或Codeship持续集成服务上失败

2 个答案: