Execute Rspec from Ruby

Question

I am trying to execute rspec from ruby, and get the status or number of failures from a method or something like that. Actually I am running something like this:

<         


        

Answer 1

This one prints to console and at the same time captures the message. The formatter.stop is just a stub function, I don't know what it is for normally, I had to include it to use DocumentationFormatter. Also the formatter output contains console coloring codes.

formatter = RSpec::Core::Formatters::DocumentationFormatter.new(StringIO.new)
def formatter.stop(arg1)
end
RSpec.configuration.reporter.register_listener(formatter, :message, :dump_summary, :dump_profile, :stop, :seed, :close, :start, :example_group_started)

RSpec::Core::Runner.run(['test.rb','-fdocumentation'])

puts formatter.output.string

Answer 2

I think the best way would be using RSpec's configuration and Formatter. This would not involve parsing the IO stream, also gives much richer result customisation programmatically.

RSpec 2:

require 'rspec'

config = RSpec.configuration

# optionally set the console output to colourful
# equivalent to set --color in .rspec file
config.color = true

# using the output to create a formatter
# documentation formatter is one of the default rspec formatter options
json_formatter = RSpec::Core::Formatters::JsonFormatter.new(config.output)

# set up the reporter with this formatter
reporter =  RSpec::Core::Reporter.new(json_formatter)
config.instance_variable_set(:@reporter, reporter)

# run the test with rspec runner
# 'my_spec.rb' is the location of the spec file
RSpec::Core::Runner.run(['my_spec.rb'])

Now you can use the json_formatter object to get result and summary of a spec test.

# gets an array of examples executed in this test run
json_formatter.output_hash

An example of output_hash value can be found here:

RSpec 3

require 'rspec'
require 'rspec/core/formatters/json_formatter'

config = RSpec.configuration

formatter = RSpec::Core::Formatters::JsonFormatter.new(config.output_stream)

# create reporter with json formatter
reporter =  RSpec::Core::Reporter.new(config)
config.instance_variable_set(:@reporter, reporter)

# internal hack
# api may not be stable, make sure lock down Rspec version
loader = config.send(:formatter_loader)
notifications = loader.send(:notifications_for, RSpec::Core::Formatters::JsonFormatter)

reporter.register_listener(formatter, *notifications)

RSpec::Core::Runner.run(['spec.rb'])

# here's your json hash
p formatter.output_hash

Other Resources

• Detailed work through
• Gist example

Answer 3

I suggest you to take a look into rspec source code to find out the answer. I think you can start with example_group_runner

Edit: Ok here is the way:

RSpec::Core::Runner::run(options, err, out)

Options - array of directories, err & out - streams. For example

RSpec::Core::Runner.run(['spec', 'another_specs'], $stderr, $stdout) 

Answer 4

Your problem is that you're using the Kernel#system method to execute your command, which only returns true or false based on whether or not it can find the command and run it successfully. Instead you want to capture the output of the rspec command. Essentially you want to capture everything that rspec outputs to STDOUT. You can then iterate through the output to find and parse the line which will tell you how many examples were run and how many failures there were.

Something along the following lines:

require 'open3'
stdin, stdout, stderr = Open3.popen3('rspec spec/models/my_crazy_spec.rb')
total_examples = 0
total_failures = 0
stdout.readlines.each do |line|
  if line =~ /(\d*) examples, (\d*) failures/
    total_examples = $1
    total_failures = $2
  end
end
puts total_examples
puts total_failures

This should output the number of total examples and number of failures - adapt as needed.