Written by Benoit Daloze @eregontp
on 

A New Expectation Syntax for ruby/spec

ruby/spec is a test suite for the behavior of the Ruby programming language. The utility to run the test suite is called MSpec and is very similar to RSpec 2.

The reason to have its own runner and not simply using RSpec is that MSpec is significantly simpler than RSpec 2. For example, it does not require any standard library. Early-stage Ruby implementations typically do not support the entire standard library, but based on the simplicity of MSpec they are still able to run language and core library specs with minimal efforts.

MSpec also provides a few features that RSpec does or did not have such as automatically tagging/untagging failing specs, various guards useful for ruby/spec, etc.

Existing Syntax in RSpec 2 and RSpec 3

MSpec uses a syntax very similar to RSpec 2:

describe "String#start_with?" do
  it "returns true only if beginning match" do
    "hello".start_with?('hel').should == true
  end
end

This contrasts to the RSpec 3 style:

RSpec.describe "String#start_with?" do
  it "returns true only if beginning match" do
    expect("hello".start_with?('hel')).to eq true
  end
end

I don’t particularly like the RSpec 3 style:

These reasons are why I am mostly happy with the existing RSpec 2 style in ruby/spec.

Using expect has of course advantages too, notably when using delegation, as explained in the original blog post about expect.

The Various Equality Methods

One aspect discussed in that blog post is that should == generates Ruby warnings. This used to indeed be annoying. However in Ruby 2.4+ it’s easy to automatically filter out these warnings by overriding Warning.warn, which MSpec does.

When I compare actual.should eq(expected) to actual.should == expected, I strongly prefer the latter, because it is much closer to normal Ruby code which would just test actual == expected. There are also various equality methods in Ruby: ==, eql? and equal?. I’d like to not have to remember any kind of mapping between regular code and specs; I’d like to use the exact same method names in specs. Continuing on that idea, I’m not such a huge fan of having so many matchers in MSpec to do very basic tests I could do more simply in Ruby code. Here are a few matchers from MSpec: ==, eql, equal, =~, <=, include, be_true, be_false, be_empty, be_nan, be_an_instance_of, be_kind_of, be_ancestor_of, respond_to, have_instance_method, …

So many already, and there are many more! And of course, there will always be some missing matchers, for instance there is no matcher for String#start_with?. Contrary to RSpec, there are no magic be_.../have_... matchers in MSpec.

The New Simple and Consistent Syntax

Could we do without having to remember all these mappings and just use normal predicates in Ruby code? That’s what I wanted to try with this new syntax in MSpec:

describe "String#start_with?" do
  it "returns true only if beginning match" do
    "hello".should.start_with?('hel')
  end
end

The idea is: add .should just before the predicate you expect to be truthy, and done! And this works for any method, because we can implement this through #method_missing. This syntax is not new, it was actually already used for should ==, but now it can be used for any predicate:

(1 + 2).should == 3
(1 + 2).should_not == 4
(1 + 2).should != 4
1.should < 2
"abc".should =~ /b/
4.should.eql? 4.0
self.should.equal?(self)
"end".should.end_with?("d")
[].should.empty?
(1..3).should.include?(2)
File.should.exist?("README.txt")
(1..5).step(2).should.all? { |e| e.odd? }

I think some of these look really nice, some do not read as nicely, but overall they are all consistent and extremely simple to learn.

The simplicity of removing .should to go from specs to regular code is very useful for debugging purposes. Having a minimal amount of self-contained code to reproduce an issue is often key to understand and fix a bug.

A significant advantage of this approach is that all matchers have a similar output for errors, and provide a lot of information if they fail.

With the old syntax (s.start_with?('b').should == true), the error wouldn’t be very helpful and one would need to look at the test code to get an idea of what went wrong:

String#start_with? returns true only if beginning match FAILED
Expected false to equal true

With the new syntax (s.should.start_with?('b')), we get to see the receiver and arguments, no matter what predicate is used, and it’s immediately clear which expectation failed:

String#start_with? returns true only if beginning match FAILED
Expected "a".start_with? "b"
to be truthy but was false

And it’s all consistent, so once we learn the formatting of just one error, we know it for all other errors.

Expected 1 == 2
to be truthy but was false

Expected [1, 3].include? 2
to be truthy but was false

Previously, the message and formatting would differ for every matcher:

Expected 1 to equal 2

Expected [1, 3] to include 2

I no longer need to reverse map equal to == (how confusing!) and include to include?, it just shows me the method which was called and returned something unexpected.

Expected Actual to Equal Expected

Moreover actual and expected are often unclear in many test framework outputs, such as Expected ACTUAL to equal EXPECTED above. Some test-unit frameworks even disagree on which should be first for assert_equal. We no longer need a notion of actual and expected values in the new syntax, we always expect a whole predicate to be truthy or falsy, and that’s it. There is no question of order, it’s plain Ruby code and we can see == was called on the left receiver value, with the right value(s) as the argument(s).

Of course, this new syntax is not the answer to everything. For instance, it does not address the fact we need a different syntax for exceptions (and mocks). We could extend the new syntax to deal with exceptions as going through method_missing gives a lot of control, but I did not find a nice syntax for that yet.

Conclusion

The existing RSpec 2-like syntax will still remain available in MSpec, for compatibility and because migrating ruby/spec would be a very large amount of work. I also don’t want to force anyone to use the new syntax. I started using the new syntax in a few specs where I found it to be a clear gain, such as for start_with? specs. The most frequent matcher, should ==, already uses the new error output, because that matcher uses the same syntax in the new syntax and therefore the new implementation is used. As a fun fact, the implementation of the new syntax actually removes more lines than it adds.

In this blog post I presented a new expectation syntax for ruby/spec. The main advantages are:

What do you think? Is this new syntax a good or a bad idea?