Written by Benoit Daloze @eregontp
on 

A Migration Path to Bundler 2+

Bundler 2 did not arrive quietly. It was noticed by almost every CI build failing when running bundle install. As a result, it seems many still avoid Bundler 2 and just use Bundler 1. In this post, I present some ideas on how to get more people to use Bundler 2, and no longer need Bundler 1 which will not be maintained forever.

The RubyGems Requirement of Bundler 2

The original release of Bundler 2, which is version 2.0.0, required such a recent RubyGems version that none of the released Ruby versions had a recent enough RubyGems version shipped with them. Bundler 2 also requires Ruby 2.3+, and finally drops support for Ruby 1.8.

This RubyGems version requirement was quickly found as problematic, and as a result Bundler 2.0.1 was released to lower the RubyGems version requirement to RubyGems 2.5. All versions of Ruby supported by Bundler 2 (that is, Ruby 2.3+) ship with RubyGems 2.5 or newer, but the blog post did not make this clear.

This resulted in a lot of confusion. Most repositories ended up explicitly updating RubyGems in CI to make Bundler 2.0.0 work, as mentioned in the original blog post.

To summarize, it is not needed to update RubyGems to use Bundler 2.0.1+.

In fact, I would even argue against updating RubyGems in CI because:

Bundler Version Autoswitching

Bundler 2 came with another feature, which to say the least is very confusing. This feature is actually implemented directly in RubyGems and it unfortunately included a bug, which led to even more confusion.

Supposing you have both Bundler 1.17.2 and Bundler 2.0.2 installed, what should this print?

$ bundle --version

For every other gem, the answer would be the latest version of the gem installed, so in this case 2.0.2.

The actual answer though, due to Bundler version autoswitching is “it depends on many things”. Specifically, if the current directory or any parent directory has a Gemfile.lock file with a BUNDLED WITH section, then that exact version, or the closest version with the same major version (depending on the RubyGems version), will be used. If such a version is not available, it fails with Could not find 'bundler' (<VERSION>) (e.g., when only Bundler 2 is installed with a Gemfile.lock BUNDLED WITH 1.17.2).

Otherwise, if there is no Gemfile.lock in the current directory or any parent directory, bundle behaves like a normal gem and uses the latest version.

As an example, this feature means that as long as there is a Gemfile.lock BUNDLED WITH 1.x, even if Bundler 2 is installed, Bundler 1 would be used, or the error above would be shown.

I think this “feature” is so confusing that it should be considered a bug. It’s counter-intuitive and prevents using Bundler 2 on Gemfile.lock BUNDLED WITH 1.x.

The Migration Problem

The problem I see with all this is I think very few people use Bundler 2, and rather they just keep using Bundler 1. I tried to estimate how often Bundler 2 is used by various means:

However, Bundler 1 seems no longer maintained (which is fair enough, who would want to maintain Ruby 1.8 compatibility?) and likely will no longer receive bug fixes and security patches.

So it is important that Rubyists migrate to Bundler 2, but I think the current situation made it so inconvenient that very few did. To address those concerns, I propose some ideas below to make migration to Bundler 2 easier.

A Better Migration Path for Bundler 2+

Here are my suggestions to make migration to Bundler 2 easier. I think it would be good to integrate them in future Bundler releases, including 2.x releases.

  1. Do not require a RubyGems version newer than what Ruby versions supported by Bundler ship. Requiring to update RubyGems causes many troubles as seen wth Bundler 2.0.0.
  2. Drop the version autoswitching mechanism in RubyGems for Bundler entirely. For older RubyGems versions with autoswitching, do whatever is necessary in Bundler to avoid that behavior, or at least warn.
  3. Every future version of Bundler should support previous versions of Gemfile and Gemfile.lock.
  4. Do not modify the Gemfile.lock if there would be no other change than the BUNDLED WITH version (e.g., for bundle install with an existing up-to-date Gemfile.lock).

This third point about backward compatibility might require extra maintenance work in Bundler. However, I think almost every software of the scale of Bundler needs to do this kind of things, and I would expect keeping a few extra parsers for older formats of Gemfile.lock is not so much work. For instance, even in Bundler 10, I expect installing gems from an already-resolved Gemfile.lock is basically just installing every listed gem with the listed version. Why Gemfile and not just Gemfile.lock? Because Bundler evaluates the Gemfile in all cases in its current design.

I think supporting older Gemfile and Gemfile.lock is the key for more people to adopt newer versions of Bundler. And importantly, this is the only way people can use newer versions of Bundler and stop using unmaintained and old Bundler versions for existing Gemfile.lock files with BUNDLED WITH 1.x (not all Gemfile.lock will be migrated immediately to Bundler 2).

Conclusion

I shared these suggestions with the Bundler team in October 2019 and they generally agreed and already discussed these subjects at RubyKaigi 2019.

Some of the suggestions listed above have already been addressed. For instance, the RubyGems version requirement (#1) was fixed in 2.0.1, one day after the 2.0.0 release.

The latest RubyGems and Bundler 2 can actually already install a Gemfile.lock BUNDLED WITH 1.x (#3), but it unfortunately changes the BUNDLED WITH section for seemingly no reason (#4 not yet). This can be seen on 2.7.0-preview2:

$ chruby 2.7.0-preview2
$ bundle _2.1.0.pre.2_ install
...
Warning: the lockfile is being updated to Bundler 2,
after which you will be unable to return to Bundler 1.
Bundle complete! 2 Gemfile dependencies, 7 gems now installed.
Use `bundle info [gemname]` to see where a bundled gem is installed.
$ git diff
--- Gemfile.lock
+++ Gemfile.lock
 BUNDLED WITH
-   1.17.2
+   2.1.0.pre.2

That changes is annoying. Suppose I’m contributing to Rails and I’m just doing bundle install (without modifying any Gemfile). If this changes the Gemfile.lock I will have to constantly undo that change, or be careful to not commit it and keep a dirty working tree. If it is committed and merged, it would force every single contributor of Rails (and the CI) to use Bundler 2 once they pull that change. And every such contributor, when doing bundle install on their own projects would also see the BUNDLED WITH version updated, making this a cascading effect. That sounds to me like a recipe for a really not smooth migration forcing everyone to update in a really short time, so that’s why I think Bundler should not touch the Gemfile.lock if there would be no other change than the BUNDLED WITH version (#4).

On the other hand, it seems fine to use BUNDLED WITH 2.x if there is no Gemfile.lock or the dependencies need to be re-resolved due to, e.g., adding a gem, since Bundler 2 might resolve gems differently than Bundler 1.

So #1 and #3 seem mostly done for the latest 2.x release, but #2 and #4 are not yet addressed.

Does that sound like a good migration plan to you?

Would you be willing to help making these changes in Bundler and RubyGems?