A Migration Path to Bundler 2+
Bundler 2 did not arrive quietly.
It was noticed by almost every CI build failing when running
As a result, it seems many still avoid Bundler 2 and just use Bundler 1.
In this post, I present some ideas on how to get more people to use Bundler 2,
and no longer need Bundler 1 which will not be maintained forever.
The RubyGems Requirement of Bundler 2
The original release of Bundler 2, which is version 2.0.0, required such a recent RubyGems version that none of the released Ruby versions had a recent enough RubyGems version shipped with them. Bundler 2 also requires Ruby 2.3+, and finally drops support for Ruby 1.8.
This RubyGems version requirement was quickly found as problematic, and as a result Bundler 2.0.1 was released to lower the RubyGems version requirement to RubyGems 2.5. All versions of Ruby supported by Bundler 2 (that is, Ruby 2.3+) ship with RubyGems 2.5 or newer, but the blog post did not make this clear.
This resulted in a lot of confusion. Most repositories ended up explicitly updating RubyGems in CI to make Bundler 2.0.0 work, as mentioned in the original blog post.
To summarize, it is not needed to update RubyGems to use Bundler 2.0.1+.
In fact, I would even argue against updating RubyGems in CI because:
- It makes the CI slower.
- It makes the CI less reliable by frequently changing the RubyGems version used.
- It uses the latest RubyGems instead of the well known version bundled with that Ruby version.
Bundler Version Autoswitching
Bundler 2 came with another feature, which to say the least is very confusing. This feature is actually implemented directly in RubyGems and it unfortunately included a bug, which led to even more confusion.
Supposing you have both Bundler 1.17.2 and Bundler 2.0.2 installed, what should this print?
$ bundle --version
For every other gem, the answer would be the latest version of the gem installed, so in this case 2.0.2.
The actual answer though, due to Bundler version autoswitching is “it depends on many things”.
Specifically, if the current directory or any parent directory has a
Gemfile.lock file with a
BUNDLED WITH section,
then that exact version, or the closest version with the same major version (depending on the RubyGems version), will be used. If such a version is not available, it fails with
Could not find 'bundler' (<VERSION>) (e.g., when only Bundler 2 is installed with a
BUNDLED WITH 1.17.2).
Otherwise, if there is no
Gemfile.lock in the current directory or any parent directory,
bundle behaves like a normal gem and uses the latest version.
As an example, this feature means that as long as there is a
BUNDLED WITH 1.x,
even if Bundler 2 is installed, Bundler 1 would be used, or the error above would be shown.
I think this “feature” is so confusing that it should be considered a bug.
It’s counter-intuitive and prevents using Bundler 2 on
BUNDLED WITH 1.x.
The Migration Problem
The problem I see with all this is I think very few people use Bundler 2, and rather they just keep using Bundler 1. I tried to estimate how often Bundler 2 is used by various means:
- I made a poll on Twitter but that got rather few responses, with rather surprising results.
- Some usage statistics from RubyGems.org show that Bundler 2 is only used in about 2% of requests to RubyGems.org.
- TravisCI seems to currently use Bundler 1 for every Ruby version.
- Most Ruby projects I have seen with a
BUNDLED WITH 1.x, showing Bundler 1 is used.
However, Bundler 1 seems no longer maintained (which is fair enough, who would want to maintain Ruby 1.8 compatibility?) and likely will no longer receive bug fixes and security patches.
So it is important that Rubyists migrate to Bundler 2, but I think the current situation made it so inconvenient that very few did. To address those concerns, I propose some ideas below to make migration to Bundler 2 easier.
A Better Migration Path for Bundler 2+
Here are my suggestions to make migration to Bundler 2 easier. I think it would be good to integrate them in future Bundler releases, including 2.x releases.
- Do not require a RubyGems version newer than what Ruby versions supported by Bundler ship. Requiring to update RubyGems causes many troubles as seen wth Bundler 2.0.0.
- Drop the version autoswitching mechanism in RubyGems for Bundler entirely. For older RubyGems versions with autoswitching, do whatever is necessary in Bundler to avoid that behavior, or at least warn.
- Every future version of Bundler should support previous versions of
- Do not modify the
Gemfile.lockif there would be no other change than the
BUNDLED WITHversion (e.g., for
bundle installwith an existing up-to-date
This third point about backward compatibility might require extra maintenance work in Bundler.
However, I think almost every software of the scale of Bundler needs to do this kind of things,
and I would expect keeping a few extra parsers for older formats of
Gemfile.lock is not so much work.
For instance, even in Bundler 10, I expect installing gems from an already-resolved
Gemfile.lock is basically just installing every listed gem with the listed version.
Gemfile and not just
Gemfile.lock? Because Bundler evaluates the
Gemfile in all cases in its current design.
I think supporting older
Gemfile.lock is the key for more people to adopt newer versions of Bundler.
And importantly, this is the only way people can use newer versions of Bundler and stop using unmaintained and old Bundler versions for existing
Gemfile.lock files with
BUNDLED WITH 1.x (not all
Gemfile.lock will be migrated immediately to Bundler 2).
I shared these suggestions with the Bundler team in October 2019 and they generally agreed and already discussed these subjects at RubyKaigi 2019.
Some of the suggestions listed above have already been addressed. For instance, the RubyGems version requirement (#1) was fixed in 2.0.1, one day after the 2.0.0 release.
The latest RubyGems and Bundler 2 can actually already install a
BUNDLED WITH 1.x (#3), but it unfortunately changes the
BUNDLED WITH section for seemingly no reason (#4 not yet). This can be seen on 2.7.0-preview2:
$ chruby 2.7.0-preview2 $ bundle _2.1.0.pre.2_ install ... Warning: the lockfile is being updated to Bundler 2, after which you will be unable to return to Bundler 1. Bundle complete! 2 Gemfile dependencies, 7 gems now installed. Use `bundle info [gemname]` to see where a bundled gem is installed. $ git diff --- Gemfile.lock +++ Gemfile.lock BUNDLED WITH - 1.17.2 + 2.1.0.pre.2
That changes is annoying.
Suppose I’m contributing to Rails and I’m just doing
bundle install (without modifying any
If this changes the
Gemfile.lock I will have to constantly undo that change, or be careful to not commit it and keep a dirty working tree.
If it is committed and merged, it would force every single contributor of Rails (and the CI) to use Bundler 2 once they pull that change.
And every such contributor, when doing
bundle install on their own projects would also see the
BUNDLED WITH version updated, making this a cascading effect.
That sounds to me like a recipe for a really not smooth migration forcing everyone to update in a really short time, so that’s why I think Bundler should not touch the
Gemfile.lock if there would be no other change than the
BUNDLED WITH version (#4).
On the other hand, it seems fine to use
BUNDLED WITH 2.x if there is no
Gemfile.lock or the dependencies need to be re-resolved due to, e.g., adding a gem, since Bundler 2 might resolve gems differently than Bundler 1.
So #1 and #3 seem mostly done for the latest 2.x release, but #2 and #4 are not yet addressed.
Does that sound like a good migration plan to you?
Would you be willing to help making these changes in Bundler and RubyGems?