No DidYouMean constants in Travis CI

[styd]

Hi @yuki24!

I was trying to use DYM for my gem's KeyError. So, I added this code:

module DidYouMean
  SPELL_CHECKERS.merge!({
    'MyGem::KeyError' => KeyErrorChecker
  })
end

It's working in my local computer, but failed in Travis CI, saying uninitialized constant DidYouMean::SPELL_CHECKERS.
So I added this code to print DidYouMean methods and constants inside the module:

puts "Methods: " + methods.inspect
puts "Constants: " + constants.inspect

To my surprise, it was printing these lines in Travis CI:

Methods: [...] # a lot of methods
Constants: [] # empty

This only happens in CI. It's fine in my local computer.
Do you know what causes this?

[yuki24]

Ok so looking at https://travis-ci.org/styd/smart_kv/jobs/497686270, it's possible that the did_you_mean gem isn't installed. RVM does not install bundled gems for some reason, and it may have to be installed explicitly in Travis. I think adding:

gem 'did_you_mean'

to your Gemfile will fix your failure on CI. This ways bundler will pull in an apprppriate version for the Ruby version so you don't have to worry about installing it in .travis.yml.

Update: I no longer suggest adding did_you_mean to the Gemfile as it causes more problems depending on the version of Rubygems or Bundler. I would recommend manually installing it with e.g. before_install: gem i did_you_mean in Travis or whatever interface is provided by the choice of your CI.

Update 2: Further investigation revealed that the first update may not work in some cases. Please see the comment below for more details.

[styd]

@yuki24, thank you for looking into my gem's travis log.

I tried your suggestion, but now error happens before running the test.
In ruby 2.5, the error was:

Gem::LoadError: You have already activated did_you_mean 1.2.0, but your Gemfile requires did_you_mean 1.3.0

In ruby 2.6, the error was:

Could not find did_you_mean-1.3.0 in any of the sources
Run `bundle install` to install missing gems.

So, I tried adding bundle install to before_install in travis.yml (which apparently is redundant, because the CI is calling bundle install again after that but with some options/flags).
Surprisingly, my test are now running successfully in ruby 2.5, but ruby 2.6 still outputting the same error.

So, I have two confusions now:

1. why the gem built successfully in ruby 2.5 just because I added another bundle install (so, now travis is running it twice).
2. while in ruby 2.6, adding another bundle install doesn't work.

[yuki24]

@styd I'll look into your repository since it's open and I should be able to look into it.

[styd]

@yuki24, thank you for taking the trouble to help me.

[jeroenj]

We've been having the same issue since we upgraded to Ruby 2.6 (and thus rubygems 3.0). Basically if the version defined in your Gemfile is the same as the one bundled with Ruby, rubygems 3.0 no longer installs it locally. However with rugyems 2.7.9 it did install it locally. That's also why it didn't appear on Ruby 2.5 (if you didn't upgrade ruygems, since that ships with rubygems 2.7 by default).

The reason you'd want to have it locally installed (e.g. when using bundler's --path option) is because if it doesn't install locally bundler can't find it (even though it's installed on the system with Ruby's bundled gems):

/usr/local/bundle/gems/bundler-2.0.1/lib/bundler/spec_set.rb:87:in `block in materialize': Could not find did_you_mean-1.3.0 in any of the sources (Bundler::GemNotFound)

You can reproduce this issue with the ruby:2.6.2-slim Docker image (which currently contains rubygems 3.0.3 and bundler 1.17.2):

$ docker run -it --rm ruby:2.6.2-slim bash
$ gem list did_you_mean | grep did_you_mean
did_you_mean (1.3.0)
$ printf "source 'https://rubygems.org'\ngem 'did_you_mean', '1.3.0'" > Gemfile
$ bundle install --path bundle
Fetching gem metadata from https://rubygems.org/...
Resolving dependencies...
Using bundler 1.17.2
Using did_you_mean 1.3.0
Bundle complete! 1 Gemfile dependency, 2 gems now installed.
Bundled gems are installed into `./bundle`

As you can see did_you_mean is not being installed but instead used from the system gems.

However if you try this with minitest for example (which just like did_you_mean is just bundled with ruby) it does install it:

$ docker run -it --rm ruby:2.6.2-slim bash
$ gem list minitest | grep minitest
did_you_mean (5.11.3)
$ printf "source 'https://rubygems.org'\ngem 'minitest', '5.11.3'" > Gemfile
$ bundle install --path bundle
Fetching gem metadata from https://rubygems.org/.............
Resolving dependencies...
Using bundler 1.17.2
Fetching minitest 5.11.3
Installing minitest 5.11.3
Bundle complete! 1 Gemfile dependency, 2 gems now installed.
Bundled gems are installed into `./bundle`

If you use bundler 2.0 it also does not install did_you_mean:

$ docker run -it --rm ruby:2.6.2-slim bash
$ gem install bundler -v 2.0.1
Fetching bundler-2.0.1.gem
Successfully installed bundler-2.0.1
1 gem installed
$ gem list did_you_mean | grep did_you_mean
did_you_mean (1.3.0)
$ printf "source 'https://rubygems.org'\ngem 'did_you_mean', '1.3.0'" > Gemfile
$ bundle install --path bundle
Fetching gem metadata from https://rubygems.org/...
Resolving dependencies...
Using bundler 2.0.1
Using did_you_mean 1.3.0
Bundle complete! 1 Gemfile dependency, 2 gems now installed.
Bundled gems are installed into `./bundle`

However, downgrading rubygems to the latest 2.7.x release does make it so that did_you_mean gets installed:

$ docker run -it --rm ruby:2.6.2-slim bash
$ gem update --system 2.7.9
Updating rubygems-update
Fetching rubygems-update-2.7.9.gem
Successfully installed rubygems-update-2.7.9
Installing RubyGems 2.7.9
Bundler 1.16.6 installed
RubyGems 2.7.9 installed
Regenerating binstubs
------------------------------------------------------------------------------
RubyGems installed the following executables:
        /usr/local/bin/gem
        /usr/local/bin/bundle
RubyGems system software updated
$ gem list did_you_mean | grep did_you_mean
did_you_mean (1.3.0)
$ printf "source 'https://rubygems.org'\ngem 'did_you_mean', '1.3.0'" > Gemfile
$ bundle install --path bundle
Fetching gem metadata from https://rubygems.org/...
Resolving dependencies...
Using bundler 1.16.6
Fetching did_you_mean 1.3.0
Installing did_you_mean 1.3.0
Bundle complete! 1 Gemfile dependency, 2 gems now installed.
Bundled gems are installed into `/bundle`

We have been using a downgraded rubygems for a while now. But we switched to this workaround by tricking rubygems into thinking the bundled did_you_mean is not present (apparently for that removing the gemspec is enough):

$ docker run -it --rm ruby:2.6.2-slim bash
$ rm /usr/local/lib/ruby/gems/2.6.0/specifications/did_you_mean-1.3.0.gemspec
$ gem list did_you_mean | grep did_you_mean # returns nothing as it's no longer found
$ printf "source 'https://rubygems.org'\ngem 'did_you_mean', '1.3.0'" > Gemfile
$ bundle install --path bundle
Fetching gem metadata from https://rubygems.org/...
Resolving dependencies...
Using bundler 1.17.2
Fetching did_you_mean 1.3.0
Installing did_you_mean 1.3.0
Bundle complete! 1 Gemfile dependency, 2 gems now installed.
Bundled gems are installed into `./bundle`

To me this looks like an issue with rubygems 3.0. But the weird thing is that it only seems to affect did_you_mean. I have tried the same thing with minitest (see above) and rake and both actually get installed locally (even though they're exactly the same versions as the ones bundled with ruby).

I have yet to try to track down what exactly causes the difference (either in rubygems or in did_you_mean). But for now either downgrading rubygems to 2.7.9 or removing the system installed did_you_mean works around the issue.

[yuki24]

@jeroenj Thanks for reporting. Something is definitely off here and I'll look into it.

[yuki24]

Okay, I thought there were two distinct issues here but turns out it's actually just one issue that awfully affects Bundler's --path bundle option.

1. Ruby forcefully loads the did_you_mean gem on every boot
2. When the --path option is specified on bundle install, the CLI sets a new path to Bundler.settings, which puts Bundler in a clean state (== no gems are installed other than the default gems)
3. However, by the time it tries to install external gems to the specified path, DYM is already loaded (Bundler looks at Gem::Specification.stubs in Rubygems to determine what gems are already loaded). This makes Bundler think that it's a default gem (which is obviously incorrect)
4. Because of that Bundler thinks DYM is already installed and it skips installing it
5. When running bundle exec after the completion of bundle install --path ..., Bundler will make all default gems and the gems installed in the path (notice that DYM is missing due to step 4)
6. Ruby will fail to look up the DidYouMean constant at runtime

This commit in Rubygems rubygems/rubygems@42bba27#diff-05477d38d75c98664b0e7dec6eb73b45R823 seems to have introduced this behavior by accident, but we can't just revert it otherwise it may break other things.

As @jeroenj pointed out, Rubygems 2.7.9 does not have this issue becaue the commit mentioned above is not part of the version 2.7.9.

The other workaround I came up with is to specify RUBYOPT='--disable-did_you_mean' on bundle install --path bundle. This also requires Gemfile to have gem 'did_you_mean'. This way Ruby will temporarily disable DYM and Bundler will install it to the specified path properly:

# In Gemfile
gem 'did_you_mean'

RUBYOPT='--disable-did_you_mean' bundle install --path bundle

I think we should promote DYM up to a default gem to avoid confusions like this. It's very odd that it gets loaded with no clear notice despite the fact that it can be missing at any time.

[jeroenj]

What an investigation. 👏 Thanks for looking into this this quickly.

I’d indeed think promoting did_you_mean to a default gem makes sense here. Now I also understand why minitest didn’t have the issue.

[jeroenj]

Sad to report back here, but bundler 2.0.2 broke the RUBYOPT='--disable-did_you_mean' bundle install --path bundle workaround. 😅

I was able to track it down to this change: https://github.com/bundler/bundler/pull/7067/files#diff-d08082204d5579c541f2c1206ffeb652R4

If I understand this (and the original issue (#117 (comment))) correctly this is caused by (the vendored version of) thor now requiring did_you_mean.

It seems that thor (master) no longer requires did you mean: rails/thor@8c57f1d. So a future version of bundler (with an updated version of thor) might no longer have that issue.

So if I understand it all correctly none of this would be an issue of did_you_mean would be promoted to a default gem, right?

[eregon]

Agreed having did_you_mean as a default gem would make a lot of sense since it's required by default during startup.
@hsbt What do you think?

[hsbt]

I'm also +1 to promote did_you_mean to default gems.

[deivid-rodriguez]

Me too, for what's it's worth.

[kddnewton]

I opened a PR for it, though I'm not sure if I did it correctly. Feel free to leave comments over there ^^^^

[yuki24]

closing in favor of ruby/ruby#2689.