bundle package --all is broken in bundler 1.11.0

[janost]

• What did you do?
  I ran the command /usr/local/bin/bundle package --all
• What did you expect to happen?
  I expected bundle package --all to work.
• What happened instead?
  Instead, the command seems to create vendor/cache/PROJECTNAME/vendor/cache/PROJECTNAME/... in and endless loop until the file system permits.

~ $ bundle -v
Bundler version 1.11.0
~ $ bundle gem something
Creating gem 'something'...
      create  something/Gemfile
      create  something/.gitignore
      create  something/lib/something.rb
      create  something/lib/something/version.rb
      create  something/something.gemspec
      create  something/Rakefile
      create  something/README.md
      create  something/bin/console
      create  something/bin/setup
Initializing git repo in /Users/user/something
~ $ cd something
~/something (master*) $ bundle package --all
Fetching gem metadata from https://rubygems.org/..
Fetching version metadata from https://rubygems.org/.
Resolving dependencies...
Using rake 10.4.2
Using bundler 1.11.0
Using something 0.1.0 from source at `.`
Bundle complete! 3 Gemfile dependencies, 3 gems now installed.
Use `bundle show [gemname]` to see where a bundled gem is installed.
Updating files in vendor/cache
  * rake-10.4.2.gem
--- ERROR REPORT TEMPLATE -------------------------------------------------------
- What did you do?

  I ran the command `/usr/local/bin/bundle package --all`

- What did you expect to happen?

  I expected Bundler to...

- What happened instead?

  Instead, what actually happened was...


Error details

    Errno::ENAMETOOLONG: File name too long - /Users/user/something/vendor/cache/something/vendor/cache/something/vendor/cache/something/vendor/cache/something/vendor/cache/something/vendor/cache/something/vendor/cache/something/vendor/cache/something/vendor/cache/som
ething/vendor/cache/something/vendor/cache/something/vendor/cache/something/vendor/cache/something/vendor/cache/something/vendor/cache/something/vendor/cache/something/vendor/cache/something/vendor/cache/something/vendor/cache/something/vendor/cache/something/vendor/cac
he/something/vendor/cache/something/vendor/cache/something/vendor/cache/something/vendor/cache/something/vendor/cache/something/vendor/cache/something/vendor/cache/something/vendor/cache/something/vendor/cache/something/vendor/cache/something/vendor/cache/something/vend
or/cache/something/vendor/cache/something/vendor/cache/something/vendor/cache/something/vendor/cache/something/vendor/cache/something/vendor/cache/something/vendor/cache/something/vendor/cache/something/vendor/cache/something/.git/hooks/prepare-commit-msg.sample
      /System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/lib/ruby/2.0.0/fileutils.rb:1381:in `initialize'
      /System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/lib/ruby/2.0.0/fileutils.rb:1381:in `open'
      /System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/lib/ruby/2.0.0/fileutils.rb:1381:in `block in copy_file'
      /System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/lib/ruby/2.0.0/fileutils.rb:1380:in `open'
      /System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/lib/ruby/2.0.0/fileutils.rb:1380:in `copy_file'
      /System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/lib/ruby/2.0.0/fileutils.rb:1348:in `copy'
      /System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/lib/ruby/2.0.0/fileutils.rb:463:in `block in copy_entry'
...

[unn]

Just slammed into this myself while testing our build process with the new version of bundler.

[segiddins]

Thanks for the bug report! Well try and debug unless someone beats us to making a PR with a fix.

[RochesterinNYC]

@segiddins I've been working on a PR for this but I'm a bit stuck.

It seems that the issue is that in Bundler::Runtime#cache, when we try to cache each specification, in packaging a gem, the gem tries to cache itself. This results in the following line in Bundler::Source::Path#cache being called:

# @original_path == '.'
# app_cache_path == '/Users/User/workspace/gem_name/vendor/cache/gem_name'
FileUtils.cp_r("#{@original_path}/.", app_cache_path)

This tries to copy all the contents of the current directory . (which is the directory of the current gem trying to be packaged) to what is essentially ./vendor/cache, which is itself included in the contents trying to be copied. It seems that FileUtils.cp_r would then keep recursively trying to copy these new contents into vendor/cache, which essentially creates new content that needs to be copied.

A potential fix I see is that in Bundler::Runtime#cache, there's a line that prevents the bundler spec from being cached:

next if spec.name == "bundler"

If we changed it to

next if spec.name == "bundler" || spec.name == "name_of_gem_being_packaged"

then the caching operation for the gem wouldn't try to cache itself and all of its directory contents. However, not sure if there's anywhere during runtime that we'd be able to pull the name of the gem being packaged from?

[glennpratt]

This still seems to occur.

https://gist.github.com/glennpratt/c6148b3724998103998a

[RochesterinNYC]

@glennpratt could you please open up a new issue? The command you're running seems to be slightly different/have extra flags. Additionally, in your issue, could you note whether running bundle package --all on your Gemfile/project causes the infinite recursion? Thanks!

[RochesterinNYC]

Can't replicate this with running bundle package --all --all-platforms --no-install with a fresh gem (i.e. after bundle gem something), so it must be something with your specific Gemfile or configuration that's causing Bundler to infinitely recurse.

[glennpratt]

@RochesterinNYC thanks for looking, I figured out how to reproduce it: #4392. Pretty easy to workaround.