Downloading URL artifact can cause Errno::ENAMETOOLONG: File name too long

[bbaugher]

Looks like the code generates a unique name for the cached file by converting the URL into Base64. This can then easily go over the 255 character file name limit if the URL is long enough

(/tmp/kitchen/cache/cookbooks/poise-archive/files/halite_gem/poise_archive/archive_providers/base.rb line 84) had an error: Errno::ENAMETOOLONG: File name too long @ utime_internal - /tmp/kitchen/cache/aHR0cDovL3JlcG8uYmlnZGF0YS5jZXJuZXIuY29ycC9uZXh1cy9jb250ZW50L3JlcG9zaXRvcmllcy9iaWdkYXRhLXNuYXBzaG90L2NvbS9jZXJuZXIvdm9sdGUvdm9sdGUtdG9vbHMvMS4wLVNOQVBTSE9UL3ZvbHRlLXRvb2xzLTEuMC0yMDE2MTIwNi4xODA2MjctMjUtYXNzZW1ibHkudGFyLmd6_some-tools-1.0-20161206.180627-25-assembly.tar.gz
/opt/chef/embedded/lib/ruby/2.3.0/fileutils.rb:1154:in `utime'
/opt/chef/embedded/lib/ruby/2.3.0/fileutils.rb:1154:in `block in touch'
/opt/chef/embedded/lib/ruby/2.3.0/fileutils.rb:1151:in `each'
/opt/chef/embedded/lib/ruby/2.3.0/fileutils.rb:1151:in `touch'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.16.42/lib/chef/file_content_management/deploy/mv_unix.rb:34:in `create'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.16.42/lib/chef/provider/file.rb:375:in `block in do_create_file'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.16.42/lib/chef/mixin/why_run.rb:52:in `add_action'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.16.42/lib/chef/provider.rb:176:in `converge_by'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.16.42/lib/chef/provider/file.rb:374:in `do_create_file'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.16.42/lib/chef/provider/file.rb:153:in `action_create'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.16.42/lib/chef/provider.rb:145:in `run_action'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.16.42/lib/chef/resource.rb:622:in `run_action'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.16.42/lib/chef/runner.rb:69:in `run_action'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.16.42/lib/chef/runner.rb:97:in `block (2 levels) in converge'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.16.42/lib/chef/runner.rb:97:in `each'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.16.42/lib/chef/runner.rb:97:in `block in converge'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.16.42/lib/chef/resource_collection/resource_list.rb:94:in `block in execute_each_resource'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.16.42/lib/chef/resource_collection/stepable_iterator.rb:114:in `call_iterator_block'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.16.42/lib/chef/resource_collection/stepable_iterator.rb:85:in `step'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.16.42/lib/chef/resource_collection/stepable_iterator.rb:103:in `iterate'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.16.42/lib/chef/resource_collection/stepable_iterator.rb:55:in `each_with_index'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.16.42/lib/chef/resource_collection/resource_list.rb:92:in `execute_each_resource'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.16.42/lib/chef/runner.rb:96:in `converge'
/tmp/kitchen/cache/cookbooks/poise/files/halite_gem/poise/helpers/notifying_block.rb:69:in `notifying_block'
/tmp/kitchen/cache/cookbooks/poise-archive/files/halite_gem/poise_archive/archive_providers/base.rb:82:in `download_file'
/tmp/kitchen/cache/cookbooks/poise-archive/files/halite_gem/poise_archive/archive_providers/base.rb:59:in `action_unpack'

[coderanger]

Sad trombone dot gif, didn't think of this. I should probably check what the limit on Windows is too.

[coderanger]

Because you filed the bug you get to have an opinion, switch to a hash for anything over 255 chars (or whatever the max on Windows is)? It won't be as nicely readable but should be better than nothing.

[bbaugher]

I'm not totally sure I understand what we get out of adding anything like that to the file name.

[coderanger]

We need some kind of unique name for the cache file, the goal with Base64 was to make something that could be reversed to the original URL during manual debugging if needed, while a hash could not be, but is fixed-length.

[bbaugher]

I would have thought converting URL to the filename would be sufficient for most if not all. Generating a hash seems simplest. I don't think you lose a lot from doing that

[coderanger]

I did think that, but as you noticed this can make invalid filenames when very long ;-)

[bbaugher]

I meant converting http://www-eu.apache.org/dist/kafka/0.10.1.0/kafka_2.10-0.10.1.0.tgz into kafka_2.10-0.10.1.0.tgz.

[coderanger]

Unfortunately that is not unique enough, I ran into problems early on with downloads of the same name but in different folders.

[coderanger]

Anyways, I'll fix this up shortly.

[tanner-bruce]

What about hashing the URL with SHA1? We are also running into this problem. I can submit a PR if this solution is acceptable.

[coderanger]

I changed this a while ago to use a base64 of the resource name and only the last path component of the URL, you're still hitting something too long though?

[gchpaco]

Oh yes. Especially with consul-template. Why not just use Digest::SHA256.base64digest?

[gchpaco]

An example:

[2017-05-05T16:34:45+00:00] ERROR: consul_template_installation[0.16.0] (test_consul_template::default line 19) had an error: Errno::ENAMETOOLONG: poise_archive[https://releases.hashicorp.com/consul-template/0.16.0/consul-template_0.16.0_linux_amd64.zip] (/opt/kitchen/cache/cookbooks/consul_template/libraries/consul_template_installation_binary.rb line 42) had an error: Errno::ENAMETOOLONG: remote_file[/opt/kitchen/cache/aHR0cHM6Ly9yZWxlYXNlcy5oYXNoaWNvcnAuY29tL2NvbnN1bC10ZW1wbGF0ZS8wLjE2LjAvY29uc3VsLXRlbXBsYXRlXzAuMTYuMF9saW51eF9hbWQ2NC56aXA_consul-template_0.16.0_linux_amd64.zip] (/opt/kitchen/cache/cookbooks/poise-archive/files/halite_gem/poise_archive/archive_providers/base.rb line 84) had an error: Errno::ENAMETOOLONG: File name too long @ utime_internal - /opt/kitchen/cache/aHR0cHM6Ly9yZWxlYXNlcy5oYXNoaWNvcnAuY29tL2NvbnN1bC10ZW1wbGF0ZS8wLjE2LjAvY29uc3VsLXRlbXBsYXRlXzAuMTYuMF9saW51eF9hbWQ2NC56aXA_consul-template_0.16.0_linux_amd64.zip

[coderanger]

Ahh okay, so the part of the workaround I didn't mention is you can use the path property directly rather than the resource name for the long URL, I'll fix that in the next release to be smarter though.