Fetch deps based on urls #935

stevendanna · 2016-08-18T13:24:58Z

This extends the dependency feature to include support for url-based
dependencies. It takes some deviations from the current support for
URLs that we'll likely want to make more consistent.

By default, we store downloaded archives in the cache rather than the
unpacked archive. However, to facilitate debugging, we will prefer the
unpacked archive if we find it in the cache.

This partially addresses #915 by:

Supporting URL based sources (since most of our sources will resolve to URL based sources, the rest of the sources should be straightforward additions)
Storing downloaded dependencies in the vendor cache
Using the cached dependencies when they exist

Still to do:

Integrate other source types
Write out a inspec.lock file based on the fully resolved dependencies
Read the lock file rather than inspec.yml when it exists

Signed-off-by: Steven Danna [email protected]

chris-rock · 2016-08-18T19:42:16Z

Great work @stevendanna I really like the split up into smaller pieces. I would prefer to store all dependency related classes under lib/inspec/dependencies

chris-rock · 2016-08-18T19:42:39Z

lib/inspec/dependencies.rb

-require 'molinillo'
-require 'inspec/errors'
-require 'inspec/requirement'
+require 'inspec/vendor_index'


👍 on the splitup

chris-rock · 2016-08-18T19:59:43Z

We need to be careful with the cwd:

inspec exec test/unit/mock/profiles/dependencies-file/inheritance 
Could not fetch inspec profile in "/Users/chartmann/Development/compliance/profile_a".

stevendanna · 2016-08-18T20:03:45Z

@chris-rock Re the current working directory, I was going back and forth about what people expect there. Currently, as you can see, it is using the current directory, but we could get the base directory of the inspec.yml at the head of the tree if you think that is what people would expect.

chris-rock · 2016-08-18T20:03:58Z

test/unit/mock/profiles/dependencies-file/inheritance/inspec.yml

@@ -10,3 +10,5 @@ depends:
    path: ../profile_a
  - name: profile_b
    path: ../profile_b
+  - name: os-hardening


we should rename the parent directory to depndencies, since we are testing more complex content here that files. we may split that up later

chris-rock · 2016-08-18T20:15:00Z

I think, users would expect that they are relative to the partent profile location. I just tried to execute it that way as I normally execute profiles

chris-rock · 2016-08-18T20:41:19Z

@stevendanna Could you do me a favor and highlight, which parts of #915 are addressed in that PR?

stevendanna · 2016-08-18T21:39:55Z

@chris-rock I've updated this with:

Requested renames
A small fix for the relative paths (your example should now work)
Basic comments for each new class

I am 👍 on the change you recommended for fetcher but I think we should do that in a follow up PR.

- Move classes into their own files - Remove classes that aren't used Signed-off-by: Steven Danna <[email protected]>

This extends the dependency feature to include support for url-based dependencies. It takes some deviations from the current support for URLs that we'll likely want to make more consistent. By default, we store downloaded archives in the cache rather than the unpacked archive. However, to facilitate debugging, we will prefer the unpacked archive if we find it in the cache. Signed-off-by: Steven Danna <[email protected]>

Signed-off-by: Steven Danna <[email protected]>

chris-rock · 2016-08-26T08:38:11Z

related to #888