Xcode 9 support

[ksuther]

Additional Xcode 9 support (thank you to #321 and #338).
Most of the tests pass when run against Xcode 9, but 4 tests still fail due to expecting Xcode 8's directory layout. These tests can be updated once Travis is switched over to Xcode 9.

[coveralls]

Coverage decreased (-0.3%) to 95.435% when pulling 8f6c451 on ksuther:xcode9 into f478561 on SlatherOrg:master.

[yakimant]

@ksuther, works for me for both Xcode 9 GM and 8.3.2.

[dcordero]

It works like a charm also here using Xcode 9 GM

[mikelupo]

@ksuther, Curious what the plan is for xcode 9 updates such as this one? Thanks in advance!

[ksuther]

Hoping to do it today. Sorry for the wait!

[mikelupo]

No problem @ksuther. Thank you for the reply.
You may be interested to know that I tried to test the change of this commit against my installed 2.4.2 gem. (I edited project.rb by hand), it's not finding a coverage directory.
What's concerning is that searching my project's directory(which includes the build dir), using the following you can see that it is finding the Coverage.profdata file,
find . -iname Coverage.profdata
./build/Build/ProfileData/f45d0bf751c75159bc5b0c563569744d56bc4779/Coverage.profdata

This is Xcode9, with iPhone 7Plus running iOS 10.3.3
What can I do to help debug?

[ksuther]

What do you get if you run slather with --verbose? Are you building through Xcode or with xcodebuild and specifying a custom derivedDataDir? The first thing I'd check is if build_directory or profdata_coverage_dir are correct in project.rb.

[mikelupo]

--verbose does not print out anything more than I'm already getting printed. Digging a little deeper.
This is a scripted build that builds using xcodebuild. in project.rb, I'll print out those two variables and see what the values are.

[mikelupo]

looks like build_directory is fine. it's the profdata_coverage_dir that's not getting resolved. I added another case. This resolved dir problem.

    if dir == nil
        dir = Dir[File.join("#{build_directory}","/**/ProfileData")].first
    end

With that above it finds the Coverage.profdata file.

The next problem is that no product binary is being found.
No product binary found in /Users/shared/jenkins/slave/workspace/My_IOS_TEST/MySdk/build/Build/ProfileData.

Any ideas? should this be finding our .framework file? No .framework file is in the above mentioned location.

[ksuther]

What is your xcodebuild command? The root problem may be that slather isn't able to automatically detect your directories because it is using xcodebuild -showBuildSettings to determine the path of directories, but if you're overriding those at build time then they won't match.

[mikelupo]

ok, I'm trying to get the build it print the entire string. I'll advise when I know for certain. Thank you again for the insight.

[mikelupo]

I got the xcodebuild command.

xcodebuild test -project MySdk.xcodeproj \
-scheme MySdkNightlyTests \ 
-sdk iphoneos \
-configuration Debug \
-derivedDataPath /Users/shared/jenkins/slave/workspace/My_TEST_JOB/MySdk/build \ 
-destination name=iPhone \ 
SHARED_PRECOMPS_DIR="/Users/shared/jenkins/slave/workspace/My_TEST_JOB/MySdk/build/precompiled_headers"  \
ARCHS=arm64 VALID_ARCHS=arm64 ONLY_ACTIVE_ARCH=YES 

Is passing in derivedDataPath the problem? We have been doing this for a few years now and using slather with no issue, not till we moved to Xcode 9.

For convenience, below is my slather command (slightly edited to get rid of a bunch of --ignore paths.

/usr/local/bin/slather coverage 
--input-format profdata 
--cobertura-xml  
--output-directory slather-report 
--scheme MySdk 
--build-directory /Users/shared/jenkins/slave/workspace/My_TEST_JOB/MySdk/build
--workspace /Users/shared/jenkins/slave/workspace/My_TEST_JOB/My.xcworkspace 
MySdk.xcodeproj

[ksuther]

What does your ProfileData/UUID/Coverage.profdata directory structure look like? I'm trying something similar to you and it works fine.

The check for the Xcode version and coverage_files = Dir[File.join(build_directory, "/**/ProfileData/*/Coverage.profdata")] should be equivalent to the change that you made. Can you check if Slather.xcode_version is [9, 0]?

find_binary_files also has an Xcode 9-specific workaround, so if that's not working either then maybe slather doesn't think you're on Xcode 9.

[mikelupo]

Coverage.profdata itself is a file, but if you're referring to their parent directory, this is what the UUID dir contains:
Coverage.profdata
MyTestHostApp-405.profraw

[ksuther]

I meant the directory structure leading up to Coverage.profdata. Did you verify that xcode_version is 9?

[tzm41]

Yes! thanks for doing this @ksuther

[mikelupo]

puts Slather.xcode_version[0]
prints 9

pwd && ls -la
/Users/shared/jenkins/slave/workspace/My_SDK_Test/MySdk/build/Build
total 0
drwxr-xr-x 6 jenkins admin 204 Sep 19 13:36 .
drwxr-xr-x@ 8 jenkins admin 272 Sep 19 13:35 ..
drwxr-xr-x@ 5 jenkins admin 170 Sep 19 13:35 Intermediates
drwxr-xr-x@ 6 jenkins admin 204 Sep 19 13:35 Intermediates.noindex
drwxr-xr-x@ 3 jenkins admin 102 Sep 19 13:35 Products
drwxr-xr-x 3 jenkins admin 102 Sep 19 13:36 ProfileData

ProfileData contains only the uuid which in turn contains the Coverage.profdata and profraw file.
./ProfileData/f45d0bf751c75159bc5b0c563569744d56bc4779/

Thanks!

[mikelupo]

@ksuther , I realize this is merged. And I'm still trying to understand why it's still broke for us in particular.

I have a question in regards to the error case.
With xcode 9, I can confirm that your code appends to the search_dir (search_dir = File.join(search_dir, '../..'))

However, is the error that is raised containing the correct path? it's currently printing a different directory path than the search_dir.

raise StandardError, "No product binary found in #{profdata_coverage_dir}." unless found_binaries.count > 0

Thanks,
Mike

[ksuther]

That error probably needs to be updated. It's outside of the branch where search_dir is defined, so that probably needs to be lifted out of the conditional.

search_dir is just profdata_coverage_dir with ../.. appended, so it should be easy to figure out where it's looking.

Are you able to reproduce the problem in a sample project with the same xcodebuild command? If so, maybe I can debug things that way. Otherwise it's hard to give more specific suggestions other than tracing out what find_binary_files does.