Migrate CI build to Azure DevOps

[mallibone]

Currently the CI build relies on Travis-CI and App-Veyor to build on macOS and Windows. Would it be in the interest of the project if one would migrate the build to Azure DevOps?
The improvements would be:

• One CI config as yaml
• Improved build times compared to Travis as the the Xamarin toolchain is pre-installed.

But a maintainer would have to hook it up to a Azure DevOps portal.

[TimLariviere]

That could be a good thing, especially with the new GTK template that may need a Linux build.
Do you have some data on how much faster the build may be?
Not sure if I have enough rights on this repo to hook it to Azure DevOps though.

[mallibone]

That is a good question regarding the build time, but I will try it out on my fork and get back to you when I have some details (but I hope a lot less than 20 minutes 🙃). I can also add a Linux agent to the mix. GTK should also build on macOS and Windows. But the UI usually is sub par when running it on Windows - at least the default app of Fabulous looks off on Windows.

[mallibone]

So I had a run at it under Azure DevOps. There is an issue with installing FAKE or in general any .Net Core tools. While I got it working for Windows (build Time is around 5 Minutes), under macOS I ran into another problem due to .Net Core not being installed at the default location. Linux fails for yet another reason (build failure -> need to investigate further). That being said I will keep at it 🙂

[slang25]

I had the same issue for one of my projects, they are planning to fix it (set the DOTNET_ROOT to the appropriate location). The only way I could get it to work was to install a local copy of the dotnet SDK.

[slang25]

Here's the issue: https://github.com/Microsoft/azure-pipelines-image-generation/issues/531

[mallibone]

Thanks for pointing that out, I have now the Windows and macOS build running. The Linux build fails because it fails to find RELEASE_NOTES.MD which is strange since when running the build on a local VM it passes with no error.

Build times:

• Windows ~6 minutes
• macOS ~4 minutes

The builds run in parallel so only the longest duration needs to be taken into account.

[TimLariviere]

@mallibone I fixed this issue in a ninja commit the other day. Wrong casing on the extension that only Linux sees. 2c3df06

Impressive build time.
Did you try running ./build.sh Test ?

[mallibone]

@TimLariviere rebase from upstream did solve that issue on Linux - but the next one is already here 🙈

Adjusted to running build.sh Test now the tests are failing on the Azure DevOps mac agent... when running the tests on my local mac all is fine. I will have to dig a bit deeper to find out why / which tests are failing (still a bit new to Fabulous Core to figure that one out).

Here is the overview to my (sofar) unsuccseful build history...

[TimLariviere]

The error comes from the interpreter. FSharpChecker fails to parse the code sent by the unit tests.
This line returns an Aborted result.
https://github.com/fsprojects/Fabulous/blob/5150a8f3a4d6c0d43247a15628cd3501e90ad7e4/src/Fabulous.Cli/fscd.fs#L134

An error message should be associated about why it fails. (need a slight change to print the error message into the build output)

[TimLariviere]

@mallibone The current Travis CI randomly fails while installing Xamarin.Android SDK.
Switching to Azure Pipelines will prevent those errors?

Warning: An error occurred while preparing SDK package Android SDK Build-Tools 27.0.1: Network is unreachable (connect failed).

--2019-01-17 11:20:11--  https://dl.xamarin.com/MonoTouch/Mac/xamarin.ios-11.10.1.178.pkg

Resolving dl.xamarin.com (dl.xamarin.com)... failed: nodename nor servname provided, or not known.

wget: unable to resolve host address ‘dl.xamarin.com’

--2019-01-17 11:20:11--  https://dl.xamarin.com/MonoforAndroid/Mac/xamarin.android-8.3.0-19.pkg

Resolving dl.xamarin.com (dl.xamarin.com)... failed: nodename nor servname provided, or not known.

wget: unable to resolve host address ‘dl.xamarin.com’

travis_time:end:004c4c68:start=1547723641667864000,finish=1547724011503038000,duration=369835174000
�[0K�[31;1mThe command "./install-xamarin-android-ios.sh" exited with 1.�[0m


------------------------------------------------------------


�[39;49m�[39;49m�[37m  /Users/travis/build/fsprojects/Fabulous/testapp222/testapp222.Android/testapp222.Android.fsproj(194,11):
error MSB4226: The imported project "/Library/Frameworks/Mono.framework/Versions/5.18.0/lib/mono/xbuild/Xamarin/Android/Xamarin.Android.FSharp.targets" was not found.
Also, tried to find "Xamarin/Android/Xamarin.Android.FSharp.targets" in the fallback search path(s) for $(MSBuildExtensionsPath) - "/Library/Frameworks/Mono.framework/External/xbuild/" .
These search paths are defined in "/Library/Frameworks/Mono.framework/Versions/5.18.0/lib/mono/msbuild/15.0/bin/MSBuild.dll.config".
Confirm that the path in the <Import> declaration is correct, and that the file exists on disk in one of the search paths.

[mallibone]

@TimLariviere yes I do believe that problem would go away because the Agents come with Android pre-installed. Here are the links to what the agents and the installed SDKs, tools, et al. (just in case you are wondering 🙂):

• Linux
• macOS
• Windows

Builds are mainly faster because most of the toolchain comes preinstalled. That being said I am still not any further with the macOS build 🙁 since I am still new to F#/FAKE.. any tip/link how I could print the error message you mentioned before?

[TimLariviere]

@mallibone The unit tests of Fabulous.Cli are failing to execute the F# interpreter checker.ParseAndCheckFileInProject.
This method returns 2 results according to the documentation.
We currently only check the second part CheckResults, which only says that it has been aborted.
So maybe its first part ParseResults contains more information on why it's failing?

This code already prints to the build log output.
We can see that with the aborted message appearing in your overview.
direct link

My guess is that if you change lines 134 to 148 of "Fabulous/src/Fabulous.Cli/fscd.fs" with

let parseResults, checkResults = checker.ParseAndCheckFileInProject(sourceFile, 0, File.ReadAllText(sourceFile), options) |> Async.RunSynchronously  
match parseResults.ParseHadErrors, checkResults with
| false, FSharpCheckFileAnswer.Aborted -> 
    printfn "Check aborted"
    Result.Error ()

| true, FSharpCheckFileAnswer.Aborted ->
    for error in parseResults.Errors do
        printfn "%s" (error.ToString())
    printfn "Parse had errors; Check aborted"
    Result.Error ()

| _, FSharpCheckFileAnswer.Succeeded res -> 
    let mutable hasErrors = false

    if parseResults.ParseHadErrors
        for error in parseResults.Errors do 
            printfn "Parse error: %s" (error.ToString())
            if error.Severity = FSharpErrorSeverity.Error then 
                hasErrors <- true

    for error in res.Errors do 
        printfn "Check error: %s" (error.ToString())
        if error.Severity = FSharpErrorSeverity.Error then 
            hasErrors <- true

    if hasErrors then 
        Result.Error ()
    else
        Result.Ok res.ImplementationFile

It should print all error messages to the build output.

[mallibone]

Okay so the error now is: Parse errors: startup (1,1)-(1,1) parameter error Error opening binary file '/usr/local/share/dotnet/sdk/NuGetFallbackFolder/netstandard.library/2.0.3/build/netstandard2.0/ref/netstandard.dll': Could not find a part of the path '/usr/local/share/dotnet/sdk/NuGetFallbackFolder/netstandard.library/2.0.3/build/netstandard2.0/ref/netstandard.dll'.
Taken from this buildlog

I found the path defined in Tests.fs on Line 163. Next step is to find out the location of this folder on the hosted agent...

[TimLariviere]

@mallibone I was suspecting this fallback folder might be a problem.
dotnet-cli 2.X behaves differently than 1.X.
It searches for packages in the following folders, before downloading them:

1. /usr/local/share/dotnet/sdk/NuGetFallbackFolder
2. ~/.nuget/packages

This makes it quite inconsistent. For instance, netstandard.library 2.0.3 is present in the fallback folder on my machine.
But it appears it's not on the build machine. So dotnet-cli will mostly likely downloads it in the ~/.nuget/packages folder...

I currently have dotnet-cli 2.1.500, the build machine appears to have 2.2.102.
Maybe that's the difference?

[mallibone]

@TimLariviere that is an interesting insight. I found the NuGetFallbackFolder in the home directory where .Net Core is installed on the hosted macOS agents. For the pull request, it seems to be running stable now for Windows (5-6 minutes build config) and macOS (~12 minutes test config). I will have to circle back for the Linux build there is an issue in the Generator project.

I would have created a new branch/pull request for the Linux (GTK) build, but am open to the idea add it to this issue/pr. If you have any questions regarding the setup, I am more than happy to help 🙂