diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index f79ffe598..dd234c79f 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -2,6 +2,11 @@ Contributions are highly welcome, however, except for very small changes, kindly file an issue and let's have a discussion before you open a pull request. +## Requirements + +.NET SDK 2.2 https://dotnet.microsoft.com/download/dotnet-core/2.2 +.NET SDK 3.1 https://dotnet.microsoft.com/download/dotnet-core/3.1 + ## Building the Project Clone this repo: diff --git a/Directory.Build.props b/Directory.Build.props index 99cdb5278..d40cabc6e 100644 --- a/Directory.Build.props +++ b/Directory.Build.props @@ -4,11 +4,12 @@ $(MSBuildThisFileDirectory) Debug $(MSBuildThisFileDirectory)bin\$(Configuration)\Packages\ - true true true snupkg + true + preview diff --git a/Documentation/Changelog.md b/Documentation/Changelog.md index 6ad878910..bb8777845 100644 --- a/Documentation/Changelog.md +++ b/Documentation/Changelog.md @@ -6,6 +6,20 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] +### Fixed + +-Fixed ExcludeFromCodeCoverage attribute bugs [#129](https://github.com/tonerdo/coverlet/issues/129) and [#670](https://github.com/tonerdo/coverlet/issues/670) with [#671](https://github.com/tonerdo/coverlet/pull/671) by https://github.com/matteoerigozzi + +### Improvements + +-Trim whitespace between values when reading from configuration from runsettings [#679](https://github.com/tonerdo/coverlet/pull/679) by https://github.com/EricStG + +## Release date 2020-01-03 +### Packages +coverlet.msbuild 2.8.0 +coverlet.console 1.7.0 +coverlet.collector 1.2.0 + ### Added -Add log to tracker [#553](https://github.com/tonerdo/coverlet/pull/553) -Exclude by assembly level System.Diagnostics.CodeAnalysis.ExcludeFromCodeCoverage [#589](https://github.com/tonerdo/coverlet/pull/589) @@ -24,7 +38,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Improvements --Improve exception message for unsupported runtime [#569](https://github.com/tonerdo/coverlet/pull/569) by https://github.com/daveMueller +-Improve exception message for unsupported runtime [#569](https://github.com/tonerdo/ +coverlet/pull/569) by https://github.com/daveMueller +-Improve cobertura absolute/relative path report generation [#661](https://github.com/tonerdo/coverlet/pull/661) by https://github.com/daveMueller ## Release date 2019-09-23 ### Packages diff --git a/Documentation/Examples/MSBuild/MergeWith/HowTo.md b/Documentation/Examples/MSBuild/MergeWith/HowTo.md index ef5b66205..d975d8f2b 100644 --- a/Documentation/Examples/MSBuild/MergeWith/HowTo.md +++ b/Documentation/Examples/MSBuild/MergeWith/HowTo.md @@ -7,4 +7,11 @@ Last command will join and create final needed format file. dotnet test XUnitTestProject1\XUnitTestProject1.csproj /p:CollectCoverage=true /p:CoverletOutput=../CoverageResults/ dotnet test XUnitTestProject2\XUnitTestProject2.csproj /p:CollectCoverage=true /p:CoverletOutput=../CoverageResults/ /p:MergeWith="../CoverageResults/coverage.json" dotnet test XUnitTestProject3\XUnitTestProject3.csproj /p:CollectCoverage=true /p:CoverletOutput=../CoverageResults/ /p:MergeWith="../CoverageResults/coverage.json" /p:CoverletOutputFormat="opencover" -``` \ No newline at end of file +``` + +You can merge also running `dotnet test` and merge with single command from a solution file, but you need to ensure that tests will run sequentially(`-m:1`). This slow down testing but avoid invalid coverage result. + +``` +dotnet test /p:CollectCoverage=true /p:CoverletOutput=../CoverageResults/ /p:MergeWith="../CoverageResults/coverage.json" /p:CoverletOutputFormat=\"opencover,json\" -m:1 +``` +N.B. You need to specify `json` format plus another format(the final one), because Coverlet can only merge proprietary format. At the end you can delete temporary `coverage.json` file. \ No newline at end of file diff --git a/Documentation/ReleasePlan.md b/Documentation/ReleasePlan.md index 1479dd693..fdfd7a513 100644 --- a/Documentation/ReleasePlan.md +++ b/Documentation/ReleasePlan.md @@ -28,9 +28,9 @@ We plan 1 release [once per quarter](https://en.wikipedia.org/wiki/Calendar_year | Package | **coverlet.msbuild** | | :-------------: |:-------------:| -|**coverlet.msbuild** | 2.7.0 | -|**coverlet.console** | 1.6.0 | -|**coverlet.collector** | 1.1.0 | +|**coverlet.msbuild** | 2.8.0 | +|**coverlet.console** | 1.7.0 | +|**coverlet.collector** | 1.2.0 | ### Proposed next versions @@ -41,11 +41,59 @@ We MANUALLY bump versions on production release, so we have different release pl | Release Date | **coverlet.msbuild** | **coverlet.console** | **coverlet.collector** | **commit hash**| **notes** | | :-------------: |:-------------:|:-------------:|:-------------:|:-------------:|:-------------:| +| <01 April 2020> | 2.8.1 | 1.7.1 | 1.2.1 | +| 03 January 2019 | 2.8.0 | 1.7.0 | 1.2.0 | 72a688f1c47fa92059540d5fbb1c4b0b4bf0dc8c | | | 23 September 2019 | 2.7.0 | 1.6.0 | 1.1.0 | 4ca01eb239038808739699470a61fad675af6c79 | | -| 1 July 2019 | 2.6.3 | 1.5.3 | 1.0.1 | e1593359497fdfe6befbb86304b8f4e09a656d14 | | -| 6 June 2019 | 2.6.2 | 1.5.2 | 1.0.0 | 3e7eac9df094c22335711a298d359890aed582e8 | first collector release | +| 01 July 2019 | 2.6.3 | 1.5.3 | 1.0.1 | e1593359497fdfe6befbb86304b8f4e09a656d14 | | +| 06 June 2019 | 2.6.2 | 1.5.2 | 1.0.0 | 3e7eac9df094c22335711a298d359890aed582e8 | first collector release | + +*< date > Expected next release date To get the list of commits between two version use git command ```bash git log --oneline hashbefore currenthash -``` \ No newline at end of file +``` + +# How to manually release packages to Nuget.org + +This is the steps to do to release new packages to Nuget.org + +1) Clone repo, **remember to build packages from master and not from your fork or metadata links will point to your forked repo.** +Run `git log -5` from repo root to verify last commit. + +2) Update project versions in file: + +Collector +https://github.com/tonerdo/coverlet/blob/master/src/coverlet.collector/version.json +.NET tool +https://github.com/tonerdo/coverlet/blob/master/src/coverlet.console/version.json +Msbuild tasks +https://github.com/tonerdo/coverlet/blob/master/src/coverlet.msbuild.tasks/version.json + +Core lib project file https://github.com/tonerdo/coverlet/blob/master/src/coverlet.core/coverlet.core.csproj. +The version of core lib project file is the version we'll report on github repo releases https://github.com/tonerdo/coverlet/releases + + +Sample of updated version PR https://github.com/tonerdo/coverlet/pull/675/files + +3) From new cloned, aligned and versions updated repo root run pack command +``` +dotnet pack -c release /p:PublicRelease=true +... + coverlet.console -> D:\git\coverlet\src\coverlet.console\bin\Release\netcoreapp2.2\coverlet.console.dll + coverlet.console -> D:\git\coverlet\src\coverlet.console\bin\Release\netcoreapp2.2\publish\ + Successfully created package 'D:\git\coverlet\bin\Release\Packages\coverlet.msbuild.2.8.1.nupkg'. + Successfully created package 'D:\git\coverlet\bin\Release\Packages\coverlet.msbuild.2.8.1.snupkg'. + Successfully created package 'D:\git\coverlet\bin\Release\Packages\coverlet.console.1.7.1.nupkg'. + Successfully created package 'D:\git\coverlet\bin\Release\Packages\coverlet.console.1.7.1.snupkg'. + Successfully created package 'D:\git\coverlet\bin\Release\Packages\coverlet.collector.1.2.1.nupkg'. + Successfully created package 'D:\git\coverlet\bin\Release\Packages\coverlet.collector.1.2.1.snupkg'. +``` + +4) Upload *.nupkg files to Nuget.org site. **Check all metadata(url links etc...) before "Submit"** + +5) **On your fork**: +* Align to master +* Update versions in files accordingly to new release and commit/merge to master +* Create release on repo https://github.com/tonerdo/coverlet/releases using https://github.com/tonerdo/coverlet/blob/master/src/coverlet.core/coverlet.core.csproj assembly version +* Update the [Release Plan](https://github.com/tonerdo/coverlet/blob/master/Documentation/ReleasePlan.md)(this document) and [ChangeLog](https://github.com/tonerdo/coverlet/blob/master/Documentation/Changelog.md) \ No newline at end of file diff --git a/Documentation/Troubleshooting.md b/Documentation/Troubleshooting.md index 0a5e47415..e78afbc9a 100644 --- a/Documentation/Troubleshooting.md +++ b/Documentation/Troubleshooting.md @@ -235,4 +235,12 @@ You'll get this message during test run dotnet test -p:Include="[test_coverage.]" -p:Exclude="[*.Test.*]*" -p:CollectCoverage=true -p:CoverletOutputFormat=cobertura -p:CoverletOutput=coverage.cobertura.xml Coverlet msbuild instrumentation task debugging is enabled. Please attach debugger to process to continue Process Id: 29228 Name: dotnet -``` \ No newline at end of file +``` + +## Enable collector instrumentation debugging + +You can live attach and debug collectors with `COVERLET_DATACOLLECTOR_OUTOFPROC_DEBUG` env variable +``` + set COVERLET_DATACOLLECTOR_OUTOFPROC_DEBUG=1 +``` +You will be asket to attach a debugger through UI popup. \ No newline at end of file diff --git a/Documentation/VSTestIntegration.md b/Documentation/VSTestIntegration.md index 1ade115c8..97d9c09c6 100644 --- a/Documentation/VSTestIntegration.md +++ b/Documentation/VSTestIntegration.md @@ -30,13 +30,13 @@ These are a list of options that are supported by coverlet. These can be specifi | Option | Summary | |------------- |------------------------------------------------------------------------------------------| |Format | Coverage output format. These are either cobertura, json, lcov, opencover or teamcity as well as combinations of these formats. | -|MergeWith | Combine the output of multiple coverage runs into a single result([check the sample](Examples.md)). | |Exclude | Exclude from code coverage analysing using filter expressions. | |ExcludeByFile | Ignore specific source files from code coverage. | |Include | Explicitly set what to include in code coverage analysis using filter expressions. | |IncludeDirectory| Explicitly set which directories to include in code coverage analysis. | |SingleHit | Specifies whether to limit code coverage hit reporting to a single hit for each location.| |UseSourceLink | Specifies whether to use SourceLink URIs in place of file system paths. | +|IncludeTestAssembly | Include coverage of the test assembly. | How to specify these options via runsettings? ``` @@ -46,8 +46,7 @@ How to specify these options via runsettings? - json,cobertura - /custom/path/result.json + json,cobertura [coverlet.*.tests?]*,[*]Coverlet.Core* [coverlet.*]*,[*]Coverlet.Core* Obsolete,GeneratedCodeAttribute,CompilerGeneratedAttribute @@ -55,6 +54,7 @@ How to specify these options via runsettings? ../dir1/,../dir2/, false true + true @@ -67,6 +67,8 @@ This runsettings file can easily be provided using command line option as given 2. `dotnet vstest --settings coverletArgs.runsettings` +Take a look at our [`HelloWorld`](Examples/VSTest/HelloWorld/HowTo.md) sample. + ## Implementation Details The proposed solution is implemented with the help of [datacollectors](https://github.com/Microsoft/vstest-docs/blob/master/docs/extensions/datacollector.md). diff --git a/README.md b/README.md index ac05e94a0..d2698ac08 100644 --- a/README.md +++ b/README.md @@ -24,7 +24,7 @@ Coverlet can be used through three different *drivers* ### VSTest Integration (preferred due to [know issue](https://github.com/tonerdo/coverlet/blob/master/Documentation/KnowIssues.md#1-vstest-stops-process-execution-earlydotnet-test)) -### Insallation +### Installation ```bash dotnet add package coverlet.collector ``` @@ -51,7 +51,7 @@ See [documentation](Documentation/VSTestIntegration.md) for advanced usage. * Important [know issue](Documentation/KnowIssues.md#2-upgrade-coverletcollector-to-version--100) ### MSBuild Integration -### Insallation +### Installation ```bash dotnet add package coverlet.msbuild ``` diff --git a/_assets/coverlet-icon.png b/_assets/coverlet-icon.png new file mode 100644 index 000000000..fb062f307 Binary files /dev/null and b/_assets/coverlet-icon.png differ diff --git a/eng/build.yml b/eng/build.yml index dfc587262..e12b5069a 100644 --- a/eng/build.yml +++ b/eng/build.yml @@ -1,7 +1,12 @@ steps: - task: UseDotNet@2 inputs: - version: 2.2.402 + version: 2.2.207 + displayName: Install .NET Core SDK + +- task: UseDotNet@2 + inputs: + version: 3.1.100 displayName: Install .NET Core SDK - script: dotnet restore @@ -10,7 +15,7 @@ steps: - script: dotnet build -c $(BuildConfiguration) --no-restore displayName: Build -- script: dotnet pack -c $(BuildConfiguration) --no-build +- script: dotnet pack -c $(BuildConfiguration) displayName: Pack - task: DotNetCoreCLI@2 @@ -18,4 +23,4 @@ steps: inputs: command: test arguments: -c $(BuildConfiguration) --no-build /p:CollectCoverage=true /p:CoverletOutputFormat=opencover /p:Include=[coverlet.*]* /p:Exclude=[coverlet.tests.remoteexecutor]* - testRunTitle: $(Agent.JobName) + testRunTitle: $(Agent.JobName) \ No newline at end of file diff --git a/global.json b/global.json index 323ede7fb..e9aac8c22 100644 --- a/global.json +++ b/global.json @@ -1,5 +1,5 @@ { - "sdk": { - "version": "2.2.402" - } -} \ No newline at end of file + "sdk": { + "version": "3.1.100" + } +} diff --git a/src/coverlet.collector/DataCollection/CoverletCoverageCollector.cs b/src/coverlet.collector/DataCollection/CoverletCoverageCollector.cs index 9004fc292..d77fcfcf2 100644 --- a/src/coverlet.collector/DataCollection/CoverletCoverageCollector.cs +++ b/src/coverlet.collector/DataCollection/CoverletCoverageCollector.cs @@ -37,6 +37,15 @@ internal CoverletCoverageCollector(TestPlatformEqtTrace eqtTrace, ICoverageWrapp _countDownEventFactory = countDownEventFactory; } + private void AttachDebugger() + { + if (int.TryParse(Environment.GetEnvironmentVariable("COVERLET_DATACOLLECTOR_OUTOFPROC_DEBUG"), out int result) && result == 1) + { + Debugger.Launch(); + Debugger.Break(); + } + } + /// /// Initializes data collector /// @@ -52,6 +61,9 @@ public override void Initialize( DataCollectionLogger logger, DataCollectionEnvironmentContext environmentContext) { + + AttachDebugger(); + if (_eqtTrace.IsInfoEnabled) { _eqtTrace.Info("Initializing {0} with configuration: '{1}'", CoverletConstants.DataCollectorName, configurationElement?.OuterXml); diff --git a/src/coverlet.collector/DataCollection/CoverletSettingsParser.cs b/src/coverlet.collector/DataCollection/CoverletSettingsParser.cs index 705a334f9..cbaaca3d1 100644 --- a/src/coverlet.collector/DataCollection/CoverletSettingsParser.cs +++ b/src/coverlet.collector/DataCollection/CoverletSettingsParser.cs @@ -86,8 +86,7 @@ private string[] ParseReportFormats(XmlElement configurationElement) if (configurationElement != null) { XmlElement reportFormatElement = configurationElement[CoverletConstants.ReportFormatElementName]; - formats = reportFormatElement?.InnerText?.Split(',').Select(format => format.Trim()) - .Where(format => !string.IsNullOrEmpty(format)).ToArray(); + formats = this.SplitElement(reportFormatElement); } return formats is null || formats.Length == 0 ? new[] { CoverletConstants.DefaultReportFormat } : formats; @@ -101,7 +100,7 @@ private string[] ParseReportFormats(XmlElement configurationElement) private string[] ParseIncludeFilters(XmlElement configurationElement) { XmlElement includeFiltersElement = configurationElement[CoverletConstants.IncludeFiltersElementName]; - return includeFiltersElement?.InnerText?.Split(','); + return this.SplitElement(includeFiltersElement); } /// @@ -112,7 +111,7 @@ private string[] ParseIncludeFilters(XmlElement configurationElement) private string[] ParseIncludeDirectories(XmlElement configurationElement) { XmlElement includeDirectoriesElement = configurationElement[CoverletConstants.IncludeDirectoriesElementName]; - return includeDirectoriesElement?.InnerText?.Split(','); + return this.SplitElement(includeDirectoriesElement); } /// @@ -127,7 +126,7 @@ private string[] ParseExcludeFilters(XmlElement configurationElement) if (configurationElement != null) { XmlElement excludeFiltersElement = configurationElement[CoverletConstants.ExcludeFiltersElementName]; - string[] filters = excludeFiltersElement?.InnerText?.Split(','); + string[] filters = this.SplitElement(excludeFiltersElement); if (filters != null) { excludeFilters.AddRange(filters); @@ -145,7 +144,7 @@ private string[] ParseExcludeFilters(XmlElement configurationElement) private string[] ParseExcludeSourceFiles(XmlElement configurationElement) { XmlElement excludeSourceFilesElement = configurationElement[CoverletConstants.ExcludeSourceFilesElementName]; - return excludeSourceFilesElement?.InnerText?.Split(','); + return this.SplitElement(excludeSourceFilesElement); } /// @@ -156,7 +155,7 @@ private string[] ParseExcludeSourceFiles(XmlElement configurationElement) private string[] ParseExcludeAttributes(XmlElement configurationElement) { XmlElement excludeAttributesElement = configurationElement[CoverletConstants.ExcludeAttributesElementName]; - return excludeAttributesElement?.InnerText?.Split(','); + return this.SplitElement(excludeAttributesElement); } /// @@ -205,5 +204,15 @@ private bool ParseIncludeTestAssembly(XmlElement configurationElement) bool.TryParse(includeTestAssemblyElement?.InnerText, out bool includeTestAssembly); return includeTestAssembly; } + + /// + /// Splits a comma separated elements into an array + /// + /// The element to split + /// An array of the values in the element + private string[] SplitElement(XmlElement element) + { + return element?.InnerText?.Split(',', StringSplitOptions.RemoveEmptyEntries).Where(value => !string.IsNullOrWhiteSpace(value)).Select(value => value.Trim()).ToArray(); + } } } diff --git a/src/coverlet.collector/coverlet.collector.csproj b/src/coverlet.collector/coverlet.collector.csproj index d1fc144ec..b23f2a210 100644 --- a/src/coverlet.collector/coverlet.collector.csproj +++ b/src/coverlet.collector/coverlet.collector.csproj @@ -1,22 +1,35 @@ - + netcoreapp2.0 coverlet.collector - coverlet.collector + true + true + false + + true + $(TargetsForTfmSpecificContentInPackage);PackBuildOutputs + + $(NoWarn);NU5127 + + + + coverlet.collector + coverlet.collector tonerdo MIT http://github.com/tonerdo/coverlet https://raw.githubusercontent.com/tonerdo/coverlet/master/_assets/coverlet-icon.svg?sanitize=true + coverlet-icon.png false - true Coverlet is a cross platform code coverage library for .NET, with support for line, branch and method coverage. coverage testing unit-test lcov opencover quality git - true - false - true - $(TargetsForTfmSpecificContentInPackage);PackBuildOutputs @@ -25,6 +38,7 @@ + diff --git a/src/coverlet.collector/version.json b/src/coverlet.collector/version.json index e315416f7..95095d4ef 100644 --- a/src/coverlet.collector/version.json +++ b/src/coverlet.collector/version.json @@ -1,6 +1,6 @@ { "$schema": "https://raw.githubusercontent.com/AArnott/Nerdbank.GitVersioning/master/src/NerdBank.GitVersioning/version.schema.json", - "version": "1.1", + "version": "1.2", "publicReleaseRefSpec": [ "^refs/heads/master$" ] diff --git a/src/coverlet.console/coverlet.console.csproj b/src/coverlet.console/coverlet.console.csproj index 8d3855d6e..07b0d6374 100644 --- a/src/coverlet.console/coverlet.console.csproj +++ b/src/coverlet.console/coverlet.console.csproj @@ -1,4 +1,4 @@ - + Exe @@ -6,11 +6,16 @@ coverlet true coverlet.console - tonerdo + + + + $(AssemblyTitle) + tonerdo Coverlet is a cross platform code coverage tool for .NET, with support for line, branch and method coverage. coverage;testing;unit-test;lcov;opencover;quality https://raw.githubusercontent.com/tonerdo/coverlet/master/_assets/coverlet-icon.svg?sanitize=true + coverlet-icon.png https://github.com/tonerdo/coverlet MIT git @@ -24,4 +29,8 @@ + + + + diff --git a/src/coverlet.console/version.json b/src/coverlet.console/version.json index 57c456fb2..119599712 100644 --- a/src/coverlet.console/version.json +++ b/src/coverlet.console/version.json @@ -1,6 +1,6 @@ { "$schema": "https://raw.githubusercontent.com/AArnott/Nerdbank.GitVersioning/master/src/NerdBank.GitVersioning/version.schema.json", - "version": "1.6", + "version": "1.7", "publicReleaseRefSpec": [ "^refs/heads/master$" ] diff --git a/src/coverlet.core/Instrumentation/Instrumenter.cs b/src/coverlet.core/Instrumentation/Instrumenter.cs index 824d83588..fbedf313d 100644 --- a/src/coverlet.core/Instrumentation/Instrumenter.cs +++ b/src/coverlet.core/Instrumentation/Instrumenter.cs @@ -39,6 +39,8 @@ internal class Instrumenter private MethodReference _customTrackerRecordHitMethod; private List _excludedSourceFiles; private List _branchesInCompiledGeneratedClass; + private List<(MethodDefinition, int)> _excludedMethods; + private List _excludedCompilerGeneratedTypes; public bool SkipModule { get; set; } = false; @@ -180,8 +182,19 @@ private void InstrumentModule() // Instrumenting Interlocked which is used for recording hits would cause an infinite loop. && (!_isCoreLibrary || actualType.FullName != "System.Threading.Interlocked") && !_instrumentationHelper.IsTypeExcluded(_module, actualType.FullName, _excludeFilters) - && _instrumentationHelper.IsTypeIncluded(_module, actualType.FullName, _includeFilters)) - InstrumentType(type); + && _instrumentationHelper.IsTypeIncluded(_module, actualType.FullName, _includeFilters) + ) + { + if (IsSynthesizedMemberToBeExcluded(type)) + { + _excludedCompilerGeneratedTypes ??= new List(); + _excludedCompilerGeneratedTypes.Add(type.FullName); + } + else + { + InstrumentType(type); + } + } } // Fixup the custom tracker class constructor, according to all instrumented types @@ -335,6 +348,10 @@ private void AddCustomModuleTrackerToModule(ModuleDefinition module) private void InstrumentType(TypeDefinition type) { var methods = type.GetMethods(); + + // We keep ordinal index because it's the way used by compiler for generated types/methods to + // avoid ambiguity + int ordinal = -1; foreach (var method in methods) { MethodDefinition actualMethod = method; @@ -349,8 +366,21 @@ private void InstrumentType(TypeDefinition type) customAttributes = customAttributes.Union(prop.CustomAttributes); } + ordinal++; + if (IsSynthesizedMemberToBeExcluded(method)) + { + continue; + } + if (!customAttributes.Any(IsExcludeAttribute)) + { InstrumentMethod(method); + } + else + { + _excludedMethods ??= new List<(MethodDefinition, int)>(); + _excludedMethods.Add((method, ordinal)); + } } var ctors = type.GetConstructors(); @@ -604,6 +634,61 @@ private static MethodBody GetMethodBody(MethodDefinition method) } } + // Check if the member (type or method) is generated by the compiler from a method excluded from code coverage + private bool IsSynthesizedMemberToBeExcluded(IMemberDefinition definition) + { + if (_excludedMethods is null) + { + return false; + } + + TypeDefinition declaringType = definition.DeclaringType; + + // We check all parent type of current one bottom-up + while (declaringType != null) + { + + // If parent type is excluded return + if (_excludedCompilerGeneratedTypes != null && + _excludedCompilerGeneratedTypes.Any(t => t == declaringType.FullName)) + { + return true; + } + + // Check methods members and compiler generated types + foreach (var excludedMethods in _excludedMethods) + { + // Exclude this member if declaring type is the same of the excluded method and + // the name is synthesized from the name of the excluded method. + // + if (declaringType.FullName == excludedMethods.Item1.DeclaringType.FullName && + IsSynthesizedNameOf(definition.Name, excludedMethods.Item1.Name, excludedMethods.Item2)) + { + return true; + } + } + declaringType = declaringType.DeclaringType; + } + + return false; + } + + // Check if the name is synthesized by the compiler + // Refer to https://github.com/dotnet/roslyn/blob/master/src/Compilers/CSharp/Portable/Symbols/Synthesized/GeneratedNames.cs + // to see how the compiler generate names for lambda, local function, yield or async/await expressions + internal bool IsSynthesizedNameOf(string name, string methodName, int methodOrdinal) + { + return + // Lambda method + name.IndexOf($"<{methodName}>b__{methodOrdinal}") != -1 || + // Lambda display class + name.IndexOf($"<>c__DisplayClass{methodOrdinal}_") != -1 || + // State machine + name.IndexOf($"<{methodName}>d__{methodOrdinal}") != -1 || + // Local function + (name.IndexOf($"<{methodName}>g__") != -1 && name.IndexOf($"|{methodOrdinal}_") != -1); + } + /// /// A custom importer created specifically to allow the instrumentation of System.Private.CoreLib by /// removing the external references to netstandard that are generated when instrumenting a typical diff --git a/src/coverlet.core/Reporters/CoberturaReporter.cs b/src/coverlet.core/Reporters/CoberturaReporter.cs index d3c11f956..cebaa569a 100644 --- a/src/coverlet.core/Reporters/CoberturaReporter.cs +++ b/src/coverlet.core/Reporters/CoberturaReporter.cs @@ -32,8 +32,8 @@ public string Report(CoverageResult result) coverage.Add(new XAttribute("timestamp", (int)(DateTime.UtcNow - new DateTime(1970, 1, 1)).TotalSeconds)); XElement sources = new XElement("sources"); - var rootDirs = GetRootDirs(result.Modules, result.UseSourceLink).ToList(); - rootDirs.ForEach(x => sources.Add(new XElement("source", x))); + var absolutePaths = GetBasePaths(result.Modules, result.UseSourceLink).ToList(); + absolutePaths.ForEach(x => sources.Add(new XElement("source", x))); XElement packages = new XElement("packages"); foreach (var module in result.Modules) @@ -51,7 +51,7 @@ public string Report(CoverageResult result) { XElement @class = new XElement("class"); @class.Add(new XAttribute("name", cls.Key)); - @class.Add(new XAttribute("filename", GetRelativePathFromBase(rootDirs, document.Key, result.UseSourceLink))); + @class.Add(new XAttribute("filename", GetRelativePathFromBase(absolutePaths, document.Key, result.UseSourceLink))); @class.Add(new XAttribute("line-rate", (summary.CalculateLineCoverage(cls.Value).Percent / 100).ToString(CultureInfo.InvariantCulture))); @class.Add(new XAttribute("branch-rate", (summary.CalculateBranchCoverage(cls.Value).Percent / 100).ToString(CultureInfo.InvariantCulture))); @class.Add(new XAttribute("complexity", summary.CalculateCyclomaticComplexity(cls.Value))); @@ -133,28 +133,70 @@ public string Report(CoverageResult result) return Encoding.UTF8.GetString(stream.ToArray()); } - private static IEnumerable GetRootDirs(Modules modules, bool useSourceLink) + private static IEnumerable GetBasePaths(Modules modules, bool useSourceLink) { + /* + Workflow + + Path1 c:\dir1\dir2\file1.cs + Path2 c:\dir1\file2.cs + Path3 e:\dir1\file2.cs + + 1) Search for root dir + c:\ -> c:\dir1\dir2\file1.cs + c:\dir1\file2.cs + e:\ -> e:\dir1\file2.cs + + 2) Split path on directory separator i.e. for record c:\ ordered ascending by fragment elements + Path1 = [c:|dir1|file2.cs] + Path2 = [c:|dir1|dir2|file1.cs] + + 3) Find longest shared path comparing indexes + Path1[0] = Path2[0], ..., PathY[0] -> add to final fragment list + Path1[n] = Path2[n], ..., PathY[n] -> add to final fragment list + Path1[n+1] != Path2[n+1], ..., PathY[n+1] -> break, Path1[n] was last shared fragment + + 4) Concat created fragment list + */ if (useSourceLink) { return new[] { string.Empty }; } - return modules.Values.SelectMany(k => k.Keys).Select(Directory.GetDirectoryRoot).Distinct(); + return modules.Values.SelectMany(k => k.Keys).GroupBy(Directory.GetDirectoryRoot).Select(group => + { + var splittedPaths = group.Select(absolutePath => absolutePath.Split(Path.DirectorySeparatorChar)) + .OrderBy(absolutePath => absolutePath.Length).ToList(); + if (splittedPaths.Count == 1) + { + return group.Key; + } + + var basePathFragments = new List(); + + splittedPaths[0].Select((value, index) => (value, index)).ToList().ForEach(fragmentIndexPair => + { + if (splittedPaths.All(sp => fragmentIndexPair.value.Equals(sp[fragmentIndexPair.index]))) + { + basePathFragments.Add(fragmentIndexPair.value); + } + }); + return string.Concat(string.Join(Path.DirectorySeparatorChar.ToString(), basePathFragments), Path.DirectorySeparatorChar); + }); } - private static string GetRelativePathFromBase(IEnumerable rootPaths, string path, bool useSourceLink) + private static string GetRelativePathFromBase(IEnumerable basePaths, string path, bool useSourceLink) { if (useSourceLink) { return path; } - foreach (var root in rootPaths) + foreach (var basePath in basePaths) { - if (path.StartsWith(root)) + if (path.StartsWith(basePath)) { - return path.Substring(root.Length); + return path.Substring(basePath.Length); } } diff --git a/src/coverlet.core/coverlet.core.csproj b/src/coverlet.core/coverlet.core.csproj index 4add60886..357d42e2f 100644 --- a/src/coverlet.core/coverlet.core.csproj +++ b/src/coverlet.core/coverlet.core.csproj @@ -3,9 +3,8 @@ Library netstandard2.0 - 5.2.0 + 5.3.0 false - preview diff --git a/src/coverlet.msbuild.tasks/ReportWriter.cs b/src/coverlet.msbuild.tasks/ReportWriter.cs index 346822a56..e55963247 100644 --- a/src/coverlet.msbuild.tasks/ReportWriter.cs +++ b/src/coverlet.msbuild.tasks/ReportWriter.cs @@ -35,8 +35,8 @@ public void WriteReport() else if (Path.HasExtension(filename)) { // filename with extension for instance c:\reportpath\file.ext - // c:\reportpath\file.ext.reportedextension - filename = $"{Path.GetFileNameWithoutExtension(filename)}{separatorPoint}{_coverletMultiTargetFrameworksCurrentTFM}{Path.GetExtension(filename)}.{_reporter.Extension}"; + // we keep user specified name + filename = $"{Path.GetFileNameWithoutExtension(filename)}{separatorPoint}{_coverletMultiTargetFrameworksCurrentTFM}{Path.GetExtension(filename)}"; } else { diff --git a/src/coverlet.msbuild.tasks/coverlet.msbuild.tasks.csproj b/src/coverlet.msbuild.tasks/coverlet.msbuild.tasks.csproj index 3c3bf2ec6..bcd6a0498 100644 --- a/src/coverlet.msbuild.tasks/coverlet.msbuild.tasks.csproj +++ b/src/coverlet.msbuild.tasks/coverlet.msbuild.tasks.csproj @@ -4,21 +4,33 @@ Library netstandard2.0 coverlet.msbuild.tasks + true + $(TargetsForTfmSpecificContentInPackage);PackBuildOutputs + build + false + + true + + + + coverlet.msbuild coverlet.msbuild tonerdo MIT http://github.com/tonerdo/coverlet https://raw.githubusercontent.com/tonerdo/coverlet/master/_assets/coverlet-icon.svg?sanitize=true + coverlet-icon.png false true Coverlet is a cross platform code coverage library for .NET, with support for line, branch and method coverage. coverage testing unit-test lcov opencover quality git - true - $(TargetsForTfmSpecificContentInPackage);PackBuildOutputs - build - false @@ -30,7 +42,7 @@ - + @@ -38,6 +50,10 @@ + + + + diff --git a/src/coverlet.msbuild.tasks/version.json b/src/coverlet.msbuild.tasks/version.json index 767d4f1e2..8b3745feb 100644 --- a/src/coverlet.msbuild.tasks/version.json +++ b/src/coverlet.msbuild.tasks/version.json @@ -1,6 +1,6 @@ { "$schema": "https://raw.githubusercontent.com/AArnott/Nerdbank.GitVersioning/master/src/NerdBank.GitVersioning/version.schema.json", - "version": "2.7", + "version": "2.8", "publicReleaseRefSpec": [ "^refs/heads/master$" ] diff --git a/test/coverlet.collector.tests/CoverletSettingsParserTests.cs b/test/coverlet.collector.tests/CoverletSettingsParserTests.cs index 757e00bf1..40d0ec112 100644 --- a/test/coverlet.collector.tests/CoverletSettingsParserTests.cs +++ b/test/coverlet.collector.tests/CoverletSettingsParserTests.cs @@ -42,17 +42,33 @@ public void ParseShouldSelectFirstTestModuleFromTestModulesList() Assert.Equal("module1.dll", coverletSettings.TestModule); } - [Fact] - public void ParseShouldCorrectlyParseConfigurationElement() + [Theory] + [InlineData("[*]*,[coverlet]*", "[coverlet.*.tests?]*,[coverlet.*.tests.*]*", @"E:\temp,/var/tmp", "module1.cs,module2.cs", "Obsolete,GeneratedCodeAttribute,CompilerGeneratedAttribute")] + [InlineData("[*]*,,[coverlet]*", "[coverlet.*.tests?]*,,[coverlet.*.tests.*]*", @"E:\temp,,/var/tmp", "module1.cs,,module2.cs", "Obsolete,,GeneratedCodeAttribute,,CompilerGeneratedAttribute")] + [InlineData("[*]*, ,[coverlet]*", "[coverlet.*.tests?]*, ,[coverlet.*.tests.*]*", @"E:\temp, ,/var/tmp", "module1.cs, ,module2.cs", "Obsolete, ,GeneratedCodeAttribute, ,CompilerGeneratedAttribute")] + [InlineData("[*]*,\t,[coverlet]*", "[coverlet.*.tests?]*,\t,[coverlet.*.tests.*]*", "E:\\temp,\t,/var/tmp", "module1.cs,\t,module2.cs", "Obsolete,\t,GeneratedCodeAttribute,\t,CompilerGeneratedAttribute")] + [InlineData("[*]*, [coverlet]*", "[coverlet.*.tests?]*, [coverlet.*.tests.*]*", @"E:\temp, /var/tmp", "module1.cs, module2.cs", "Obsolete, GeneratedCodeAttribute, CompilerGeneratedAttribute")] + [InlineData("[*]*,\t[coverlet]*", "[coverlet.*.tests?]*,\t[coverlet.*.tests.*]*", "E:\\temp,\t/var/tmp", "module1.cs,\tmodule2.cs", "Obsolete,\tGeneratedCodeAttribute,\tCompilerGeneratedAttribute")] + [InlineData("[*]*, \t[coverlet]*", "[coverlet.*.tests?]*, \t[coverlet.*.tests.*]*", "E:\\temp, \t/var/tmp", "module1.cs, \tmodule2.cs", "Obsolete, \tGeneratedCodeAttribute, \tCompilerGeneratedAttribute")] + [InlineData("[*]*,\r\n[coverlet]*", "[coverlet.*.tests?]*,\r\n[coverlet.*.tests.*]*", "E:\\temp,\r\n/var/tmp", "module1.cs,\r\nmodule2.cs", "Obsolete,\r\nGeneratedCodeAttribute,\r\nCompilerGeneratedAttribute")] + [InlineData("[*]*, \r\n [coverlet]*", "[coverlet.*.tests?]*, \r\n [coverlet.*.tests.*]*", "E:\\temp, \r\n /var/tmp", "module1.cs, \r\n module2.cs", "Obsolete, \r\n GeneratedCodeAttribute, \r\n CompilerGeneratedAttribute")] + [InlineData("[*]*,\t\r\n\t[coverlet]*", "[coverlet.*.tests?]*,\t\r\n\t[coverlet.*.tests.*]*", "E:\\temp,\t\r\n\t/var/tmp", "module1.cs,\t\r\n\tmodule2.cs", "Obsolete,\t\r\n\tGeneratedCodeAttribute,\t\r\n\tCompilerGeneratedAttribute")] + [InlineData("[*]*, \t \r\n \t [coverlet]*", "[coverlet.*.tests?]*, \t \r\n \t [coverlet.*.tests.*]*", "E:\\temp, \t \r\n \t /var/tmp", "module1.cs, \t \r\n \t module2.cs", "Obsolete, \t \r\n \t GeneratedCodeAttribute, \t \r\n \t CompilerGeneratedAttribute")] + [InlineData(" [*]* , [coverlet]* ", " [coverlet.*.tests?]* , [coverlet.*.tests.*]* ", " E:\\temp , /var/tmp ", " module1.cs , module2.cs ", " Obsolete , GeneratedCodeAttribute , CompilerGeneratedAttribute ")] + public void ParseShouldCorrectlyParseConfigurationElement(string includeFilters, + string excludeFilters, + string includeDirectories, + string excludeSourceFiles, + string excludeAttributes) { var testModules = new List { "abc.dll" }; var doc = new XmlDocument(); var configElement = doc.CreateElement("Configuration"); - this.CreateCoverletNodes(doc, configElement, CoverletConstants.IncludeFiltersElementName, "[*]*"); - this.CreateCoverletNodes(doc, configElement, CoverletConstants.ExcludeFiltersElementName, "[coverlet.*.tests?]*"); - this.CreateCoverletNodes(doc, configElement, CoverletConstants.IncludeDirectoriesElementName, @"E:\temp"); - this.CreateCoverletNodes(doc, configElement, CoverletConstants.ExcludeSourceFilesElementName, "module1.cs,module2.cs"); - this.CreateCoverletNodes(doc, configElement, CoverletConstants.ExcludeAttributesElementName, "Obsolete,GeneratedCodeAttribute,CompilerGeneratedAttribute"); + this.CreateCoverletNodes(doc, configElement, CoverletConstants.IncludeFiltersElementName, includeFilters); + this.CreateCoverletNodes(doc, configElement, CoverletConstants.ExcludeFiltersElementName, excludeFilters); + this.CreateCoverletNodes(doc, configElement, CoverletConstants.IncludeDirectoriesElementName, includeDirectories); + this.CreateCoverletNodes(doc, configElement, CoverletConstants.ExcludeSourceFilesElementName, excludeSourceFiles); + this.CreateCoverletNodes(doc, configElement, CoverletConstants.ExcludeAttributesElementName, excludeAttributes); this.CreateCoverletNodes(doc, configElement, CoverletConstants.MergeWithElementName, "/path/to/result.json"); this.CreateCoverletNodes(doc, configElement, CoverletConstants.UseSourceLinkElementName, "false"); this.CreateCoverletNodes(doc, configElement, CoverletConstants.SingleHitElementName, "true"); @@ -62,24 +78,76 @@ public void ParseShouldCorrectlyParseConfigurationElement() Assert.Equal("abc.dll", coverletSettings.TestModule); Assert.Equal("[*]*", coverletSettings.IncludeFilters[0]); + Assert.Equal("[coverlet]*", coverletSettings.IncludeFilters[1]); Assert.Equal(@"E:\temp", coverletSettings.IncludeDirectories[0]); + Assert.Equal("/var/tmp", coverletSettings.IncludeDirectories[1]); Assert.Equal("module1.cs", coverletSettings.ExcludeSourceFiles[0]); Assert.Equal("module2.cs", coverletSettings.ExcludeSourceFiles[1]); Assert.Equal("Obsolete", coverletSettings.ExcludeAttributes[0]); Assert.Equal("GeneratedCodeAttribute", coverletSettings.ExcludeAttributes[1]); + Assert.Equal("CompilerGeneratedAttribute", coverletSettings.ExcludeAttributes[2]); Assert.Equal("/path/to/result.json", coverletSettings.MergeWith); Assert.Equal("[coverlet.*]*", coverletSettings.ExcludeFilters[0]); + Assert.Equal("[coverlet.*.tests?]*", coverletSettings.ExcludeFilters[1]); + Assert.Equal("[coverlet.*.tests.*]*", coverletSettings.ExcludeFilters[2]); + Assert.False(coverletSettings.UseSourceLink); Assert.True(coverletSettings.SingleHit); Assert.True(coverletSettings.IncludeTestAssembly); } + [Fact] + public void ParseShouldCorrectlyParseConfigurationElementWithNullInnerText() + { + var testModules = new List { "abc.dll" }; + var doc = new XmlDocument(); + var configElement = doc.CreateElement("Configuration"); + this.CreateCoverleteNullInnerTextNodes(doc, configElement, CoverletConstants.IncludeFiltersElementName); + this.CreateCoverleteNullInnerTextNodes(doc, configElement, CoverletConstants.ExcludeFiltersElementName); + this.CreateCoverleteNullInnerTextNodes(doc, configElement, CoverletConstants.IncludeDirectoriesElementName); + this.CreateCoverleteNullInnerTextNodes(doc, configElement, CoverletConstants.ExcludeSourceFilesElementName); + this.CreateCoverleteNullInnerTextNodes(doc, configElement, CoverletConstants.ExcludeAttributesElementName); + + CoverletSettings coverletSettings = _coverletSettingsParser.Parse(configElement, testModules); + + Assert.Equal("abc.dll", coverletSettings.TestModule); + Assert.Empty(coverletSettings.IncludeFilters); + Assert.Empty(coverletSettings.IncludeDirectories); + Assert.Empty(coverletSettings.ExcludeSourceFiles); + Assert.Empty(coverletSettings.ExcludeAttributes); + Assert.Single(coverletSettings.ExcludeFilters, "[coverlet.*]*"); + } + + [Fact] + public void ParseShouldCorrectlyParseConfigurationElementWithNullElements() + { + var testModules = new List { "abc.dll" }; + var doc = new XmlDocument(); + var configElement = doc.CreateElement("Configuration"); + + CoverletSettings coverletSettings = _coverletSettingsParser.Parse(configElement, testModules); + + Assert.Equal("abc.dll", coverletSettings.TestModule); + Assert.Null(coverletSettings.IncludeFilters); + Assert.Null(coverletSettings.IncludeDirectories); + Assert.Null(coverletSettings.ExcludeSourceFiles); + Assert.Null(coverletSettings.ExcludeAttributes); + Assert.Single(coverletSettings.ExcludeFilters, "[coverlet.*]*"); + } + [Theory] [InlineData(" , json", 1, new[] { "json" })] [InlineData(" , json, ", 1, new[] { "json" })] [InlineData("json,cobertura", 2, new[] { "json", "cobertura" })] + [InlineData("json,\r\ncobertura", 2, new[] { "json", "cobertura" })] + [InlineData("json, \r\n cobertura", 2, new[] { "json", "cobertura" })] + [InlineData("json,\tcobertura", 2, new[] { "json", "cobertura" })] + [InlineData("json, \t cobertura", 2, new[] { "json", "cobertura" })] + [InlineData("json,\t\r\n\tcobertura", 2, new[] { "json", "cobertura" })] + [InlineData("json, \t \r\n \tcobertura", 2, new[] { "json", "cobertura" })] [InlineData(" , json,, cobertura ", 2, new[] { "json", "cobertura" })] [InlineData(" , json, , cobertura ", 2, new[] { "json", "cobertura" })] + [InlineData(",json,\t,\r\n,cobertura", 2, new[] { "json", "cobertura" })] public void ParseShouldCorrectlyParseMultipleFormats(string formats, int formatsCount, string[] expectedReportFormats) { var testModules = new List { "abc.dll" }; @@ -115,5 +183,12 @@ private void CreateCoverletNodes(XmlDocument doc, XmlElement configElement, stri node.InnerText = nodeValue; configElement.AppendChild(node); } + + private void CreateCoverleteNullInnerTextNodes(XmlDocument doc, XmlElement configElement, string nodeSetting) + { + var node = doc.CreateNode("element", nodeSetting, string.Empty); + node.InnerText = null; + configElement.AppendChild(node); + } } } diff --git a/test/coverlet.collector.tests/coverlet.collector.tests.csproj b/test/coverlet.collector.tests/coverlet.collector.tests.csproj index b8ff66c36..0e2c3c697 100644 --- a/test/coverlet.collector.tests/coverlet.collector.tests.csproj +++ b/test/coverlet.collector.tests/coverlet.collector.tests.csproj @@ -2,9 +2,8 @@ - netcoreapp2.2 + netcoreapp3.1 false - preview diff --git a/test/coverlet.core.performancetest/coverlet.core.performancetest.csproj b/test/coverlet.core.performancetest/coverlet.core.performancetest.csproj index 893340c17..1c30298df 100644 --- a/test/coverlet.core.performancetest/coverlet.core.performancetest.csproj +++ b/test/coverlet.core.performancetest/coverlet.core.performancetest.csproj @@ -2,7 +2,7 @@ - netcoreapp2.0 + netcoreapp3.1 false diff --git a/test/coverlet.core.tests/Coverage/CoverageTests.cs b/test/coverlet.core.tests/Coverage/CoverageTests.cs index 785a5c322..7d5dc0bfe 100644 --- a/test/coverlet.core.tests/Coverage/CoverageTests.cs +++ b/test/coverlet.core.tests/Coverage/CoverageTests.cs @@ -259,5 +259,107 @@ public void Lambda_Issue343() File.Delete(path); } } + + + [Fact] + public void ExcludeFromCodeCoverage_CompilerGeneratedMethodsAndTypes() + { + string path = Path.GetTempFileName(); + try + { + RemoteExecutor.Invoke(async pathSerialize => + { + CoveragePrepareResult coveragePrepareResult = await TestInstrumentationHelper.Run(instance => + { + ((Task)instance.Test("test")).ConfigureAwait(false).GetAwaiter().GetResult(); + return Task.CompletedTask; + }, pathSerialize); + + return 0; + + }, path).Dispose(); + + CoverageResult result = TestInstrumentationHelper.GetCoverageResult(path); + + var document = result.Document("Instrumentation.ExcludeFromCoverage.cs"); + + // Invoking method "Test" of class "MethodsWithExcludeFromCodeCoverageAttr" we expect to cover 100% lines for MethodsWithExcludeFromCodeCoverageAttr + Assert.DoesNotContain(document.Lines, l => + (l.Value.Class == "Coverlet.Core.Samples.Tests.MethodsWithExcludeFromCodeCoverageAttr" || + // Compiler generated + l.Value.Class.StartsWith("Coverlet.Core.Samples.Tests.MethodsWithExcludeFromCodeCoverageAttr/")) && + l.Value.Hits == 0); + // and 0% for MethodsWithExcludeFromCodeCoverageAttr2 + Assert.DoesNotContain(document.Lines, l => + (l.Value.Class == "Coverlet.Core.Samples.Tests.MethodsWithExcludeFromCodeCoverageAttr2" || + // Compiler generated + l.Value.Class.StartsWith("Coverlet.Core.Samples.Tests.MethodsWithExcludeFromCodeCoverageAttr2/")) && + l.Value.Hits == 1); + } + finally + { + File.Delete(path); + } + } + + [Fact] + public void ExcludeFromCodeCoverage_CompilerGeneratedMethodsAndTypes_NestedMembers() + { + string path = Path.GetTempFileName(); + try + { + RemoteExecutor.Invoke(async pathSerialize => + { + CoveragePrepareResult coveragePrepareResult = await TestInstrumentationHelper.Run(instance => + { + instance.Test(); + return Task.CompletedTask; + }, pathSerialize); + + return 0; + + }, path).Dispose(); + + CoverageResult result = TestInstrumentationHelper.GetCoverageResult(path); + + result.Document("Instrumentation.ExcludeFromCoverage.NestedStateMachines.cs") + .AssertLinesCovered(BuildConfiguration.Debug, (14, 1), (15, 1), (16, 1)) + .AssertNonInstrumentedLines(BuildConfiguration.Debug, 9, 11); + } + finally + { + File.Delete(path); + } + } + + [Fact] + public void ExcludeFromCodeCoverageCompilerGeneratedMethodsAndTypes_Issue670() + { + string path = Path.GetTempFileName(); + try + { + RemoteExecutor.Invoke(async pathSerialize => + { + CoveragePrepareResult coveragePrepareResult = await TestInstrumentationHelper.Run(instance => + { + instance.Test("test"); + return Task.CompletedTask; + }, pathSerialize); + + return 0; + + }, path).Dispose(); + + CoverageResult result = TestInstrumentationHelper.GetCoverageResult(path); + + result.Document("Instrumentation.ExcludeFromCoverage.Issue670.cs") + .AssertLinesCovered(BuildConfiguration.Debug, (8, 1), (9, 1), (10, 1), (11, 1)) + .AssertNonInstrumentedLines(BuildConfiguration.Debug, 15, 53); + } + finally + { + File.Delete(path); + } + } } } \ No newline at end of file diff --git a/test/coverlet.core.tests/Coverage/InstrumenterHelper.cs b/test/coverlet.core.tests/Coverage/InstrumenterHelper.cs index 35b4ff1d5..3034c07d6 100644 --- a/test/coverlet.core.tests/Coverage/InstrumenterHelper.cs +++ b/test/coverlet.core.tests/Coverage/InstrumenterHelper.cs @@ -219,6 +219,30 @@ public static Document AssertLinesCovered(this Document document, BuildConfigura return document; } + public static Document AssertNonInstrumentedLines(this Document document, BuildConfiguration configuration, int from, int to) + { + if (document is null) + { + throw new ArgumentNullException(nameof(document)); + } + + BuildConfiguration buildConfiguration = GetAssemblyBuildConfiguration(); + + if ((buildConfiguration & configuration) != buildConfiguration) + { + return document; + } + + int[] lineRange = Enumerable.Range(from, to - from + 1).ToArray(); + + if (document.Lines.Select(l => l.Value.Number).Intersect(lineRange).Count() > 0) + { + throw new XunitException($"Unexpected instrumented lines, '{string.Join(',', lineRange)}'"); + } + + return document; + } + private static BuildConfiguration GetAssemblyBuildConfiguration() { var configurationAttribute = Assembly.GetExecutingAssembly().GetCustomAttribute(); diff --git a/test/coverlet.core.tests/Helpers/InstrumentationHelperTests.cs b/test/coverlet.core.tests/Helpers/InstrumentationHelperTests.cs index 3772040d2..e311bfce3 100644 --- a/test/coverlet.core.tests/Helpers/InstrumentationHelperTests.cs +++ b/test/coverlet.core.tests/Helpers/InstrumentationHelperTests.cs @@ -143,7 +143,7 @@ public void TestIsTypeExcludedWithoutFilter() [Fact] public void TestIsTypeExcludedNamespace() { - var result = _instrumentationHelper.IsTypeExcluded("Module.dll", "Namespace.Namespace.Type", new string[]{ "[Module]Namespace.Namespace.*" }); + var result = _instrumentationHelper.IsTypeExcluded("Module.dll", "Namespace.Namespace.Type", new string[] { "[Module]Namespace.Namespace.*" }); Assert.True(result); result = _instrumentationHelper.IsTypeExcluded("Module.dll", "Namespace.Namespace.TypeB", new string[] { "[Module]Namespace.Namespace.*" }); @@ -206,23 +206,24 @@ public void TestIncludeDirectories() { string module = typeof(InstrumentationHelperTests).Assembly.Location; - var currentDirModules = _instrumentationHelper.GetCoverableModules(module, - new[] { Environment.CurrentDirectory }, false) - .Where(m => !m.StartsWith("testgen_")).ToArray(); + DirectoryInfo newDir = Directory.CreateDirectory("TestIncludeDirectories"); + DirectoryInfo newDir2 = Directory.CreateDirectory("TestIncludeDirectories2"); + File.Copy(module, Path.Combine(newDir.FullName, Path.GetFileName(module))); + module = Path.Combine(newDir.FullName, Path.GetFileName(module)); + File.Copy("coverlet.msbuild.tasks.dll", Path.Combine(newDir.FullName, "coverlet.msbuild.tasks.dll")); + File.Copy("coverlet.core.dll", Path.Combine(newDir2.FullName, "coverlet.core.dll")); - var parentDirWildcardModules = _instrumentationHelper.GetCoverableModules(module, - new[] { Path.Combine(Directory.GetParent(Environment.CurrentDirectory).FullName, "*") }, false) - .Where(m => !m.StartsWith("testgen_")).ToArray(); + var currentDirModules = _instrumentationHelper.GetCoverableModules(module, Array.Empty(), false); + Assert.Single(currentDirModules); + Assert.Equal("coverlet.msbuild.tasks.dll", Path.GetFileName(currentDirModules[0])); - // There are at least as many modules found when searching the parent directory's subdirectories - Assert.True(parentDirWildcardModules.Length >= currentDirModules.Length); + var moreThanOneDirectory = _instrumentationHelper.GetCoverableModules(module, new string[] { newDir2.FullName }, false); + Assert.Equal(2, moreThanOneDirectory.Length); + Assert.Equal("coverlet.msbuild.tasks.dll", Path.GetFileName(moreThanOneDirectory[0])); + Assert.Equal("coverlet.core.dll", Path.GetFileName(moreThanOneDirectory[1])); - var relativePathModules = _instrumentationHelper.GetCoverableModules(module, - new[] { "." }, false) - .Where(m => !m.StartsWith("testgen_")).ToArray(); - - // Same number of modules found when using a relative path - Assert.Equal(currentDirModules.Length, relativePathModules.Length); + newDir.Delete(true); + newDir2.Delete(true); } public static IEnumerable ValidModuleFilterData => diff --git a/test/coverlet.core.tests/Instrumentation/InstrumenterTests.cs b/test/coverlet.core.tests/Instrumentation/InstrumenterTests.cs index 899161f74..aca0caa51 100644 --- a/test/coverlet.core.tests/Instrumentation/InstrumenterTests.cs +++ b/test/coverlet.core.tests/Instrumentation/InstrumenterTests.cs @@ -346,7 +346,7 @@ public void TestInstrument_ExcludedFilesHelper(string[] excludeFilterHelper, Val [Fact] public void SkipEmbeddedPpdbWithoutLocalSource() { - string xunitDll = Directory.GetFiles(Directory.GetCurrentDirectory(), "xunit.*.dll").First(); + string xunitDll = Directory.GetFiles(Directory.GetCurrentDirectory(), "xunit.core.dll").First(); var loggerMock = new Mock(); Instrumenter instrumenter = new Instrumenter(xunitDll, "_xunit_instrumented", Array.Empty(), Array.Empty(), Array.Empty(), Array.Empty(), false, loggerMock.Object, _instrumentationHelper, new FileSystem()); Assert.True(_instrumentationHelper.HasPdb(xunitDll, out bool embedded)); @@ -460,5 +460,93 @@ public void TestInstrument_NetstandardAwareAssemblyResolver_PreserveCompilationC AssemblyDefinition asm = netstandardResolver.TryWithCustomResolverOnDotNetCore(new AssemblyNameReference("Microsoft.Extensions.Logging.Abstractions", new Version("2.2.0"))); Assert.NotNull(asm); } + + [Fact] + public void TestInstrument_LambdaInsideMethodWithExcludeAttributeAreExcluded() + { + var instrumenterTest = CreateInstrumentor(); + var result = instrumenterTest.Instrumenter.Instrument(); + + var doc = result.Documents.Values.FirstOrDefault(d => Path.GetFileName(d.Path) == "Instrumentation.ExcludeFromCoverage.cs"); + Assert.NotNull(doc); + + Assert.Contains(doc.Lines.Values, l => l.Method == "System.Int32 Coverlet.Core.Samples.Tests.MethodsWithExcludeFromCodeCoverageAttr::TestLambda(System.String,System.Int32)"); + Assert.DoesNotContain(doc.Lines.Values, l => l.Method == "System.Int32 Coverlet.Core.Samples.Tests.MethodsWithExcludeFromCodeCoverageAttr::TestLambda(System.String)"); + Assert.DoesNotContain(doc.Lines.Values, l => l.Class.StartsWith("Coverlet.Core.Samples.Tests.MethodsWithExcludeFromCodeCoverageAttr/") && + instrumenterTest.Instrumenter.IsSynthesizedNameOf(l.Method, "TestLambda", 0)); + Assert.DoesNotContain(doc.Lines.Values, l => l.Method == "System.Int32 Coverlet.Core.Samples.Tests.MethodsWithExcludeFromCodeCoverageAttr2::TestLambda(System.String,System.Int32)"); + Assert.DoesNotContain(doc.Lines.Values, l => l.Class.StartsWith("Coverlet.Core.Samples.Tests.MethodsWithExcludeFromCodeCoverageAttr2/") && + instrumenterTest.Instrumenter.IsSynthesizedNameOf(l.Method, "TestLambda", 1)); + Assert.Contains(doc.Lines.Values, l => l.Method == "System.Int32 Coverlet.Core.Samples.Tests.MethodsWithExcludeFromCodeCoverageAttr2::TestLambda(System.String)"); + + instrumenterTest.Directory.Delete(true); + } + + [Fact] + public void TestInstrument_LocalFunctionInsideMethodWithExcludeAttributeAreExcluded() + { + var instrumenterTest = CreateInstrumentor(); + var result = instrumenterTest.Instrumenter.Instrument(); + + var doc = result.Documents.Values.FirstOrDefault(d => Path.GetFileName(d.Path) == "Instrumentation.ExcludeFromCoverage.cs"); + Assert.NotNull(doc); + + Assert.Contains(doc.Lines.Values, l => l.Method == "System.Int32 Coverlet.Core.Samples.Tests.MethodsWithExcludeFromCodeCoverageAttr::TestLocalFunction(System.String,System.Int32)"); + Assert.DoesNotContain(doc.Lines.Values, l => l.Method == "System.Int32 Coverlet.Core.Samples.Tests.MethodsWithExcludeFromCodeCoverageAttr::TestLocalFunction(System.String)"); + Assert.DoesNotContain(doc.Lines.Values, l => l.Class == "Coverlet.Core.Samples.Tests.MethodsWithExcludeFromCodeCoverageAttr" && + instrumenterTest.Instrumenter.IsSynthesizedNameOf(l.Method, "TestLocalFunction", 6)); + Assert.Contains(doc.Lines.Values, l => l.Class == "Coverlet.Core.Samples.Tests.MethodsWithExcludeFromCodeCoverageAttr" && + instrumenterTest.Instrumenter.IsSynthesizedNameOf(l.Method, "TestLocalFunction", 7)); + Assert.DoesNotContain(doc.Lines.Values, l => l.Method == "System.Int32 Coverlet.Core.Samples.Tests.MethodsWithExcludeFromCodeCoverageAttr2::TestLocalFunction(System.String,System.Int32)"); + Assert.DoesNotContain(doc.Lines.Values, l => l.Class == "Coverlet.Core.Samples.Tests.MethodsWithExcludeFromCodeCoverageAttr2" && + instrumenterTest.Instrumenter.IsSynthesizedNameOf(l.Method, "TestLocalFunction", 7)); + Assert.Contains(doc.Lines.Values, l => l.Method == "System.Int32 Coverlet.Core.Samples.Tests.MethodsWithExcludeFromCodeCoverageAttr2::TestLocalFunction(System.String)"); + Assert.Contains(doc.Lines.Values, l => l.Class == "Coverlet.Core.Samples.Tests.MethodsWithExcludeFromCodeCoverageAttr2" && + instrumenterTest.Instrumenter.IsSynthesizedNameOf(l.Method, "TestLocalFunction", 6)); + + instrumenterTest.Directory.Delete(true); + } + + [Fact] + public void TestInstrument_YieldInsideMethodWithExcludeAttributeAreExcluded() + { + var instrumenterTest = CreateInstrumentor(); + var result = instrumenterTest.Instrumenter.Instrument(); + + var doc = result.Documents.Values.FirstOrDefault(d => Path.GetFileName(d.Path) == "Instrumentation.ExcludeFromCoverage.cs"); + Assert.NotNull(doc); + + Assert.DoesNotContain(doc.Lines.Values, l => l.Class.StartsWith("Coverlet.Core.Samples.Tests.MethodsWithExcludeFromCodeCoverageAttr/") && + instrumenterTest.Instrumenter.IsSynthesizedNameOf(l.Method, "TestYield", 2)); + Assert.Contains(doc.Lines.Values, l => l.Class.StartsWith("Coverlet.Core.Samples.Tests.MethodsWithExcludeFromCodeCoverageAttr/") && + instrumenterTest.Instrumenter.IsSynthesizedNameOf(l.Method, "TestYield", 3)); + Assert.Contains(doc.Lines.Values, l => l.Class.StartsWith("Coverlet.Core.Samples.Tests.MethodsWithExcludeFromCodeCoverageAttr2/") && + instrumenterTest.Instrumenter.IsSynthesizedNameOf(l.Method, "TestYield", 2)); + Assert.DoesNotContain(doc.Lines.Values, l => l.Class.StartsWith("Coverlet.Core.Samples.Tests.MethodsWithExcludeFromCodeCoverageAttr2/") && + instrumenterTest.Instrumenter.IsSynthesizedNameOf(l.Method, "TestYield", 3)); + + instrumenterTest.Directory.Delete(true); + } + + [Fact] + public void TestInstrument_AsyncAwaitInsideMethodWithExcludeAttributeAreExcluded() + { + var instrumenterTest = CreateInstrumentor(); + var result = instrumenterTest.Instrumenter.Instrument(); + + var doc = result.Documents.Values.FirstOrDefault(d => Path.GetFileName(d.Path) == "Instrumentation.ExcludeFromCoverage.cs"); + Assert.NotNull(doc); + + Assert.DoesNotContain(doc.Lines.Values, l => l.Class.StartsWith("Coverlet.Core.Samples.Tests.MethodsWithExcludeFromCodeCoverageAttr/") && + instrumenterTest.Instrumenter.IsSynthesizedNameOf(l.Method, "TestAsyncAwait", 4)); + Assert.Contains(doc.Lines.Values, l => l.Class.StartsWith("Coverlet.Core.Samples.Tests.MethodsWithExcludeFromCodeCoverageAttr/") && + instrumenterTest.Instrumenter.IsSynthesizedNameOf(l.Method, "TestAsyncAwait", 5)); + Assert.Contains(doc.Lines.Values, l => l.Class.StartsWith("Coverlet.Core.Samples.Tests.MethodsWithExcludeFromCodeCoverageAttr2/") && + instrumenterTest.Instrumenter.IsSynthesizedNameOf(l.Method, "TestAsyncAwait", 4)); + Assert.DoesNotContain(doc.Lines.Values, l => l.Class.StartsWith("Coverlet.Core.Samples.Tests.MethodsWithExcludeFromCodeCoverageAttr2/") && + instrumenterTest.Instrumenter.IsSynthesizedNameOf(l.Method, "TestAsyncAwait", 5)); + + instrumenterTest.Directory.Delete(true); + } } } diff --git a/test/coverlet.core.tests/Reporters/CoberturaReporterTests.cs b/test/coverlet.core.tests/Reporters/CoberturaReporterTests.cs index 83e4ab057..6a90f9f5f 100644 --- a/test/coverlet.core.tests/Reporters/CoberturaReporterTests.cs +++ b/test/coverlet.core.tests/Reporters/CoberturaReporterTests.cs @@ -137,52 +137,77 @@ public void TestEnsureParseMethodStringCorrectly( } [Fact] - public void TestReportWithTwoDifferentDirectories() + public void TestReportWithDifferentDirectories() { CoverageResult result = new CoverageResult(); result.Identifier = Guid.NewGuid().ToString(); - var isWindows = RuntimeInformation.IsOSPlatform(OSPlatform.Windows); - string absolutePath1; string absolutePath2; + string absolutePath3; + string absolutePath4; + string absolutePath5; + string absolutePath6; + string absolutePath7; - if (isWindows) + if (RuntimeInformation.IsOSPlatform(OSPlatform.Windows)) { - absolutePath1 = @"C:\projA\file.cs"; - absolutePath2 = @"E:\projB\file.cs"; + absolutePath1 = @"C:\projA\dir1\dir10\file1.cs"; + absolutePath2 = @"C:\projA\dir1\dir10\file2.cs"; + absolutePath3 = @"C:\projA\dir1\file3.cs"; + absolutePath4 = @"E:\projB\dir1\dir10\file4.cs"; + absolutePath5 = @"E:\projB\dir2\file5.cs"; + absolutePath6 = @"F:\file6.cs"; + absolutePath7 = @"F:\"; } else { - absolutePath1 = @"/projA/file.cs"; - absolutePath2 = @"/projB/file.cs"; + absolutePath1 = @"/projA/dir1/dir10/file1.cs"; + absolutePath2 = @"/projA/dir1/file2.cs"; + absolutePath3 = @"/projA/dir1/file3.cs"; + absolutePath4 = @"/projA/dir2/file4.cs"; + absolutePath5 = @"/projA/dir2/file5.cs"; + absolutePath6 = @"/file1.cs"; + absolutePath7 = @"/"; } - var classes = new Classes {{"Class", new Methods()}}; - var documents = new Documents {{absolutePath1, classes}, {absolutePath2, classes}}; + var classes = new Classes { { "Class", new Methods() } }; + var documents = new Documents { { absolutePath1, classes }, + { absolutePath2, classes }, + { absolutePath3, classes }, + { absolutePath4, classes }, + { absolutePath5, classes }, + { absolutePath6, classes }, + { absolutePath7, classes } + }; - result.Modules = new Modules {{"Module", documents}}; + result.Modules = new Modules { { "Module", documents } }; CoberturaReporter reporter = new CoberturaReporter(); string report = reporter.Report(result); var doc = XDocument.Load(new MemoryStream(Encoding.UTF8.GetBytes(report))); - List rootPaths = doc.Element("coverage").Element("sources").Elements().Select(e => e.Value).ToList(); + List basePaths = doc.Element("coverage").Element("sources").Elements().Select(e => e.Value).ToList(); List relativePaths = doc.Element("coverage").Element("packages").Element("package") .Element("classes").Elements().Select(e => e.Attribute("filename").Value).ToList(); List possiblePaths = new List(); - foreach (string root in rootPaths) + foreach (string basePath in basePaths) { foreach (string relativePath in relativePaths) { - possiblePaths.Add(Path.Combine(root, relativePath)); + possiblePaths.Add(Path.Combine(basePath, relativePath)); } } Assert.Contains(absolutePath1, possiblePaths); Assert.Contains(absolutePath2, possiblePaths); + Assert.Contains(absolutePath3, possiblePaths); + Assert.Contains(absolutePath4, possiblePaths); + Assert.Contains(absolutePath5, possiblePaths); + Assert.Contains(absolutePath6, possiblePaths); + Assert.Contains(absolutePath7, possiblePaths); } [Fact] diff --git a/test/coverlet.core.tests/Reporters/Reporters.cs b/test/coverlet.core.tests/Reporters/Reporters.cs index 6ddaf7cdd..f87cf8e9d 100644 --- a/test/coverlet.core.tests/Reporters/Reporters.cs +++ b/test/coverlet.core.tests/Reporters/Reporters.cs @@ -16,15 +16,15 @@ public class Reporters // single tfm [InlineData("", "/folder/reportFolder/", "lcov", "/folder/reportFolder/coverage.info")] [InlineData(null, "/folder/reportFolder/", "cobertura", "/folder/reportFolder/coverage.cobertura.xml")] - [InlineData(null, "/folder/reportFolder/file.ext", "cobertura", "/folder/reportFolder/file.ext.cobertura.xml")] - [InlineData(null, "/folder/reportFolder/file.ext1.ext2", "cobertura", "/folder/reportFolder/file.ext1.ext2.cobertura.xml")] + [InlineData(null, "/folder/reportFolder/file.ext", "cobertura", "/folder/reportFolder/file.ext")] + [InlineData(null, "/folder/reportFolder/file.ext1.ext2", "cobertura", "/folder/reportFolder/file.ext1.ext2")] [InlineData(null, "/folder/reportFolder/file", "cobertura", "/folder/reportFolder/file.cobertura.xml")] [InlineData(null, "file", "cobertura", "file.cobertura.xml")] // multiple tfm [InlineData("netcoreapp2.2", "/folder/reportFolder/", "lcov", "/folder/reportFolder/coverage.netcoreapp2.2.info")] [InlineData("netcoreapp2.2", "/folder/reportFolder/", "cobertura", "/folder/reportFolder/coverage.netcoreapp2.2.cobertura.xml")] - [InlineData("net472", "/folder/reportFolder/file.ext", "cobertura", "/folder/reportFolder/file.net472.ext.cobertura.xml")] - [InlineData("net472", "/folder/reportFolder/file.ext1.ext2", "cobertura", "/folder/reportFolder/file.ext1.net472.ext2.cobertura.xml")] + [InlineData("net472", "/folder/reportFolder/file.ext", "cobertura", "/folder/reportFolder/file.net472.ext")] + [InlineData("net472", "/folder/reportFolder/file.ext1.ext2", "cobertura", "/folder/reportFolder/file.ext1.net472.ext2")] [InlineData("netcoreapp2.2", "/folder/reportFolder/file", "cobertura", "/folder/reportFolder/file.netcoreapp2.2.cobertura.xml")] [InlineData("netcoreapp2.2", "file", "cobertura", "file.netcoreapp2.2.cobertura.xml")] public void Msbuild_ReportWriter(string coverletMultiTargetFrameworksCurrentTFM, string coverletOutput, string reportFormat, string expectedFileName) diff --git a/test/coverlet.core.tests/Samples/Instrumentation.ExcludeFromCoverage.Issue670.cs b/test/coverlet.core.tests/Samples/Instrumentation.ExcludeFromCoverage.Issue670.cs new file mode 100644 index 000000000..27c26aba1 --- /dev/null +++ b/test/coverlet.core.tests/Samples/Instrumentation.ExcludeFromCoverage.Issue670.cs @@ -0,0 +1,54 @@ +// Remember to use full name because adding new using directives change line numbers + +namespace Coverlet.Core.Samples.Tests +{ + public class MethodsWithExcludeFromCodeCoverageAttr_Issue670 + { + public void Test(string input) + { + MethodsWithExcludeFromCodeCoverageAttr_Issue670_Startup obj = new MethodsWithExcludeFromCodeCoverageAttr_Issue670_Startup(); + obj.ObjectExtension(input); + } + } + + [System.Diagnostics.CodeAnalysis.ExcludeFromCodeCoverage] + public class MethodsWithExcludeFromCodeCoverageAttr_Issue670_Startup + { + public void UseExceptionHandler(System.Action action) + { + action(this); + } + + public async void Run(System.Func func) + { + await func(new MethodsWithExcludeFromCodeCoverageAttr_Issue670_Context()); + } + } + + [System.Diagnostics.CodeAnalysis.ExcludeFromCodeCoverage] + public class MethodsWithExcludeFromCodeCoverageAttr_Issue670_Context + { + public System.Threading.Tasks.Task SimulateAsyncWork(int val) + { + return System.Threading.Tasks.Task.Delay(System.Math.Min(val, 50)); + } + } + + public static class MethodsWithExcludeFromCodeCoverageAttr_Issue670_Ext + { + [System.Diagnostics.CodeAnalysis.ExcludeFromCodeCoverage] + public static void ObjectExtension(this Coverlet.Core.Samples.Tests.MethodsWithExcludeFromCodeCoverageAttr_Issue670_Startup obj, string input) + { + obj.UseExceptionHandler(o => + { + o.Run(async context => + { + if (context != null) + { + await context.SimulateAsyncWork(input.Length); + } + }); + }); + } + } +} \ No newline at end of file diff --git a/test/coverlet.core.tests/Samples/Instrumentation.ExcludeFromCoverage.NestedStateMachines.cs b/test/coverlet.core.tests/Samples/Instrumentation.ExcludeFromCoverage.NestedStateMachines.cs new file mode 100644 index 000000000..d1449ad79 --- /dev/null +++ b/test/coverlet.core.tests/Samples/Instrumentation.ExcludeFromCoverage.NestedStateMachines.cs @@ -0,0 +1,18 @@ +// Remember to use full name because adding new using directives change line numbers + +namespace Coverlet.Core.Samples.Tests +{ + public class MethodsWithExcludeFromCodeCoverageAttr_NestedStateMachines + { + [System.Diagnostics.CodeAnalysis.ExcludeFromCodeCoverage] + public async System.Threading.Tasks.Task NestedStateMachines() + { + await System.Threading.Tasks.Task.Run(async () => await System.Threading.Tasks.Task.Delay(50)); + } + + public int Test() + { + return 0; + } + } +} \ No newline at end of file diff --git a/test/coverlet.core.tests/Samples/Instrumentation.ExcludeFromCoverage.cs b/test/coverlet.core.tests/Samples/Instrumentation.ExcludeFromCoverage.cs new file mode 100644 index 000000000..3f75138bc --- /dev/null +++ b/test/coverlet.core.tests/Samples/Instrumentation.ExcludeFromCoverage.cs @@ -0,0 +1,140 @@ +// Remember to use full name because adding new using directives change line numbers + +namespace Coverlet.Core.Samples.Tests +{ + public class MethodsWithExcludeFromCodeCoverageAttr + { + [System.Diagnostics.CodeAnalysis.ExcludeFromCodeCoverage] + public int TestLambda(string input) + { + System.Func lambdaFunc = s => s.Length; + return lambdaFunc(input); + } + + public int TestLambda(string input, int value) + { + System.Func lambdaFunc = s => s.Length; + return lambdaFunc(input) + value; + } + + [System.Diagnostics.CodeAnalysis.ExcludeFromCodeCoverage] + public System.Collections.Generic.IEnumerable TestYield(string input) + { + foreach (char c in input) + { + yield return c; + } + } + + public System.Collections.Generic.IEnumerable TestYield(string input, int value) + { + foreach (char c in input) + { + yield return c + value; + } + } + + [System.Diagnostics.CodeAnalysis.ExcludeFromCodeCoverage] + public async System.Threading.Tasks.Task TestAsyncAwait() + { + await System.Threading.Tasks.Task.Delay(50); + } + + public async System.Threading.Tasks.Task TestAsyncAwait(int value) + { + await System.Threading.Tasks.Task.Delay(System.Math.Min(value, 50)); // Avoid infinite delay + } + + [System.Diagnostics.CodeAnalysis.ExcludeFromCodeCoverage] + public int TestLocalFunction(string input) + { + return LocalFunction(input); + + static int LocalFunction(string input) + { + return input.Length; + } + } + + public int TestLocalFunction(string input, int value) + { + return LocalFunction(input) + value; + + static int LocalFunction(string input) + { + return input.Length; + } + } + + public async System.Threading.Tasks.Task Test(string input) + { + await TestAsyncAwait(1); + return TestLambda(input, 1) + System.Linq.Enumerable.Sum(TestYield(input, 1)) + TestLocalFunction(input, 1); + } + } + + public class MethodsWithExcludeFromCodeCoverageAttr2 + { + public int TestLambda(string input) + { + System.Func lambdaFunc = s => s.Length; + return lambdaFunc(input); + } + + [System.Diagnostics.CodeAnalysis.ExcludeFromCodeCoverage] + public int TestLambda(string input, int value) + { + System.Func lambdaFunc = s => s.Length; + return lambdaFunc(input) + value; + } + + public System.Collections.Generic.IEnumerable TestYield(string input) + { + foreach (char c in input) + { + yield return c; + } + } + + [System.Diagnostics.CodeAnalysis.ExcludeFromCodeCoverage] + public System.Collections.Generic.IEnumerable TestYield(string input, int value) + { + foreach (char c in input) + { + yield return c + value; + } + } + + public async System.Threading.Tasks.Task TestAsyncAwait() + { + await System.Threading.Tasks.Task.Delay(50); + } + + [System.Diagnostics.CodeAnalysis.ExcludeFromCodeCoverage] + public async System.Threading.Tasks.Task TestAsyncAwait(int value) + { + await System.Threading.Tasks.Task.Delay(50); + } + + public int TestLocalFunction(string input) + { + return LocalFunction(input); + + static int LocalFunction(string input) + { + return input.Length; + } + } + + [System.Diagnostics.CodeAnalysis.ExcludeFromCodeCoverage] + public int TestLocalFunction(string input, int value) + { + return LocalFunction(input) + value; + + static int LocalFunction(string input) + { + return input.Length; + } + } + } +} \ No newline at end of file diff --git a/test/coverlet.core.tests/coverlet.core.tests.csproj b/test/coverlet.core.tests/coverlet.core.tests.csproj index 56c196f76..14807ed0b 100644 --- a/test/coverlet.core.tests/coverlet.core.tests.csproj +++ b/test/coverlet.core.tests/coverlet.core.tests.csproj @@ -2,9 +2,8 @@ - netcoreapp2.2 + netcoreapp3.1 false - preview $(NoWarn);CS8002 true diff --git a/test/coverlet.integration.template/coverlet.integration.template.csproj b/test/coverlet.integration.template/coverlet.integration.template.csproj index 141809a7d..301a09754 100644 --- a/test/coverlet.integration.template/coverlet.integration.template.csproj +++ b/test/coverlet.integration.template/coverlet.integration.template.csproj @@ -1,7 +1,7 @@  - netcoreapp2.2 + netcoreapp3.1 false coverletsamplelib.integration.template diff --git a/test/coverlet.integration.tests/BaseTest.cs b/test/coverlet.integration.tests/BaseTest.cs index 739ebbf42..b983078fd 100644 --- a/test/coverlet.integration.tests/BaseTest.cs +++ b/test/coverlet.integration.tests/BaseTest.cs @@ -25,6 +25,12 @@ public abstract class BaseTest private BuildConfiguration GetAssemblyBuildConfiguration() { var configurationAttribute = Assembly.GetExecutingAssembly().GetCustomAttribute(); + + if (configurationAttribute is null) + { + throw new ArgumentNullException("AssemblyConfigurationAttribute not found"); + } + if (configurationAttribute.Configuration.Equals("Debug", StringComparison.InvariantCultureIgnoreCase)) { return BuildConfiguration.Debug; @@ -53,7 +59,7 @@ private protected string GetPackageVersion(string filter) return manifest.Metadata.Version.OriginalVersion; } - private protected ClonedTemplateProject CloneTemplateProject(bool cleanupOnDispose = true) + private protected ClonedTemplateProject CloneTemplateProject(bool cleanupOnDispose = true, string testSDKVersion = "16.4.0") { DirectoryInfo finalRoot = Directory.CreateDirectory(Guid.NewGuid().ToString("N")); foreach (string file in (Directory.GetFiles($"../../../../coverlet.integration.template", "*.cs") @@ -74,7 +80,7 @@ private protected ClonedTemplateProject CloneTemplateProject(bool cleanupOnDispo "); - AddMicrosoftNETTestSdkRef(finalRoot.FullName); + AddMicrosoftNETTestSdkRef(finalRoot.FullName, testSDKVersion); return new ClonedTemplateProject(finalRoot.FullName, cleanupOnDispose); } @@ -123,7 +129,7 @@ private protected void UpdateNugeConfigtWithLocalPackageFolder(string projectPat xml.Save(nugetFile); } - private protected void AddMicrosoftNETTestSdkRef(string projectPath) + private protected void AddMicrosoftNETTestSdkRef(string projectPath, string version) { string csproj = Path.Combine(projectPath, "coverlet.integration.template.csproj"); if (!File.Exists(csproj)) @@ -139,8 +145,7 @@ private protected void AddMicrosoftNETTestSdkRef(string projectPath) xml.Element("Project") .Element("ItemGroup") .Add(new XElement("PackageReference", new XAttribute("Include", "Microsoft.NET.Test.Sdk"), - // We use this due to know issue until official release https://github.com/tonerdo/coverlet/blob/master/Documentation/KnowIssues.md - new XAttribute("Version", "16.5.0-preview-20191115-01"))); + new XAttribute("Version", version))); xml.Save(csproj); } @@ -206,7 +211,7 @@ private protected string AddCollectorRunsettingsFile(string projectPath) return runsettingsPath; } - private protected void AssertCoverage(ClonedTemplateProject clonedTemplateProject, string filter = "coverage.json") + private protected void AssertCoverage(ClonedTemplateProject clonedTemplateProject, string filter = "coverage.json", string standardOutput = "") { bool coverageChecked = false; foreach (string coverageFile in clonedTemplateProject.GetFiles(filter)) @@ -219,7 +224,7 @@ private protected void AssertCoverage(ClonedTemplateProject clonedTemplateProjec coverageChecked = true; } - Assert.True(coverageChecked, "Coverage check fail"); + Assert.True(coverageChecked, $"Coverage check fail\n{standardOutput}"); } private protected void UpdateProjectTargetFramework(ClonedTemplateProject project, params string[] targetFrameworks) @@ -261,6 +266,11 @@ private protected void UpdateProjectTargetFramework(ClonedTemplateProject projec private protected void PinSDK(ClonedTemplateProject project, string sdkVersion) { + if (project is null) + { + throw new ArgumentNullException(nameof(project)); + } + if (string.IsNullOrEmpty(sdkVersion)) { throw new ArgumentException("Invalid sdkVersion", nameof(sdkVersion)); @@ -271,14 +281,19 @@ private protected void PinSDK(ClonedTemplateProject project, string sdkVersion) throw new FileNotFoundException("coverlet.integration.template.csproj not found", "coverlet.integration.template.csproj"); } + if (project.ProjectRootPath is null || !Directory.Exists(project.ProjectRootPath)) + { + throw new ArgumentException("Invalid ProjectRootPath"); + } + File.WriteAllText(Path.Combine(project.ProjectRootPath, "global.json"), $"{{ \"sdk\": {{ \"version\": \"{sdkVersion}\" }} }}"); } } class ClonedTemplateProject : IDisposable { - public string? ProjectRootPath { get; private set; } - public bool _cleanupOnDispose { get; set; } + public string ProjectRootPath { get; private set; } + public bool CleanupOnDispose { get; private set; } // We need to have a different asm name to avoid issue with collectors, we filter [coverlet.*]* by default // https://github.com/tonerdo/coverlet/pull/410#discussion_r284526728 @@ -286,9 +301,30 @@ class ClonedTemplateProject : IDisposable public static string ProjectFileName { get; } = "coverlet.integration.template.csproj"; public string ProjectFileNamePath => Path.Combine(ProjectRootPath, "coverlet.integration.template.csproj"); - public ClonedTemplateProject(string projectRootPath, bool cleanupOnDispose) => (ProjectRootPath, _cleanupOnDispose) = (projectRootPath, cleanupOnDispose); + public ClonedTemplateProject(string? projectRootPath, bool cleanupOnDispose) + { + ProjectRootPath = (projectRootPath ?? throw new ArgumentNullException(nameof(projectRootPath))); + CleanupOnDispose = cleanupOnDispose; + } + public bool IsMultipleTargetFramework() + { + using var csprojStream = File.OpenRead(ProjectFileNamePath); + XDocument xml = XDocument.Load(csprojStream); + return xml.Element("Project").Element("PropertyGroup").Element("TargetFramework") == null; + } + public string[] GetTargetFrameworks() + { + using var csprojStream = File.OpenRead(ProjectFileNamePath); + XDocument xml = XDocument.Load(csprojStream); + XElement element = xml.Element("Project").Element("PropertyGroup").Element("TargetFramework") ?? xml.Element("Project").Element("PropertyGroup").Element("TargetFrameworks"); + if (element is null) + { + throw new ArgumentNullException("No 'TargetFramework' neither 'TargetFrameworks' found in csproj file"); + } + return element.Value.Split(";"); + } public string[] GetFiles(string filter) { @@ -297,7 +333,7 @@ public string[] GetFiles(string filter) public void Dispose() { - if (_cleanupOnDispose) + if (CleanupOnDispose) { Directory.Delete(ProjectRootPath, true); } diff --git a/test/coverlet.integration.tests/Collectors.cs b/test/coverlet.integration.tests/Collectors.cs index 05f785853..9a87eb534 100644 --- a/test/coverlet.integration.tests/Collectors.cs +++ b/test/coverlet.integration.tests/Collectors.cs @@ -1,21 +1,55 @@ -using System.IO; +using System; +using System.IO; using System.Linq; using Xunit; namespace Coverlet.Integration.Tests { - public class Collectors : BaseTest + public class TestSDK_16_2_0 : Collectors { + public TestSDK_16_2_0() + { + TestSDKVersion = "16.2.0"; + } + + private protected override void AssertCollectorsInjection(ClonedTemplateProject clonedTemplateProject) + { + // Check out/in process collectors injection + Assert.Contains("[coverlet]", File.ReadAllText(clonedTemplateProject.GetFiles("log.datacollector.*.txt").Single())); + + // There is a bug in this SDK version https://github.com/microsoft/vstest/pull/2221 + // in-proc coverlet.collector.dll collector with version != 1.0.0.0 won't be loaded + // Assert.Contains("[coverlet]", File.ReadAllText(clonedTemplateProject.GetFiles("log.host.*.txt").Single())); + } + } + + public class TestSDK_Preview : Collectors + { + public TestSDK_Preview() + { + TestSDKVersion = "16.5.0-preview-20200110-02"; + } + } + + public abstract class Collectors : BaseTest + { + protected string? TestSDKVersion { get; set; } + private ClonedTemplateProject PrepareTemplateProject() { - ClonedTemplateProject clonedTemplateProject = CloneTemplateProject(); + if (TestSDKVersion is null) + { + throw new ArgumentNullException("Invalid TestSDKVersion"); + } + + ClonedTemplateProject clonedTemplateProject = CloneTemplateProject(testSDKVersion: TestSDKVersion); UpdateNugeConfigtWithLocalPackageFolder(clonedTemplateProject.ProjectRootPath!); AddCoverletCollectosRef(clonedTemplateProject.ProjectRootPath!); return clonedTemplateProject; } - private void AssertCollectorsInjection(ClonedTemplateProject clonedTemplateProject) + private protected virtual void AssertCollectorsInjection(ClonedTemplateProject clonedTemplateProject) { // Check out/in process collectors injection Assert.Contains("[coverlet]", File.ReadAllText(clonedTemplateProject.GetFiles("log.datacollector.*.txt").Single())); diff --git a/test/coverlet.integration.tests/DotnetTool.cs b/test/coverlet.integration.tests/DotnetTool.cs index 71aeacc58..f0672c8d5 100644 --- a/test/coverlet.integration.tests/DotnetTool.cs +++ b/test/coverlet.integration.tests/DotnetTool.cs @@ -9,7 +9,7 @@ public class DotnetGlobalTools : BaseTest { private string InstallTool(string projectPath) { - DotnetCli($"tool install coverlet.console --version {GetPackageVersion("*console*.nupkg")} --tool-path \"{Path.Combine(projectPath, "coverletTool")}\"", out string standardOutput, out string standardError, projectPath); + _ = DotnetCli($"tool install coverlet.console --version {GetPackageVersion("*console*.nupkg")} --tool-path \"{Path.Combine(projectPath, "coverletTool")}\"", out string standardOutput, out _, projectPath); Assert.Contains("was successfully installed.", standardOutput); return Path.Combine(projectPath, "coverletTool", "coverlet "); } @@ -30,7 +30,7 @@ public void DotnetTool() string publishedTestFile = clonedTemplateProject.GetFiles("*" + ClonedTemplateProject.AssemblyName + ".dll").Single(f => !f.Contains("obj")); RunCommand(coverletToolCommandPath, $"\"{publishedTestFile}\" --target \"dotnet\" --targetargs \"test {Path.Combine(clonedTemplateProject.ProjectRootPath, ClonedTemplateProject.ProjectFileName)} --no-build\" --include-test-assembly --output \"{clonedTemplateProject.ProjectRootPath}\"\\", out standardOutput, out standardError); Assert.Contains("Test Run Successful.", standardOutput); - AssertCoverage(clonedTemplateProject); + AssertCoverage(clonedTemplateProject, standardOutput: standardOutput); } } } diff --git a/test/coverlet.integration.tests/Msbuild.cs b/test/coverlet.integration.tests/Msbuild.cs index 074d1e5a3..0a8cb9228 100644 --- a/test/coverlet.integration.tests/Msbuild.cs +++ b/test/coverlet.integration.tests/Msbuild.cs @@ -1,5 +1,5 @@ using System.IO; - +using System.Linq; using Xunit; namespace Coverlet.Integration.Tests @@ -8,7 +8,7 @@ public class Msbuild : BaseTest { private ClonedTemplateProject PrepareTemplateProject() { - ClonedTemplateProject clonedTemplateProject = CloneTemplateProject(false); + ClonedTemplateProject clonedTemplateProject = CloneTemplateProject(); UpdateNugeConfigtWithLocalPackageFolder(clonedTemplateProject.ProjectRootPath!); AddCoverletMsbuildRef(clonedTemplateProject.ProjectRootPath!); return clonedTemplateProject; @@ -54,8 +54,21 @@ public void TestMsbuild_CoverletOutput_Folder_FileNameExtension() Assert.True(DotnetCli($"test \"{clonedTemplateProject.ProjectRootPath}\" /p:CollectCoverage=true /p:Include=\"[{ClonedTemplateProject.AssemblyName}]*DeepThought\" /p:IncludeTestAssembly=true /p:CoverletOutput=\"{clonedTemplateProject.ProjectRootPath}\"\\file.ext", out string standardOutput, out string standardError), standardOutput); Assert.Contains("Test Run Successful.", standardOutput); Assert.Contains("| coverletsamplelib.integration.template | 100% | 100% | 100% |", standardOutput); - Assert.True(File.Exists(Path.Combine(clonedTemplateProject.ProjectRootPath, "file.ext.json"))); - AssertCoverage(clonedTemplateProject, "file.ext.json"); + Assert.True(File.Exists(Path.Combine(clonedTemplateProject.ProjectRootPath, "file.ext"))); + AssertCoverage(clonedTemplateProject, "file.ext"); + } + + [Fact] + public void TestMsbuild_CoverletOutput_Folder_FileNameExtension_SpecifyFramework() + { + using ClonedTemplateProject clonedTemplateProject = PrepareTemplateProject(); + Assert.False(clonedTemplateProject.IsMultipleTargetFramework()); + string framework = clonedTemplateProject.GetTargetFrameworks().Single(); + Assert.True(DotnetCli($"test -f {framework} \"{clonedTemplateProject.ProjectRootPath}\" /p:CollectCoverage=true /p:Include=\"[{ClonedTemplateProject.AssemblyName}]*DeepThought\" /p:IncludeTestAssembly=true /p:CoverletOutput=\"{clonedTemplateProject.ProjectRootPath}\"\\file.ext", out string standardOutput, out string standardError), standardOutput); + Assert.Contains("Test Run Successful.", standardOutput); + Assert.Contains("| coverletsamplelib.integration.template | 100% | 100% | 100% |", standardOutput); + Assert.True(File.Exists(Path.Combine(clonedTemplateProject.ProjectRootPath, "file.ext"))); + AssertCoverage(clonedTemplateProject, "file.ext"); } [Fact] @@ -65,8 +78,8 @@ public void TestMsbuild_CoverletOutput_Folder_FileNameWithDoubleExtension() Assert.True(DotnetCli($"test \"{clonedTemplateProject.ProjectRootPath}\" /p:CollectCoverage=true /p:Include=\"[{ClonedTemplateProject.AssemblyName}]*DeepThought\" /p:IncludeTestAssembly=true /p:CoverletOutput=\"{clonedTemplateProject.ProjectRootPath}\"\\file.ext1.ext2", out string standardOutput, out string standardError), standardOutput); Assert.Contains("Test Run Successful.", standardOutput); Assert.Contains("| coverletsamplelib.integration.template | 100% | 100% | 100% |", standardOutput); - Assert.True(File.Exists(Path.Combine(clonedTemplateProject.ProjectRootPath, "file.ext1.ext2.json"))); - AssertCoverage(clonedTemplateProject, "file.ext1.ext2.json"); + Assert.True(File.Exists(Path.Combine(clonedTemplateProject.ProjectRootPath, "file.ext1.ext2"))); + AssertCoverage(clonedTemplateProject, "file.ext1.ext2"); } [Fact] @@ -123,6 +136,35 @@ public void Test_MultipleTargetFrameworkReport_CoverletOutput_Folder_FileNameWit AssertCoverage(clonedTemplateProject, "file.*.json"); } + [Fact] + public void Test_MultipleTargetFrameworkReport_CoverletOutput_Folder_FileNameWithExtension_SpecifyFramework() + { + using ClonedTemplateProject clonedTemplateProject = PrepareTemplateProject(); + string[] targetFrameworks = new string[] { "netcoreapp2.2", "netcoreapp2.1" }; + UpdateProjectTargetFramework(clonedTemplateProject, targetFrameworks); + Assert.True(clonedTemplateProject.IsMultipleTargetFramework()); + string[] frameworks = clonedTemplateProject.GetTargetFrameworks(); + Assert.Equal(2, frameworks.Length); + string framework = frameworks.FirstOrDefault(); + Assert.True(DotnetCli($"test -f {framework} \"{clonedTemplateProject.ProjectRootPath}\" /p:CollectCoverage=true /p:Include=\"[{ClonedTemplateProject.AssemblyName}]*DeepThought\" /p:IncludeTestAssembly=true /p:CoverletOutput=\"{clonedTemplateProject.ProjectRootPath}\"\\file.ext", out string standardOutput, out string standardError, clonedTemplateProject.ProjectRootPath!), standardOutput); + Assert.Contains("Test Run Successful.", standardOutput); + Assert.Contains("| coverletsamplelib.integration.template | 100% | 100% | 100% |", standardOutput); + + foreach (string targetFramework in targetFrameworks) + { + if (framework == targetFramework) + { + Assert.True(File.Exists(Path.Combine(clonedTemplateProject.ProjectRootPath, $"file.{targetFramework}.ext"))); + } + else + { + Assert.False(File.Exists(Path.Combine(clonedTemplateProject.ProjectRootPath, $"file.{targetFramework}.ext"))); + } + } + + AssertCoverage(clonedTemplateProject, "file.*.ext"); + } + [Fact] public void Test_MultipleTargetFrameworkReport_CoverletOutput_Folder_FileNameWithExtension() { @@ -135,10 +177,10 @@ public void Test_MultipleTargetFrameworkReport_CoverletOutput_Folder_FileNameWit foreach (string targetFramework in targetFrameworks) { - Assert.True(File.Exists(Path.Combine(clonedTemplateProject.ProjectRootPath, $"file.{targetFramework}.ext.json"))); + Assert.True(File.Exists(Path.Combine(clonedTemplateProject.ProjectRootPath, $"file.{targetFramework}.ext"))); } - AssertCoverage(clonedTemplateProject, "file.*.ext.json"); + AssertCoverage(clonedTemplateProject, "file.*.ext"); } [Fact] @@ -153,10 +195,10 @@ public void Test_MultipleTargetFrameworkReport_CoverletOutput_Folder_FileNameWit foreach (string targetFramework in targetFrameworks) { - Assert.True(File.Exists(Path.Combine(clonedTemplateProject.ProjectRootPath, $"file.ext1.{targetFramework}.ext2.json"))); + Assert.True(File.Exists(Path.Combine(clonedTemplateProject.ProjectRootPath, $"file.ext1.{targetFramework}.ext2"))); } - AssertCoverage(clonedTemplateProject, "file.ext1.*.ext2.json"); + AssertCoverage(clonedTemplateProject, "file.ext1.*.ext2"); } } } diff --git a/test/coverlet.integration.tests/coverlet.integration.tests.csproj b/test/coverlet.integration.tests/coverlet.integration.tests.csproj index 2ccf104a6..6172dead7 100644 --- a/test/coverlet.integration.tests/coverlet.integration.tests.csproj +++ b/test/coverlet.integration.tests/coverlet.integration.tests.csproj @@ -1,9 +1,8 @@  - netcoreapp2.2 + netcoreapp3.1 false - preview enable diff --git a/test/coverlet.tests.projectsample.excludedbyattribute/coverlet.tests.projectsample.excludedbyattribute.csproj b/test/coverlet.tests.projectsample.excludedbyattribute/coverlet.tests.projectsample.excludedbyattribute.csproj index 939b54a16..f38bbae93 100644 --- a/test/coverlet.tests.projectsample.excludedbyattribute/coverlet.tests.projectsample.excludedbyattribute.csproj +++ b/test/coverlet.tests.projectsample.excludedbyattribute/coverlet.tests.projectsample.excludedbyattribute.csproj @@ -1,7 +1,7 @@ - netcoreapp2.2 + netcoreapp3.1 false false diff --git a/test/coverlet.tests.remoteexecutor/coverlet.tests.remoteexecutor.csproj b/test/coverlet.tests.remoteexecutor/coverlet.tests.remoteexecutor.csproj index 39c97693b..6e8630c81 100644 --- a/test/coverlet.tests.remoteexecutor/coverlet.tests.remoteexecutor.csproj +++ b/test/coverlet.tests.remoteexecutor/coverlet.tests.remoteexecutor.csproj @@ -2,7 +2,7 @@ Exe - netcoreapp2.2 + netcoreapp3.1 Coverlet.Tests.RemoteExecutor false false diff --git a/test/coverlet.testsubject/coverlet.testsubject.csproj b/test/coverlet.testsubject/coverlet.testsubject.csproj index bd52eaae4..f38bbae93 100644 --- a/test/coverlet.testsubject/coverlet.testsubject.csproj +++ b/test/coverlet.testsubject/coverlet.testsubject.csproj @@ -1,7 +1,7 @@ - netcoreapp2.0 + netcoreapp3.1 false false