diff --git a/docs/workflow/README.md b/docs/workflow/README.md index 25b9f3f877d9cc..4e0b0dd3b54f23 100644 --- a/docs/workflow/README.md +++ b/docs/workflow/README.md @@ -24,7 +24,7 @@ For Linux and macOS ``` For Windows: -```bat +```cmd build.cmd ``` diff --git a/docs/workflow/building/libraries/README.md b/docs/workflow/building/libraries/README.md index a8bc829d50a84e..9e309d35c364b5 100644 --- a/docs/workflow/building/libraries/README.md +++ b/docs/workflow/building/libraries/README.md @@ -4,7 +4,7 @@ Here is one example of a daily workflow for a developer working mainly on the libraries, in this case using Windows: -```bat +```cmd :: From root: git clean -xdf git pull upstream master & git push origin master @@ -60,7 +60,7 @@ For Linux: ``` For Windows: -```bat +```cmd ./build.cmd -rc Release ``` diff --git a/docs/workflow/building/mono/README.md b/docs/workflow/building/mono/README.md index d1f2851ef2176e..1df3f3fc6b6c89 100644 --- a/docs/workflow/building/mono/README.md +++ b/docs/workflow/building/mono/README.md @@ -16,7 +16,7 @@ To build a complete runtime environment, you need to build both the Mono runtime ./build.sh --subset mono+libs ``` or on Windows, -```bat +```cmd build.cmd -subset mono+libs ``` Note that the debug configuration is the default option. It generates a 'debug' output and that includes asserts, fewer code optimizations, and is easier for debugging. If you want to make performance measurements, or just want tests to execute more quickly, you can also build the 'release' version which does not have these checks by adding the flag `-configuration release` (or `-c release`). @@ -28,7 +28,7 @@ Once you've built the complete runtime and assuming you want to work with just m ./build.sh --subset mono ``` or on Windows, -```bat +```cmd build.cmd -subset mono ``` When the build completes, product binaries will be dropped in the `artifacts\bin\mono\..` folder. @@ -69,7 +69,7 @@ To generate nuget packages: ./build.sh --subset mono -pack (with optional release configuration) ``` or on Windows, -```bat +```cmd build.cmd -subset mono -pack (with optional release configuration) ``` diff --git a/docs/workflow/testing/using-corerun.md b/docs/workflow/testing/using-corerun.md index 33e4d1fdb3a359..f3017ffee017ce 100644 --- a/docs/workflow/testing/using-corerun.md +++ b/docs/workflow/testing/using-corerun.md @@ -30,7 +30,7 @@ Consider that you already have a .NET application DLL called HelloWorld.dll and (You could make such a DLL by using 'dotnet new' 'dotnet restore' 'dotnet build' in a 'HelloWorld' directory). If you execute the following -```bat +```cmd set PATH=%PATH%;%CoreCLR%\artifacts\tests\coreclr\windows.x64.Debug\Tests\Core_Root\ set CORE_LIBRARIES=%ProgramFiles%\dotnet\shared\Microsoft.NETCore.App\1.0.0 @@ -64,7 +64,7 @@ your new runtime. When you execute 'runtime/src/tests/build.cmd' one of the things that it does is set up a directory where it gathers the CoreCLR that has just been built with the pieces of the class library that tests need. It places this runtime in the directory -```bat +```cmd runtime\artifacts\tests\coreclr\..\Tests\Core_Root ``` off the CoreCLR Repository. The way the tests are expected to work is that you set the environment @@ -72,12 +72,12 @@ variable CORE_ROOT to this directory (you don't have to set CORE_LIBRARIES) and you can run any tests. For example after building the tests (running src\tests\build from the repository base) and running 'src\tests\run') you can do the following -```bat +```cmd set PATH=%PATH%;%CoreCLR%\artifacts\Product\windows.x64.Debug set CORE_ROOT=%CoreCLR%\artifacts\tests\coreclr\windows.x64.Debug\Tests\Core_Root ``` sets you up so that corerun can run any of the test. For example -```bat +```cmd corerun artifacts\tests\coreclr\windows.X64.Debug\GC\Features\Finalizer\finalizeio\finalizeio\finalizeio.exe ``` runs the finalizerio test. diff --git a/docs/workflow/testing/using-your-build.md b/docs/workflow/testing/using-your-build.md index 33e5ef19e17409..f3d96b6a144b21 100644 --- a/docs/workflow/testing/using-your-build.md +++ b/docs/workflow/testing/using-your-build.md @@ -33,7 +33,7 @@ For another small walkthrough see [Dogfooding .NET SDK](https://github.com/dotne At this point, you can create a new 'Hello World' program in the standard way. -```bat +```cmd mkdir HelloWorld cd HelloWorld dotnet new console @@ -57,7 +57,7 @@ For Windows you will want `win-x64`, for macOS `osx-x64` and `linux-x64` for Lin Now is the time to publish. The publish step will trigger restore and build. You can iterate on build by calling `dotnet build` as needed. -```bat +```cmd dotnet publish ``` @@ -106,11 +106,11 @@ deploy your new bits. In a lot of cases it is easiest to just copy everything fr You can build just the .NET Library part of the build by doing (debug, for release add 'release' qualifier) (on Linux / OSX us ./build.sh) -```bat +```cmd .\build skiptests skipnative ``` Which builds System.Private.CoreLib.dll if you modify C# code. If you wish to only compile the coreclr.dll you can do - ```bat + ```cmd .\build skiptests skipmscorlib ``` Note that this technique does not work on .NET Apps that have not been published (that is you have not created diff --git a/docs/workflow/using-dotnet-cli.md b/docs/workflow/using-dotnet-cli.md index f63df7aac8530c..bf6f7654bf3546 100644 --- a/docs/workflow/using-dotnet-cli.md +++ b/docs/workflow/using-dotnet-cli.md @@ -104,7 +104,7 @@ static void Main(string[] args) Now is the time to publish. The publish step will trigger restore and build. You can iterate on build by calling `dotnet build` as needed. -```bat +```cmd dotnet publish ``` @@ -179,7 +179,7 @@ So the steps are: ### 1. Build the package again -```bat +```cmd build.cmd clr+libs+host+packs -c release ``` @@ -187,13 +187,13 @@ If you only changed libraries, `build.cmd libs+host+packs -c release` is a littl ### 2. Delete your local package cache -```bat +```cmd rd /s /q c:\localcache ``` ### 3. Publish again -```bat +```cmd dotnet publish ``` diff --git a/eng/testing/debug-dump-template.md b/eng/testing/debug-dump-template.md index bc8a29617b5902..02d38972367124 100644 --- a/eng/testing/debug-dump-template.md +++ b/eng/testing/debug-dump-template.md @@ -1,127 +1,148 @@ -# Debugging a CI dump +# Get the dump -This document describes how to debug a CI/PR test dump by downloading assets from helix, using a dotnet tool called `runfo`. +Click the link to the dump on the `Helix Test Logs` tab in Azure DevOps. This is the same place you got these instructions from. -## What is runfo? +# Get the Helix payload -Runfo is a dotnet global tool that helps get information about helix test runs and azure devops builds. For more information see [this](https://github.com/jaredpar/runfo/tree/master/runfo#runfo) - -### How do I install it? - -You just need to run: - -```script +[Runfo](https://github.com/jaredpar/runfo/tree/master/runfo#runfo) helps get information about helix test runs and azure devops builds. We will use it to download the payload and symbols: +```sh dotnet tool install --global runfo -``` - -If you already have it installed, make sure you have at least version `0.6.1` installed, which contains support to download helix payloads. If you don't have the latest version just run: - -```script dotnet tool update --global runfo ``` - -## Download helix payload containing symbols: - -You can just achieve this by running: - -```script -runfo get-helix-payload -j %JOBID% -w %WORKITEM% -o +If prompted, open a new command prompt to pick up the updated PATH. +```sh +# On Windows +# assumes %WOUTDIR% does not exist +runfo get-helix-payload -j %JOBID% -w %WORKITEM% -o %WOUTDIR% +# On Linux and macOS +# assumes %LOUTDIR% does not exist +runfo get-helix-payload -j %JOBID% -w %WORKITEM% -o %LOUTDIR% ``` > NOTE: if the helix job is an internal job, you need to pass down a [helix authentication token](https://helix.dot.net/Account/Tokens) using the `--helix-token` argument. -This will download the workitem contents under `\workitems\` and the correlation payload under: `\correlation-payload\`. +Now extract the files: -> The correlation payload is usually the testhost or core root, which contain the runtime and dotnet host that we use to run tests. +```sh +# On Windows +for /f %i in ('dir /s/b %WOUTDIR%\*zip') do tar -xf %i -C %WOUTDIR% -Once you have those assets, you will need to extract the testhost or core root. Then extract the workitem assets into the same location where coreclr binary is. +# On Linux and macOS +# obtain `unzip` if necessary; eg `sudo apt-get install unzip` or `sudo dnf install unzip` +find %LOUTDIR% -name '*zip' -exec unzip -d %LOUTDIR% {} \; +``` -## Windows dump on Windows +Now use the [dotnet-sos global tool](https://docs.microsoft.com/en-us/dotnet/core/diagnostics/dotnet-sos) to install the SOS debugging extension. +```sh +dotnet tool install --global dotnet-sos +dotnet tool update --global dotnet-sos +``` +If prompted, open a new command prompt to pick up the updated PATH. +```sh +# Install only one: the one matching your dump +dotnet sos install --architecture Arm +dotnet sos install --architecture Arm64 +dotnet sos install --architecture x86 +dotnet sos install --architecture x64 +``` -### Debug with WinDbg +# Now choose a section below based on your OS. -1. Install [dotnet-sos global tool](https://docs.microsoft.com/en-us/dotnet/core/diagnostics/dotnet-sos). -2. Run `dotnet sos install` (This has an architecture flag to install diferent plugin versions for specific arch scenarios). -3. Load the dump with a recent WinDbg version for it to load sos automatically (if not you can run `.update sos`). It is important that bitness of WinDbg matches the bitness of the dump. -4. Then run the following commands: +## If it's a Windows dump on Windows... -```script -!setclrpath -.sympath+ -``` -### Analyze with dotnet-dump +## ... and you want to debug with WinDbg -1. Install [dotnet-dump global tool](https://docs.microsoft.com/en-us/dotnet/core/diagnostics/dotnet-dump). -2. Run: `dotnet-dump analyze ` -3. Then run the following commands: +Install or update WinDbg if necessary ([external](https://docs.microsoft.com/en-us/windows-hardware/drivers/debugger/debugger-download-tools), [internal](https://osgwiki.com/wiki/Installing_WinDbg)). If you don't have a recent WinDbg you may have to do `.update sos`. -```script -setclrpath (To verify an incorrect DAC hasn't been loaded). -setclrpath -setsymbolserver -directory +Open WinDbg and open the dump with `File>Open Dump`. + +``` +!setclrpath %WOUTDIR%\shared\Microsoft.NETCore.App\6.0.0 +.sympath+ %WOUTDIR%\shared\Microsoft.NETCore.App\6.0.0 ``` -## Linux dumps on Windows +Now you can use regular SOS commands like `!dumpstack`, `!pe`, etc. -In order to debug a Linux dump on Windows, you will have to first go to the PR/CI build -that sent the test run and download the cross DAC. +## ... and you want to debug with Visual Studio -Download the [`CoreCLRCrossDacArtifacts`](https://dev.azure.com/dnceng/public/_apis/build/builds/%BUILDID%/artifacts?artifactName=CoreCLRCrossDacArtifacts&api-version=6.0&%24format=zip), then extract it, and copy the matching flavor of the DAC with your dump and extract it in the same location where coreclr binary is. +Currently this is not possible because mscordbi.dll is not signed. -### Debug with WinDbg +## ... and you want to debug with dotnet-dump -1. Install [dotnet-sos global tool](https://docs.microsoft.com/en-us/dotnet/core/diagnostics/dotnet-sos). -2. Run `dotnet sos install` (This has an architecture flag to install diferent plugin versions for specific arch scenarios). -3. Load the dump with a recent WinDbg version for it to load sos automatically (if not you can run `.update sos`). It is important that bitness of WinDbg matches the bitness of the dump. -4. Then run the following commands: +Install the [dotnet-dump global tool](https://docs.microsoft.com/en-us/dotnet/core/diagnostics/dotnet-dump). +```sh +dotnet tool install --global dotnet-dump +dotnet tool update --global dotnet-dump +``` +If prompted, open a new command prompt to pick up the updated PATH. +```sh +dotnet-dump analyze +``` +Within dotnet-dump: +```sh +setclrpath %WOUTDIR%\shared\Microsoft.NETCore.App\6.0.0 +setsymbolserver -directory %WOUTDIR%\shared\Microsoft.NETCore.App\6.0.0 +``` -```script -!setclrpath -.sympath+ +Now you can use regular SOS commands like `dumpstack`, `pe`, etc. +If you are debugging a 32 bit dump using 64 bit dotnet, you will get an error `SOS does not support the current target architecture`. In that case replace dotnet-dump with the 32 bit version: +```sh +dotnet tool uninstall --global dotnet-dump +"C:\Program Files (x86)\dotnet\dotnet.exe" tool install --global dotnet-dump ``` -### Analyze with dotnet-dump +--- +## If it's a Linux dump on Windows... -1. Install [dotnet-dump global tool](https://docs.microsoft.com/en-us/dotnet/core/diagnostics/dotnet-dump). -2. Run: `dotnet-dump analyze ` -3. Then run the following commands: +Download the [Cross DAC Binaries](https://dev.azure.com/dnceng/public/_apis/build/builds/%BUILDID%/artifacts?artifactName=CoreCLRCrossDacArtifacts&api-version=6.0&%24format=zip), open it and choose the flavor that matches the dump you are to debug, and copy those files to `%WOUTDIR%\shared\Microsoft.NETCore.App\6.0.0`. -```script -setclrpath (To verify an incorrect DAC hasn't been loaded). -setclrpath -setsymbolserver -directory -``` +Now you can debug with WinDbg or `dotnet-dump` as if it was a Windows dump. See above. -## Linux dumps on Linux +--- +## If it's a Linux dump on Linux... -### Debug with LLDB +## ... and you want to debug with LLDB -1. Install [dotnet-sos global tool](https://docs.microsoft.com/en-us/dotnet/core/diagnostics/dotnet-sos). -2. Run `dotnet sos install` (This has an architecture flag to install diferent plugin versions for specific arch scenarios). -3. Load the dump by running `lldb -c ` -4. Run the following commands: +Install or update LLDB if necessary ([instructions here](https://github.com/dotnet/diagnostics/blob/master/documentation/lldb/linux-instructions.md)) -```script -setclrpath -sethostruntime '' -setsymbolserver -directory -loadsymbols (if you want to resolve native symbols) +Load the dump: +```sh +lldb --core %LOUTDIR%/shared/Microsoft.NETCore.App/6.0.0/dotnet ``` -### Analyze with dotnet-dump +Within lldb: +``` +setclrpath %LOUTDIR%/shared/Microsoft.NETCore.App/6.0.0 +sethostruntime /usr/bin/dotnet +setsymbolserver -directory %LOUTDIR%/shared/Microsoft.NETCore.App/6.0.0 +``` +If you want to load native symbols +``` +loadsymbols +``` -1. Install [dotnet-dump global tool](https://docs.microsoft.com/en-us/dotnet/core/diagnostics/dotnet-dump). -2. Run: `dotnet-dump analyze ` -3. Then run the following commands: +## ... and you want to debug with dotnet-dump -```script -setclrpath (To verify an incorrect DAC hasn't been loaded). -setclrpath -setsymbolserver -directory +Install the [dotnet-dump global tool](https://docs.microsoft.com/en-us/dotnet/core/diagnostics/dotnet-dump). +```sh +dotnet tool install --global dotnet-dump +dotnet tool update --global dotnet-dump ``` +If prompted, open a new command prompt to pick up the updated PATH. +```sh +dotnet-dump analyze +``` +Within dotnet-dump: +```sh +setclrpath %LOUTDIR%/shared/Microsoft.NETCore.App/6.0.0 +setsymbolserver -directory %LOUTDIR%/shared/Microsoft.NETCore.App/6.0.0 +``` + +--- +## If it's a macOS dump -## MacOS dumps +Instructions for debugging dumps on macOS are essentially the same as [Linux](#If-it's-a-Linux-dump-on-Linux...) with one exception: `dotnet-dump` cannot analyze macOS system dumps: you must use `lldb` for those. `dotnet-dump` can only analyze dumps created by `dotnet-dump` or `createdump`, by the runtime on crashes when the appropriate environment variables are set, or the [`blame-hang` setting of `dotnet test`](https://docs.microsoft.com/en-us/dotnet/core/tools/dotnet-test). -Instructions for debugging dumps on MacOS the same as [Linux](#linux-dumps-on-linux); however there are a couple of caveats. +--- +# Other Helpful Information -1. It's only supported to debug them in `dotnet-dump` if it's a runtime generated dump. This includes hang dumps and dumps generated by `createdump`, `dotnet-dump` and the runtime itself. -2. If it's a system dump, then only `lldb` works. +* [How to debug a Linux core dump with SOS](https://github.com/dotnet/diagnostics/blob/master/documentation/debugging-coredump.md) diff --git a/eng/testing/gen-debug-dump-docs.py b/eng/testing/gen-debug-dump-docs.py index bd6798b14a6764..8aeb844db3514e 100644 --- a/eng/testing/gen-debug-dump-docs.py +++ b/eng/testing/gen-debug-dump-docs.py @@ -86,8 +86,10 @@ replace_string = '' dir_separator = '/' if platform.system() != 'Windows' else '\\' +unix_user_folder = '~/helix_payload/' +windows_user_folder = 'c:\\helix_payload\\' source_file = template_dir + dir_separator + 'debug-dump-template.md' -with open(source_file, 'r+') as f: +with open(source_file, 'r') as f: file_text = f.read() print('read file: ' + source_file) @@ -95,6 +97,8 @@ replace_string = file_text.replace('%JOBID%', job_id) replace_string = replace_string.replace('%WORKITEM%', workitem) replace_string = replace_string.replace('%BUILDID%', build_id) + replace_string = replace_string.replace('%LOUTDIR%', unix_user_folder + workitem) + replace_string = replace_string.replace('%WOUTDIR%', windows_user_folder + workitem) output_file = out_dir + dir_separator + 'how-to-debug-dump.md' with open(output_file, 'w+') as output: