diff --git a/docs/design/mono/components.md b/docs/design/mono/components.md index 2dc982dbeaaf1a..ef17836c8645c5 100644 --- a/docs/design/mono/components.md +++ b/docs/design/mono/components.md @@ -164,10 +164,10 @@ To implement `feature_X` as a component. Carry out the following steps: { feature_X_cleanup }, feature_X_hello, }; - + MonoComponentFeatureX * mono_component_feature_X_init (void) { return &fn_table; } - + void feature_X_cleanup (MonoComponent *self) { static int cleaned = 0; @@ -207,10 +207,10 @@ To implement `feature_X` as a component. Carry out the following steps: { feature_X_cleanup }, feature_X_hello, }; - + MonoComponentFeatureX * mono_component_feature_X_init (void) { return &fn_table; } - + void feature_X_cleanup (MonoComponent *self) { static int cleaned = 0; @@ -229,7 +229,7 @@ To implement `feature_X` as a component. Carry out the following steps: ```c MonoComponentFeatureX* mono_component_feature_X (void); - + ... MonoComponentFeatureX* mono_component_feature_X_stub_init (void); @@ -238,7 +238,7 @@ To implement `feature_X` as a component. Carry out the following steps: * Add an entry to the `components` list to load the component to `mono/metadata/components.c`, and also implement the getter for the component: ```c static MonoComponentFeatureX *feature_X = NULL; - + MonoComponentEntry components[] = { ... {"feature_X", "feature_X", COMPONENT_INIT_FUNC (feature_X), (MonoComponent**)&feature_X, NULL }, @@ -265,16 +265,66 @@ To implement `feature_X` as a component. Carry out the following steps: ## Detailed design - Packaging and runtime packs The components are building blocks to put together a functional runtime. The -runtime pack includes the base runtime and the components and additional -properties and targets that enable the workload to construct a runtime for -various scenarios. - -In each runtime pack we include: - -- The compiled compnents for the apropriate host architectures in a well-known subdirectory -- An MSBuild props file that defines an item group that list each component name and has metadata that indicates: - - the path to the component in the runtime pack - - the path to the stub component in the runtime pack (if components are static) -- An MSBuild targets file that defines targets to copy a specified set of components to the app publish folder (if components are dynamic); or to link the runtime together with stubs and a set of enabled components (if components are static) - -** TODO ** Write this up in more detail +runtime pack includes the base runtime and the components. The mono workload +includes the runtime pack and additional tasks, properties and targets that +enable the workload to construct a runtime for various scenarios. + +For the target RID, we expose: + +- `@(_MonoRuntimeComponentLinking)` set to either `'static'` or `'dynamic'` depending on whether the + current runtime pack for the current target includes runtime components as static archives or as + shared libraries, respectively. +- `@(_MonoRuntimeComponentSharedLibExt)` and `@(_MonoRuntimeComponentStaticLibExt)` set to the file + extension of the runtime components for the current target (ie, `'.a', '.so', '.dylib'` etc). +- `@(_MonoRuntimeAvailableComponents)` a list of component names without the `lib` prefix (if any) + or file extensions. For example: `'hot_reload; diagnostics_tracing'`. + +Each of the above item lists has `RuntimeIdentifier` metadata. For technical reasons the mono +workload will provide a single `@(_MonoRuntimeAvailableComponent)` item list for all platforms. We +use the `RuntimeIdentifier` metadata to filter out the details applicable for the current platform. + +- The target `_MonoSelectRuntimeComponents` that has the following inputs and outputs: + - input `@(_MonoComponent)` (to be set by the workload) : a list of components that a workload wants to use for the current + app. It is an error if this specifies any unknown component name. + - output `@(_MonoRuntimeSelectedComponents)` and `@(_MonoRuntimeSelectedStubComponents)` The names + of the components that were (resp, were not) selected. For example `'hot_reload; + diagnostics_tracing'`. Each item has two metadata properties `ComponentLib` and + `ComponentStubLib` (which may be empty) that specify the name of the static or dynamic library + of the component. This is not the main output of the target, it's primarily for debugging. + - output `@(_MonoRuntimeComponentLink)` a list of library names (relative to the `native/` + subdirectory of the runtime pack) that (for dynamic components) must be placed next to the + runtime in the application bundle, or (for static components) that must be linked with the + runtime to enable the components' functionality. Each item in the list has metadata + `ComponentName` (e.g. `'hot_reload'`), `IsStub` (`true` or `false`), `Linking` (`'static'` or + `'dynamic'`). This output should be used by the workloads when linking the app and runtime if + the workload uses an allow list of native libraries to link or bundle. + - output `@(_MonoRuntimeComponentDontLink)` a list of library names (relative to the `native/` + subdirectory of the runtime pack) that should be excluded from the application bundle (for + dynamic linking) or that should not be passed to the native linker (for static linking). This + output should be used by workloads that just link or bundle every native library from `native/` + in order to filter the contents of the subdirectory to exclude the disabled components (and to + exclude the static library stubs for the enabled components when static linking). + +Generally workloads should only use one of `@(_MonoRuntimeComponentLink)` or +`@(_MonoRuntimeComponentDontLink)`, depending on whether they use an allow or block list for the +contents of the `native/` subdirectory. + +Example fragment (assuming the mono workload has been imported): + +```xml + + + <_MonoComponent Include="hot_reload;diagnostics_tracing" /> + + + + + + + + + + + + +``` diff --git a/src/installer/pkg/sfx/Microsoft.NETCore.App/Microsoft.NETCore.App.Runtime.props b/src/installer/pkg/sfx/Microsoft.NETCore.App/Microsoft.NETCore.App.Runtime.props index 2489544cad8b59..4eea2cf850fa0a 100644 --- a/src/installer/pkg/sfx/Microsoft.NETCore.App/Microsoft.NETCore.App.Runtime.props +++ b/src/installer/pkg/sfx/Microsoft.NETCore.App/Microsoft.NETCore.App.Runtime.props @@ -70,6 +70,12 @@ runtimes/$(RuntimeIdentifier)/native/include/%(RecursiveDir) + + runtimes/$(RuntimeIdentifier)/build/%(RecursiveDir) + + runtimes/$(CoreCLRCrossTargetComponentDirName)_$(TargetArchitecture)/native diff --git a/src/mono/mono.proj b/src/mono/mono.proj index e459f872177169..c26eb9f4e2e545 100644 --- a/src/mono/mono.proj +++ b/src/mono/mono.proj @@ -431,6 +431,9 @@ <_MonoCMakeArgs Include="-DSTATIC_COMPONENTS=1" /> + + <_MonoCMakeArgs Include="-DMONO_COMPONENTS_RID=$(TargetOS)-$(TargetArchitecture)" /> + <_MonoCFLAGSOption>-DCMAKE_C_FLAGS="@(_MonoCPPFLAGS, ' ') @(_MonoCFLAGS, ' ')" @@ -770,6 +773,7 @@ $(RuntimeBinDir)cross\$(PackageRID)\opt$(ExeExt) <_MonoIncludeArtifacts Include="$(MonoObjDir)out\include\**" /> + <_MonoRuntimeBuildArtifacts Include="$(MonoObjDir)\build\**" /> <_MonoRuntimeArtifacts Condition="'$(_MonoIncludeInterpStaticFiles)' == 'true'" Include="$(MonoObjDir)out\lib\libmono-ee-interp.a"> $(RuntimeBinDir)libmono-ee-interp.a @@ -816,6 +820,11 @@ SkipUnchangedFiles="true" Condition="'$(MonoGenerateOffsetsOSGroups)' == '' and ('$(TargetsMacCatalyst)' == 'true' or '$(TargetsiOS)' == 'true' or '$(TargetstvOS)' == 'true' or '$(TargetsAndroid)' == 'true' or '$(TargetsBrowser)' == 'true')"/> + + diff --git a/src/mono/mono/component/CMakeLists.txt b/src/mono/mono/component/CMakeLists.txt index 62ab5b34528328..1bc4eadb672910 100644 --- a/src/mono/mono/component/CMakeLists.txt +++ b/src/mono/mono/component/CMakeLists.txt @@ -82,6 +82,7 @@ set(${MONO_DIAGNOSTICS_TRACING_COMPONENT_NAME}-dependencies #define a library for each component and component stub function(define_component_libs) + # NOTE: keep library naming pattern in sync with RuntimeComponentManifest.targets if (NOT DISABLE_LIBS) foreach(component IN LISTS components) add_library("mono-component-${component}-static" STATIC $) @@ -121,6 +122,7 @@ endforeach() if(NOT DISABLE_COMPONENTS AND NOT STATIC_COMPONENTS) # define a shared library for each component foreach(component IN LISTS components) + # NOTE: keep library naming pattern in sync with RuntimeComponentManifest.targets if(HOST_WIN32) add_library("mono-component-${component}" SHARED "${${component}-sources}") target_compile_definitions("mono-component-${component}" PRIVATE -DCOMPILING_COMPONENT_DYNAMIC;-DMONO_DLL_IMPORT) @@ -172,6 +174,26 @@ foreach(component IN LISTS components) list(APPEND mono-components-stub-objects $) endforeach() +if(NOT MONO_CROSS_COMPILE) + set(TemplateMonoRuntimeComponentSharedLibExt "${CMAKE_SHARED_LIBRARY_SUFFIX}") + set(TemplateMonoRuntimeComponentStaticLibExt "${CMAKE_STATIC_LIBRARY_SUFFIX}") + set(TemplateRuntimeIdentifier "${MONO_COMPONENTS_RID}") + if(DISABLE_COMPONENTS) + set(TemplateMonoRuntimeComponentLinking "static") + set(TemplateMonoRuntimeAvailableComponents "") + else() + list(TRANSFORM components REPLACE "^(.+)$" "{ \"identity\": \"\\1\", \"RuntimeIdentifier\": \"${TemplateRuntimeIdentifier}\" }," OUTPUT_VARIABLE TemplateMonoRuntimeAvailableComponentsList) + list(JOIN TemplateMonoRuntimeAvailableComponentsList "\n" TemplateMonoRuntimeAvailableComponents) + if(STATIC_COMPONENTS) + set(TemplateMonoRuntimeComponentLinking "static") + else() + set(TemplateMonoRuntimeComponentLinking "dynamic") + endif() + endif() + # Write a RuntimeComponentManifest.json file in the artifacts/obj/mono//build/ directory + # without the ../.. the file would go in artifacts/obj/mono//mono/mini + configure_file( "${MONO_COMPONENT_PATH}/RuntimeComponentManifest.json.in" "../../build/RuntimeComponentManifest.json") +endif() # component tests set(MONO_EVENTPIPE_TEST_SOURCE_PATH "${MONO_EVENTPIPE_SHIM_SOURCE_PATH}/test") diff --git a/src/mono/mono/component/RuntimeComponentManifest.json.in b/src/mono/mono/component/RuntimeComponentManifest.json.in new file mode 100644 index 00000000000000..a7ee6a8a83f6b6 --- /dev/null +++ b/src/mono/mono/component/RuntimeComponentManifest.json.in @@ -0,0 +1,16 @@ +{ + "items": { + "_MonoRuntimeComponentLinking": [ + { "identity": "${TemplateMonoRuntimeComponentLinking}", "RuntimeIdentifier": "${TemplateRuntimeIdentifier}" }, + ], + "_MonoRuntimeComponentSharedLibExt": [ + { "identity": "${TemplateMonoRuntimeComponentSharedLibExt}", "RuntimeIdentifier": "${TemplateRuntimeIdentifier}" }, + ], + "_MonoRuntimeComponentStaticLibExt": [ + { "identity": "${TemplateMonoRuntimeComponentStaticLibExt}", "RuntimeIdentifier": "${TemplateRuntimeIdentifier}" }, + ], + "_MonoRuntimeAvailableComponents": [ + ${TemplateMonoRuntimeAvailableComponents} + ], + } +} diff --git a/src/mono/nuget/Microsoft.NET.Runtime.MonoTargets.Sdk/Microsoft.NET.Runtime.MonoTargets.Sdk.pkgproj b/src/mono/nuget/Microsoft.NET.Runtime.MonoTargets.Sdk/Microsoft.NET.Runtime.MonoTargets.Sdk.pkgproj new file mode 100644 index 00000000000000..adf1b80c2f3398 --- /dev/null +++ b/src/mono/nuget/Microsoft.NET.Runtime.MonoTargets.Sdk/Microsoft.NET.Runtime.MonoTargets.Sdk.pkgproj @@ -0,0 +1,23 @@ + + + + + Provides the tasks+targets, for consumption by mono-based workloads + + + + + + + + + + + + + + + + + + diff --git a/src/mono/nuget/Microsoft.NET.Runtime.MonoTargets.Sdk/README.md b/src/mono/nuget/Microsoft.NET.Runtime.MonoTargets.Sdk/README.md new file mode 100644 index 00000000000000..9cb722152afdcd --- /dev/null +++ b/src/mono/nuget/Microsoft.NET.Runtime.MonoTargets.Sdk/README.md @@ -0,0 +1,38 @@ +# Mono Runtime Host support targets + +This Sdk provides additional tasks and targets for workloads hosting the MonoVM .NET runtime. + +## component-manifest.targets + +See https://github.com/dotnet/runtime/blob/main/docs/design/mono/components.md + +## RuntimeConfigParserTask +The `RuntimeConfigParserTask` task converts a json `runtimeconfig.json` to a binary blob for MonoVM's `monovm_runtimeconfig_initialize` API. +To use the task in a project, reference the NuGet package, with the appropriate nuget source. + +### NuGet.config +```xml + + + + + + +``` + +### In the project file +```xml + + + + + + + + + + +``` diff --git a/src/mono/nuget/Microsoft.NET.Runtime.MonoTargets.Sdk/Sdk/RuntimeComponentManifest.props b/src/mono/nuget/Microsoft.NET.Runtime.MonoTargets.Sdk/Sdk/RuntimeComponentManifest.props new file mode 100644 index 00000000000000..d173ff67c2b882 --- /dev/null +++ b/src/mono/nuget/Microsoft.NET.Runtime.MonoTargets.Sdk/Sdk/RuntimeComponentManifest.props @@ -0,0 +1,14 @@ + + + $(MSBuildThisFileDirectory)..\tasks\net6.0\JsonToItemsTaskFactory.dll + $(MSBuildThisFileDirectory)..\tasks\net472\JsonToItemsTaskFactory.dll + + + + <_MonoRuntimeComponentSharedLibExt ParameterType="Microsoft.Build.Framework.ITaskItem[]" Required="false" Output="true" /> + <_MonoRuntimeComponentStaticLibExt ParameterType="Microsoft.Build.Framework.ITaskItem[]" Required="false" Output="true" /> + <_MonoRuntimeComponentLinking ParameterType="Microsoft.Build.Framework.ITaskItem[]" Required="false" Output="true" /> + <_MonoRuntimeAvailableComponents ParameterType="Microsoft.Build.Framework.ITaskItem[]" Required="false" Output="true" /> + + + diff --git a/src/mono/nuget/Microsoft.NET.Runtime.MonoTargets.Sdk/Sdk/RuntimeComponentManifest.targets b/src/mono/nuget/Microsoft.NET.Runtime.MonoTargets.Sdk/Sdk/RuntimeComponentManifest.targets new file mode 100644 index 00000000000000..411aeec01e563e --- /dev/null +++ b/src/mono/nuget/Microsoft.NET.Runtime.MonoTargets.Sdk/Sdk/RuntimeComponentManifest.targets @@ -0,0 +1,112 @@ + + + + + <_MonoRuntimeComponentManifestJsonFilePath Condition="'$(_MonoRuntimeComponentManifestJsonFilePath)' == ''">%(ResolvedRuntimePack.PackageDirectory)\runtimes\$(RuntimeIdentifier)\build\RuntimeComponentManifest.json + + + + + + + + + + + + + + + <_MonoRuntimeComponentCurrentSharedLibExt>@(_MonoRuntimeComponentSharedLibExt->WithMetadataValue('RuntimeIdentifier', '$(RuntimeIdentifier)')) + <_MonoRuntimeComponentCurrentStaticLibExt>@(_MonoRuntimeComponentStaticLibExt->WithMetadataValue('RuntimeIdentifier', '$(RuntimeIdentifier)')) + <_MonoRuntimeComponentCurrentLinking>@(_MonoRuntimeComponentLinking->WithMetadataValue('RuntimeIdentifier', '$(RuntimeIdentifier)')) + + + + + <_MonoRuntimeComponentName Include="@(_MonoRuntimeAvailableComponents)" Condition="'%(RuntimeIdentifier)' == '$(RuntimeIdentifier)'"> + libmono-component-%(Identity)$(_MonoRuntimeComponentCurrentSharedLibExt) + + + + <_MonoRuntimeComponentName Include="@(_MonoRuntimeAvailableComponents)" Condition="'%(RuntimeIdentifier)' == '$(RuntimeIdentifier)'"> + libmono-component-%(Identity)-static$(_MonoRuntimeComponentCurrentStaticLibExt) + libmono-component-%(Identity)-stub-static$(_MonoRuntimeComponentCurrentStaticLibExt) + + + + + + + + + + + + + + + + + + + + + + <_MonoRuntimeComponentNameForRid Include="@(_MonoRuntimeComponentName)" Condition="'%(RuntimeIdentifier)' == '$(RuntimeIdentifier)'" /> + <_MonoRuntimeSelectedComponents Include="@(_MonoRuntimeComponentNameForRid)" Condition="'@(_MonoRuntimeComponentNameForRid)' == '@(_MonoComponent)' and '%(Identity)' != ''" /> + <_MonoRuntimeUnSelectedComponents Include="@(_MonoRuntimeComponentNameForRid)" Exclude="@(_MonoComponent)" /> + <_MonoRuntimeComponentLink Include="%(_MonoRuntimeSelectedComponents.ComponentLib)"> + $(_MonoRuntimeComponentCurrentLinking) + false + %(_MonoRuntimeSelectedComponents.Identity) + + <_MonoRuntimeComponentDontLink Include="%(_MonoRuntimeUnSelectedComponents.ComponentLib)"> + $(_MonoRuntimeComponentCurrentLinking) + false + %(_MonoRuntimeUnSelectedComponents.Identity) + + + <_MonoRuntimeSelectedMissingComponents Include="@(_MonoComponent)" Exclude="@(_MonoRuntimeComponentNameForRid)" /> + + + + + <_MonoRuntimeSelectedStubComponents Include="@(_MonoRuntimeComponentNameForRid)" Exclude="@(_MonoComponent)" /> + <_MonoRuntimeUnSelectedStubComponents Include="@(_MonoRuntimeComponentNameForRid)" Condition="'@(_MonoRuntimeComponentNameForRid)' == '@(_MonoComponent)' and '%(Identity)' != ''" /> + <_MonoRuntimeComponentLink Include="%(_MonoRuntimeSelectedStubComponents.ComponentStubLib)"> + $(_MonoRuntimeComponentCurrentLinking) + true + %(_MonoRuntimeSelectedStubComponents.Identity) + + <_MonoRuntimeComponentDontLink Include="%(_MonoRuntimeUnSelectedStubComponents.ComponentStubLib)"> + $(_MonoRuntimeComponentCurrentLinking) + true + %(_MonoRuntimeUnSelectedStubComponents.Identity) + + + + + + + + diff --git a/src/mono/nuget/Microsoft.NET.Runtime.RuntimeConfigParser.Task/Sdk/Sdk.props b/src/mono/nuget/Microsoft.NET.Runtime.MonoTargets.Sdk/Sdk/RuntimeConfigParserTask.props similarity index 100% rename from src/mono/nuget/Microsoft.NET.Runtime.RuntimeConfigParser.Task/Sdk/Sdk.props rename to src/mono/nuget/Microsoft.NET.Runtime.MonoTargets.Sdk/Sdk/RuntimeConfigParserTask.props diff --git a/src/mono/nuget/Microsoft.NET.Runtime.MonoTargets.Sdk/Sdk/Sdk.props b/src/mono/nuget/Microsoft.NET.Runtime.MonoTargets.Sdk/Sdk/Sdk.props new file mode 100644 index 00000000000000..d292fe15f274a2 --- /dev/null +++ b/src/mono/nuget/Microsoft.NET.Runtime.MonoTargets.Sdk/Sdk/Sdk.props @@ -0,0 +1,4 @@ + + + + diff --git a/src/mono/nuget/Microsoft.NET.Runtime.MonoTargets.Sdk/Sdk/Sdk.targets b/src/mono/nuget/Microsoft.NET.Runtime.MonoTargets.Sdk/Sdk/Sdk.targets new file mode 100644 index 00000000000000..5fdf494324c79a --- /dev/null +++ b/src/mono/nuget/Microsoft.NET.Runtime.MonoTargets.Sdk/Sdk/Sdk.targets @@ -0,0 +1,3 @@ + + + diff --git a/src/mono/nuget/Microsoft.NET.Runtime.RuntimeConfigParser.Task/build/Microsoft.NET.Runtime.RuntimeConfigParser.Task.props b/src/mono/nuget/Microsoft.NET.Runtime.MonoTargets.Sdk/build/Microsoft.NET.Runtime.MonoTargets.Sdk.props similarity index 100% rename from src/mono/nuget/Microsoft.NET.Runtime.RuntimeConfigParser.Task/build/Microsoft.NET.Runtime.RuntimeConfigParser.Task.props rename to src/mono/nuget/Microsoft.NET.Runtime.MonoTargets.Sdk/build/Microsoft.NET.Runtime.MonoTargets.Sdk.props diff --git a/src/mono/nuget/Microsoft.NET.Runtime.RuntimeConfigParser.Task/Microsoft.NET.Runtime.RuntimeConfigParser.Task.pkgproj b/src/mono/nuget/Microsoft.NET.Runtime.RuntimeConfigParser.Task/Microsoft.NET.Runtime.RuntimeConfigParser.Task.pkgproj deleted file mode 100644 index 9591b89b4b9c56..00000000000000 --- a/src/mono/nuget/Microsoft.NET.Runtime.RuntimeConfigParser.Task/Microsoft.NET.Runtime.RuntimeConfigParser.Task.pkgproj +++ /dev/null @@ -1,16 +0,0 @@ - - - - - Provides the RuntimeConfigParser task - - - - - - - - - - - diff --git a/src/mono/nuget/Microsoft.NET.Runtime.RuntimeConfigParser.Task/README.md b/src/mono/nuget/Microsoft.NET.Runtime.RuntimeConfigParser.Task/README.md deleted file mode 100644 index 1d55762a69d637..00000000000000 --- a/src/mono/nuget/Microsoft.NET.Runtime.RuntimeConfigParser.Task/README.md +++ /dev/null @@ -1,29 +0,0 @@ -# RuntimeConfigParser MSBuild Task NuPkg -The `RuntimeConfigParser` MSBuild task is also useful outside the context of `dotnet/runtime`. The task is made available through a NuGet Package containing the `RuntimeConfigParser.dll` assembly produced from building `RuntimeConfigParser.csproj`. To use the task in a project, reference the NuGet package, with the appropriate nuget source. - -## NuGet.config -``` - - - - - - -``` - -## In the project file -``` - - - - - - - - - - -``` diff --git a/src/mono/nuget/Microsoft.NET.Workload.Mono.Toolchain.Manifest/WorkloadManifest.json.in b/src/mono/nuget/Microsoft.NET.Workload.Mono.Toolchain.Manifest/WorkloadManifest.json.in index 059efeed70ceaa..e6a7ff1ee2a025 100644 --- a/src/mono/nuget/Microsoft.NET.Workload.Mono.Toolchain.Manifest/WorkloadManifest.json.in +++ b/src/mono/nuget/Microsoft.NET.Workload.Mono.Toolchain.Manifest/WorkloadManifest.json.in @@ -83,17 +83,17 @@ "abstract": true, "description": "Shared native build tooling for Mono runtime", "packs": [ - "Microsoft.NET.Runtime.RuntimeConfigParser.Task", "Microsoft.NET.Runtime.MonoAOTCompiler.Task", + "Microsoft.NET.Runtime.MonoTargets.Sdk", ], } }, "packs": { - "Microsoft.NET.Runtime.RuntimeConfigParser.Task": { + "Microsoft.NET.Runtime.MonoAOTCompiler.Task": { "kind": "Sdk", "version": "${PackageVersion}" }, - "Microsoft.NET.Runtime.MonoAOTCompiler.Task": { + "Microsoft.NET.Runtime.MonoTargets.Sdk": { "kind": "Sdk", "version": "${PackageVersion}" }, @@ -270,4 +270,4 @@ "version": "${PackageVersion}" }, } -} \ No newline at end of file +} diff --git a/src/mono/nuget/Microsoft.NET.Workload.Mono.Toolchain.Manifest/WorkloadManifest.targets b/src/mono/nuget/Microsoft.NET.Workload.Mono.Toolchain.Manifest/WorkloadManifest.targets index 4176041469bc17..c93e50175e9216 100644 --- a/src/mono/nuget/Microsoft.NET.Workload.Mono.Toolchain.Manifest/WorkloadManifest.targets +++ b/src/mono/nuget/Microsoft.NET.Workload.Mono.Toolchain.Manifest/WorkloadManifest.targets @@ -10,7 +10,10 @@ - + + + + @@ -18,32 +21,37 @@ - - - + + - - - + + - - - + + + + + + + + + + diff --git a/src/mono/nuget/mono-packages.proj b/src/mono/nuget/mono-packages.proj index 9403ca9fb34bf0..a56c63f8827642 100644 --- a/src/mono/nuget/mono-packages.proj +++ b/src/mono/nuget/mono-packages.proj @@ -16,6 +16,9 @@ - + + + + diff --git a/src/tasks/JsonToItemsTaskFactory/JsonToItemsTaskFactory.cs b/src/tasks/JsonToItemsTaskFactory/JsonToItemsTaskFactory.cs new file mode 100644 index 00000000000000..2f7d1dc7ebfbe8 --- /dev/null +++ b/src/tasks/JsonToItemsTaskFactory/JsonToItemsTaskFactory.cs @@ -0,0 +1,398 @@ +// Licensed to the .NET Foundation under one or more agreements. +// The .NET Foundation licenses this file to you under the MIT license. + +using System; +using System.Collections.Generic; +using System.Diagnostics.CodeAnalysis; +using System.IO; +using System.Text.Json; +using System.Text.Json.Serialization; +using System.Threading.Tasks; + +using Microsoft.Build.Framework; +using Microsoft.Build.Utilities; + +#if NET472 +namespace System.Diagnostics.CodeAnalysis { + [AttributeUsage(AttributeTargets.Parameter, Inherited = false)] + + public class NotNullWhenAttribute : Attribute { + public NotNullWhenAttribute(bool returnValue) => ReturnValue = returnValue; + public bool ReturnValue { get; } + } +} +#endif + +namespace JsonToItemsTaskFactory +{ + + /// Reads a json input blob and populates some output items + /// + /// JSON should follow this structure - the toplevel "properties" and "items" keys are exact, other keys are arbitrary. + /// + /// { + /// "properties" : { + /// "propName1": "value1", + /// "propName2": "value" + /// }, + /// "items" : { + /// "itemName1": [ "stringValue", { "identity": "anotherValue", "metadataKey": "metadataValue", ... }, "thirdValue" ], + /// "itemName2": [ ... ] + /// } + /// + /// + /// A task can be declared by + /// + /// + /// + /// + /// + /// + /// + /// + /// + /// + /// And then used in a target. The `JsonFilePath' attribute is used to specify the json file to read. + /// + /// + /// + /// + /// + /// + /// + /// + /// + /// + /// + public class JsonToItemsTaskFactory : ITaskFactory + { + private const string JsonFilePath = "JsonFilePath"; + private TaskPropertyInfo[]? _taskProperties; + private string? _taskName; + + private bool _logDebugTask; + + public JsonToItemsTaskFactory() {} + + public string FactoryName => "JsonToItemsTaskFactory"; + + public Type TaskType => typeof(JsonToItemsTask); + + public bool Initialize(string taskName, IDictionary parameterGroup, string? taskBody, IBuildEngine taskFactoryLoggingHost) + { + _taskName = taskName; + if (taskBody != null && taskBody.StartsWith("debug", StringComparison.InvariantCultureIgnoreCase)) + _logDebugTask = true; + var log = new TaskLoggingHelper(taskFactoryLoggingHost, _taskName); + if (!ValidateParameterGroup (parameterGroup, log)) + return false; + _taskProperties = new TaskPropertyInfo[parameterGroup.Count + 1]; + _taskProperties[0] = new TaskPropertyInfo(nameof(JsonFilePath), typeof(string), output: false, required: true); + parameterGroup.Values.CopyTo(_taskProperties, 1); + return true; + } + + public TaskPropertyInfo[] GetTaskParameters() => _taskProperties!; + + public ITask CreateTask(IBuildEngine taskFactoryLoggingHost) + { + var log = new TaskLoggingHelper(taskFactoryLoggingHost, _taskName); + if (_logDebugTask) log.LogMessage(MessageImportance.Low, "CreateTask called"); + return new JsonToItemsTask(_taskName!, _logDebugTask); + } + + public void CleanupTask(ITask task) {} + + internal bool ValidateParameterGroup(IDictionary parameterGroup, TaskLoggingHelper log) + { + var taskName = _taskName ?? ""; + foreach (var kvp in parameterGroup) + { + var propName = kvp.Key; + var propInfo = kvp.Value; + if (string.Equals(propName, nameof(JsonFilePath), StringComparison.InvariantCultureIgnoreCase)) + { + log.LogError($"Task {taskName}: {nameof(JsonFilePath)} parameter must not be declared. It is implicitly added by the task."); + continue; + } + + if (!propInfo.Output) + { + log.LogError($"Task {taskName}: parameter {propName} is not an output. All parameters except {nameof(JsonFilePath)} must be outputs"); + continue; + } + if (propInfo.Required) + { + log.LogError($"Task {taskName}: parameter {propName} is an output but is marked required. That's not supported."); + } + if (typeof(ITaskItem[]).IsAssignableFrom(propInfo.PropertyType)) + continue; // ok, an item list + if (typeof(string).IsAssignableFrom(propInfo.PropertyType)) + continue; // ok, a string property + + log.LogError($"Task {taskName}: parameter {propName} is not an output of type System.String or Microsoft.Build.Framework.ITaskItem[]"); + } + return !log.HasLoggedErrors; + } + + public class JsonToItemsTask : IGeneratedTask + { + private IBuildEngine? _buildEngine; + public IBuildEngine BuildEngine { get => _buildEngine!; set { _buildEngine = value; SetBuildEngine(value);} } + public ITaskHost? HostObject { get; set; } + + private TaskLoggingHelper? _log; + private TaskLoggingHelper Log { get => _log!; set { _log = value; } } + + private void SetBuildEngine(IBuildEngine buildEngine) + { + Log = new TaskLoggingHelper(buildEngine, TaskName); + } + + public static JsonSerializerOptions JsonOptions => new() + { + PropertyNameCaseInsensitive = true, + AllowTrailingCommas = true, + }; + private string? jsonFilePath; + + private readonly bool _logDebugTask; // print stuff to the log for debugging the task + + private JsonModelRoot? jsonModel; + public string TaskName {get;} + public JsonToItemsTask(string taskName, bool logDebugTask = false) + { + TaskName = taskName; + _logDebugTask = logDebugTask; + } + + public bool Execute() + { + if (jsonFilePath == null) + { + Log.LogError($"no {nameof(JsonFilePath)} specified"); + return false; + } + if (!TryGetJson(jsonFilePath, out var json)) + return false; + + if (_logDebugTask) + { + LogParsedJson(json); + } + jsonModel = json; + return true; + } + + public bool TryGetJson(string jsonFilePath, [NotNullWhen(true)] out JsonModelRoot? json) + { + FileStream? file = null; + try + { + try + { + file = File.OpenRead(jsonFilePath); + } + catch (FileNotFoundException fnfe) + { + Log.LogErrorFromException(fnfe); + json = null; + return false; + } + json = GetJsonAsync(jsonFilePath, file).Result; + if (json == null) + { + // the async task may have already caught an exception and logged it. + if (!Log.HasLoggedErrors) Log.LogError($"Failed to deserialize json from file {jsonFilePath}"); + return false; + } + return true; + } + finally + { + if (file != null) + file.Dispose(); + } + } + + public async Task GetJsonAsync(string jsonFilePath, FileStream file) + { + JsonModelRoot? json = null; + try + { + json = await JsonSerializer.DeserializeAsync(file, JsonOptions).ConfigureAwait(false); + } + catch (JsonException e) + { + Log.LogError($"Failed to deserialize json from file '{jsonFilePath}', JSON Path: {e.Path}, Line: {e.LineNumber}, Position: {e.BytePositionInLine}"); + Log.LogErrorFromException(e, showStackTrace: false, showDetail: true, file: null); + } + return json; + } + + internal void LogParsedJson (JsonModelRoot json) + { + if (json.Properties != null) + { + Log.LogMessage(MessageImportance.Low, "json has properties: "); + foreach (var property in json.Properties) + { + Log.LogMessage(MessageImportance.Low, $" {property.Key} = {property.Value}"); + } + } + if (json.Items != null) + { + Log.LogMessage(MessageImportance.Low, "items: "); + foreach (var item in json.Items) + { + Log.LogMessage(MessageImportance.Low, $" {item.Key} = ["); + foreach (var value in item.Value) + { + Log.LogMessage(MessageImportance.Low, $" {value.Identity}"); + if (value.Metadata != null) + { + Log.LogMessage(MessageImportance.Low, " and some metadata, too"); + } + } + Log.LogMessage(MessageImportance.Low, " ]"); + } + } + } + + public object? GetPropertyValue(TaskPropertyInfo property) + { + bool isItem = false; + if (typeof(ITaskItem[]).IsAssignableFrom(property.PropertyType)) + { + if (_logDebugTask) Log.LogMessage(MessageImportance.Low, "GetPropertyValue called with @({0})", property.Name); + isItem = true; + } + else + { + if (_logDebugTask) Log.LogMessage(MessageImportance.Low, "GetPropertyValue called with $({0})", property.Name); + } + if (!isItem) + { + if (jsonModel?.Properties != null && jsonModel.Properties.TryGetValue(property.Name, out var value)) + { + return value; + } + Log.LogError("Property {0} not found in {1}", property.Name, jsonFilePath); + throw new Exception(); + } + else + { + if (jsonModel?.Items != null && jsonModel.Items.TryGetValue(property.Name, out var itemModels)) + { + return ConvertItems(itemModels); + } + + } + return null; + } + + public static ITaskItem[] ConvertItems(JsonModelItem[] itemModels) + { + var items = new ITaskItem[itemModels.Length]; + for (int i = 0; i < itemModels.Length; i++) + { + var itemModel = itemModels[i]; + var item = new TaskItem(itemModel.Identity); + if (itemModel.Metadata != null) + { + // assume Identity key was already removed in JsonModelItem + foreach (var metadata in itemModel.Metadata) + { + item.SetMetadata(metadata.Key, metadata.Value); + } + } + items[i] = item; + } + return items; + } + + public void SetPropertyValue(TaskPropertyInfo property, object? value) + { + if (_logDebugTask) Log.LogMessage(MessageImportance.Low, "SetPropertyValue called with {0}", property.Name); + if (property.Name == "JsonFilePath") + { + jsonFilePath = (string)value!; + } + else + throw new Exception($"JsonToItemsTask {TaskName} cannot set property {property.Name}"); + } + + } + + public class JsonModelRoot + { + [JsonConverter(typeof(CaseInsensitiveDictionaryConverter))] + public Dictionary? Properties {get; set;} + public Dictionary? Items {get; set;} + + public JsonModelRoot() {} + } + + [JsonConverter(typeof(JsonModelItemConverter))] + public class JsonModelItem + { + public string Identity {get;} + // n.b. will be deserialized case insensitive + public Dictionary? Metadata {get;} + + public JsonModelItem(string identity, Dictionary? metadata) + { + Identity = identity; + Metadata = metadata; + } + } + + public class CaseInsensitiveDictionaryConverter : JsonConverter> + { + public override Dictionary Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options) + { + var dict = JsonSerializer.Deserialize>(ref reader, options); + if (dict == null) + return null!; + return new Dictionary(dict, StringComparer.OrdinalIgnoreCase); + } + public override void Write(Utf8JsonWriter writer, Dictionary? value, JsonSerializerOptions options) => + JsonSerializer.Serialize(writer, value, options); + } + public class JsonModelItemConverter : JsonConverter + { + public JsonModelItemConverter() {} + + public override JsonModelItem Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options) + { + switch (reader.TokenType) + { + case JsonTokenType.String: + var stringItem = reader.GetString(); + if (string.IsNullOrEmpty(stringItem)) + throw new JsonException ("deserialized json string item was null or the empty string"); + return new JsonModelItem(stringItem!, metadata: null); + case JsonTokenType.StartObject: + var dict = JsonSerializer.Deserialize>(ref reader, options); + if (dict == null) + return null!; + var idict = new Dictionary (dict, StringComparer.OrdinalIgnoreCase); + if (!idict.TryGetValue("Identity", out var identity) || string.IsNullOrEmpty(identity)) + throw new JsonException ("deserialized json dictionary item did not have a non-empty Identity metadata"); + else + idict.Remove("Identity"); + return new JsonModelItem(identity, metadata: idict); + default: + throw new NotSupportedException(); + } + } + public override void Write(Utf8JsonWriter writer, JsonModelItem value, JsonSerializerOptions options) + { + if (value.Metadata == null) + JsonSerializer.Serialize(writer, value.Identity); + else + JsonSerializer.Serialize(writer, value.Metadata); /* assumes Identity is in there */ + } + } + } +} diff --git a/src/tasks/JsonToItemsTaskFactory/JsonToItemsTaskFactory.csproj b/src/tasks/JsonToItemsTaskFactory/JsonToItemsTaskFactory.csproj new file mode 100644 index 00000000000000..d6df214fce7963 --- /dev/null +++ b/src/tasks/JsonToItemsTaskFactory/JsonToItemsTaskFactory.csproj @@ -0,0 +1,32 @@ + + + $(TargetFrameworkForNETCoreTasks);$(TargetFrameworkForNETFrameworkTasks) + Library + true + false + enable + + + + + + + + + + + + + + + + + + <_PublishFramework Remove="@(_PublishFramework)" /> + <_PublishFramework Include="$(TargetFrameworks)" /> + + + + + + diff --git a/src/tasks/JsonToItemsTaskFactory/README.md b/src/tasks/JsonToItemsTaskFactory/README.md new file mode 100644 index 00000000000000..619991ac13e993 --- /dev/null +++ b/src/tasks/JsonToItemsTaskFactory/README.md @@ -0,0 +1,86 @@ +# JsonToItemsTaskFactory + +A utility for reading json blobs into MSBuild items and properties. + +## Json blob format + +The json data must be a single toplevel dictionary with a `"properties"` or an `"items"` key (both are optional). + +The `"properties"` value must be a dictionary with more string values. The keys are case-insensitive and duplicates are not allowed. + +The `"items"` value must be an array of either string or dictionary elements (or a mix of both). +String elements use the string value as the `Identity`. +Dictionary elements must have strings as values, and must include an `"Identity"` key, and as many other metadata key/value pairs as desired. This dictionary is also case-insensitive and duplicate metadata keys are also not allowed. + +#### Example + +```json +{ + "properties": { + "x1": "val1", + "X2": "val2", + }, + "items" : { + "FunFiles": ["funFile1.txt", "funFile2.txt"], + "FilesWithMeta": [{"identity": "funFile3.txt", "TargetPath": "bin/fun3"}, + "funFile3.and.a.half.txt", + {"identity": "funFile4.txt", "TargetPath": "bin/fun4"}] + } +} +``` + +## UsingTask and Writing Targets + +To use the task, you need to reference the assembly and add the task to the project, as well as declare the task parameters that correspond to the properties and items you want to retrieve from the json blob. + +```xml + + + + + + + +``` + +The parameter group parameters are all optional. They must be non-required outputs of type `System.String` or `Microsoft.Build.Framework.ITaskItem[]`. The former declares properties to capture from the file, while the latter declares item lists. + +The above declares a task `MyJsonReader` which will be used to retries the `X1` property and the `FunFiles` and `FilesWithMeta` items. + +To use the task, a `JsonFilePath` attribute specifies the file to read. + +```xml + + + + + + + + + + + + + + + +``` + +When the target `RunMe` runs, the task will read the json file and populate the outputs. Running the target, the output will be: + +```console +$ dotnet build Example + X1 = val1 + FunFiles = funFile1.txt;funFile2.txt + FilesWithMeta = funFile3.txt TargetPath='bin/fun3' + FilesWithMeta = funFile4.txt TargetPath='bin/fun4' + FilesWithMeta = funFile3.and.a.half.txt (No TargetPath) + +Build succeeded. + 0 Warning(s) + 0 Error(s) +Time Elapsed 00:00:00.15 +``` +