Error "Hosting components are already initialized. Re-initialization to execute an app is not allowed."

[yegor-pelykh]

I'm working on an app that injects a native dll into another process.
This native dll, in turn, loads the managed dll (which, in turn, loads the Ijwhost.dll).
My managed dll is compiled using .NET 8. (8.0.3).
Everything is injected and works fine if the process where the dll is injected is either an unmanaged process or managed using a .NET Framework.

But one case causes a problem for me. I made a test program using WPF/C#, the simplest one, showing just text on a window.
It is compiled for .NET 8 (8.0.3), just like the managed dll that I inject into the process is compiled.
In this case, the process is closed after injection, and this message is written in the output window:
Hosting components are already initialized. Re-initialization to execute an app is not allowed.
At the same time, I cannot catch any specific exception in the debugger.
The process simply stops and the debugger turns off with this message in the output window.
Also, an error with this text is added to the Windows event log, in the section for notifications about errors in applications.

That is, as I understand it, the problem is because the process itself is compiled with .NET 8 (8.0.3), and my managed dll injects an Ijwhost.dll belonging to the same framework. And when injecting the Ijwhost.dll, somewhere it is determined that the host has already been loaded (considering that this is a managed process.).
And the process crashes with an error.
By the way, if I run this test WPF app without injecting any dlls, then I don’t see that it loads any Ijwhost.dll itself that could conflict with mine.

In my opinion, this should not happen.
After all, if the process uses another framework or even a native one, everything loads correctly.
Why can't it just ignore the second attempt to load the CLR host without throwing an error?

Unfortunately, I wouldn't like to post the complete code of the app, since it is large.
Could we just discuss this theoretically, please?
Could you please fix this behavior?

[vitek-karas]

The native hosting layer doesn't support executing a second app in the same process. Meaning it's not allowed to call hostfxr_run_app - this is because the "running the app" is treated as "this is the process". Some more details: https://github.com/dotnet/runtime/blob/main/docs/design/features/native-hosting.md#start-the-runtime

For a plugin scenarios, or injecting managed code into a random other process, the injected code needs to be invoked via hostfxr_get_runtime_delegate probably using either hdt_load_assembly_and_get_function_pointer or hdt_get_function_pointer.

Another consideration in your scenario is that the hosting layer doesn't support loading two different versions of the CoreCLR into the same process. So if your host process is for example .NET 6 app, and you're trying to inject a .NET 8 managed component, this will fail.

If you need to inject managed code into a random process, the best option currently available is to compile your injected library with Native AOT and export C ABI functions which can then be called just like any other C ABI functions. For more details please refer to: https://learn.microsoft.com/en-us/dotnet/core/deploying/native-aot/?tabs=net7%2Cwindows#build-native-libraries
This approach doesn't have the above limitations because the Native AOT library carries its own isolated copy of the .NET runtime. There are considerations around memory management (more than one GC in the process) and diagnostics though. NativeAOT also comes with its own set of limitations https://learn.microsoft.com/en-us/dotnet/core/deploying/native-aot/?tabs=net7%2Cwindows#limitations-of-native-aot-deployment.

[yegor-pelykh]

Hi @vitek-karas, thank you for the answer!

For a plugin scenarios, or injecting managed code into a random other process, the injected code needs to be invoked via hostfxr_get_runtime_delegate probably using either hdt_load_assembly_and_get_function_pointer or hdt_get_function_pointer.

Do I understand correctly that if I do what is described here, I can still implement loading two different versions of the CoreCLR into one process? Or even that won't help?

If you need to inject managed code into a random process, the best option currently available is to compile your injected library with Native AOT and export C ABI functions which can then be called just like any other C ABI functions. For more details please refer to: https://learn.microsoft.com/en-us/dotnet/core/deploying/native-aot/?tabs=net7%2Cwindows#build-native-libraries
This approach doesn't have the above limitations because the Native AOT library carries its own isolated copy of the .NET runtime. There are considerations around memory management (more than one GC in the process) and diagnostics though. NativeAOT also comes with its own set of limitations https://learn.microsoft.com/en-us/dotnet/core/deploying/native-aot/?tabs=net7%2Cwindows#limitations-of-native-aot-deployment.

Among the limitations of the Native AOT there are only x64 and Arm64 architectures supported.
Do I understand correctly then that it is impossible to compile this under Win32 in order to inject this library into 32-bit processes?

[DaZombieKiller]

Do I understand correctly that if I do what is described here, I can still implement loading two different versions of the CoreCLR into one process? Or even that won't help?

Loading multiple instances of CoreCLR into a single process isn't supported in general right now, and you may encounter strange bugs when attempting to do so. The runtime just isn't tested for that scenario, and there are a number of things it does that make it unlikely to work reliably. For more information, take a look at #12018.

[vitek-karas]

Loading multiple instances of CoreCLR into a single process isn't supported in general right now, and you may encounter strange bugs when attempting to do so.

100% agree. On top of that the hosting layer will try to block doing this. In fact if you find a way to force the hosting layer to load multiple instances of CoreCLR into a process, I personally would consider that a bug.

Do I understand correctly then that it is impossible to compile this under Win32 in order to inject this library into 32-bit processes?

I think this is currently true. @MichalStrehovsky might know if there are plans to support this sometime in the future.

[DaZombieKiller]

@filipnavara has been working on adding 32-bit AOT support, the initial PR was actually just merged a few days ago: #99372

[yegor-pelykh]

Thank you all for your answers!

So it turns out that Native AOT is the only way to achieve this right now.

That is, I can make a C++/CLI library that has both managed and unmanaged code, and build it as a Native AOT, and then be able to inject it into a random process?

[vitek-karas]

That is, I can make a C++/CLI library that has both managed and unmanaged code, and build it as a Native AOT, and then be able to inject it into a random process?

Sorry, I missed that. NativeAOT doesn't support C++/CLI. And we don't have any plans to add support for it.

@AaronRobinsonMSFT might have some additional ideas, but I'm drawing empty right now.

[yegor-pelykh]

Yes, it would be nice to hear some ideas. I'd really like to implement a library with managed/unmanaged code that supports injection into random processes...

[jkotas]

I'd really like to implement a library with managed/unmanaged code

You can still do that with native AOT, just avoid C++/CLI.

https://learn.microsoft.com/dotnet/core/deploying/native-aot/interop#direct-pinvoke-calls explains how to do that.

[yegor-pelykh]

Sorry, I don't understand then. How can I write managed code without using C++/CLI?
Having looked at the link, I still didn’t understand it.

[jkotas]

Could you please share an example what you are not able to achieve without C++/CLI?

[yegor-pelykh]

Hi @jkotas

I'd like to implement two ideas on the managed part:

1. Pipe server.
   My DLL should create a pipe server for IPC. I can implement this in unmanaged code, although I'd prefer a managed NamedPipeServerStream for this. But if it is impossible to use managed code, I will convert the server to unmanaged one.

2. Combined managed/unmanaged code call stack. What's more important.
   My DLL should be able to get the call stack inside hook functions. And I have special requirements for this call stack.
   On the one hand, I don’t need complete information on each stack frame (function, line..), I just need information about the module/assembly.
   But on the other hand, I need to get a full stack for processes in mixed mode, that is, with support for both information about unmanaged modules and managed assemblies. For each stack frame, I need to recognize both its unmanaged module (if the called function is unmanaged) or its managed assembly (if the called function is managed code).
   To be honest, I don’t yet know how to implement this even using managed code. It seems that getting a combined managed/unmanaged stack is considered a real challenge in the internet community. But it seemed to me that I would be closer to the truth if I tried to find approaches using using C++ CLI. But I do not know. I'll be happy if this can be achieved using only native code. Btw it would be good if you, as .NET specialists, could suggest how this can be achieved. Still, somehow Visual Studio shows the combined call stack when debugging, why is this so difficult for any developer to achieve?

[jkotas]

C++ CLI won't help you in any way with what you are trying to achieve.

Still, somehow Visual Studio shows the combined call stack when debugging, why is this so difficult for any developer to achieve?

Visual Studio is a debugger. Debuggers have a lot of complicated low-level code to produce good stacktraces. It is not exactly easy to replicate what the debugger does in an app.

For unmanaged code stacktraces, you would need to use stack unwinding API like RtlVirtualUnwind on Windows or stack unwinding library like https://github.com/libunwind/libunwind , and then resolve the raw addresses produced by stack unwinding into modules.

[yegor-pelykh]

and then resolve the raw addresses produced by stack unwinding into modules.

Please, @jkotas, could you suggest how can I resolve the addresses into path to CLR assemblies? Just the path to assemblies, without function and line info.
I've already tried listing all loaded modules of a process and checking if the address is between its base address and base address + module size. But it did not help.
Are there any correct ways?

[jkotas]

You can either call .NET managed API to get the current managed stacktrace (e.g. new System.Diagnostics.StackTrace()). If the managed API is not low-level enough for your purpose, you can use the low-level .NET runtime profiler APIs.

[yegor-pelykh]

System.Diagnostics.StackTrace() only returns a small part of the full stack, only those frames with managed calls. There are no frames with unmanaged calls at all. But I need a combined stack.

I didn’t know about the profiler APIs. I'll see if this helps. Thanks a lot!