AccessViolationException when running inside of a AppDomain #586

mmanela · 2017-08-27T21:42:30Z

Note: This may be related to#383 but I am filing separately since it has its own unique repro

When I try to run very simple JS code from EdgeJS it throws a AccessViolationException when run inside of an appdomain (see stack below) 100% of the time.

This happens from my sample code below and from inside of an xunit unit test.

This code reproduces the issue:

[Serializable]
    class Program
    {
        static void Main(string[] args)
        {
            var p = new Program();
            if (args.Length > 0 && args[0] == "test")
            {
                p.Run();
            }
            else
            {
                p.RunInAppDomain();
            }
        }

        public void Run()
        {
            Test().Wait();
        }

        public void RunInAppDomain()
        {
            var assemblyFilename = typeof(Program).Assembly.Location;
            RunExeInAppDomain(assemblyFilename);
        }

        public static async Task Test()
        {
            var onMessage = (Func<object, Task<object>>)( (message) =>
            {
                Console.WriteLine($"Got this message: {message}");
                return Task.FromResult((object)"ok");
            });

            var func = Edge.Func(@"
            return function (params, callback) {

                params.onMessage('!!_!! This is cool', function(error, result) {});
                callback(null, 'Node.js welcomes ' + params.message);
            }
        ");
            
            Console.WriteLine(await func(new { message = "YES", onMessage = onMessage }));
        }

        static void RunExeInAppDomain(string assemblyFilename)
        {
            // Create an Application Domain:
            System.AppDomain newDomain = System.AppDomain.CreateDomain("NewApplicationDomain");

            // Load and execute an assembly:
            newDomain.ExecuteAssembly(assemblyFilename,new[] { "test" });

            // Unload the application domain:
            System.AppDomain.Unload(newDomain);
        }
    }

------ Run test started ------
[xUnit.net 00:00:00.2734579]   Starting:    EdgeTest
System.AccessViolationException: Attempted to read or write protected memory. This is often an indication that other memory is corrupt.
   at _atexit_helper(IntPtr func, UInt32* __pexit_list_size, (fnptr)** __ponexitend_e, (fnptr)** __ponexitbegin_e)
   at _atexit_m(IntPtr func)
   at ClrFunc.Initialize(Local<v8::Function>* , Func`2 func)
   at ClrFunc.Initialize(FunctionCallbackInfo<v8::Value>* info)
   at initializeClrFunc(FunctionCallbackInfo<v8::Value>* info)
   at Nan.imp.?A0x8dda69af.FunctionCallbackWrapper(FunctionCallbackInfo<v8::Value>* info)
   at EdgeJs.Edge.NodeStartx86(Int32 argc, String[] argv)
   at EdgeJs.Edge.<>c__DisplayClass11_0.<Func>b__0()
   at System.Threading.ThreadHelper.ThreadStart_Context(Object state)
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.ThreadHelper.ThreadStart()
   at _atexit_helper(IntPtr func, UInt32* __pexit_list_size, (fnptr)** __ponexitend_e, (fnptr)** __ponexitbegin_e)
   at _atexit_m(IntPtr func)
   at ClrFunc.Initialize(Local<v8::Function>* , Func`2 func)
   at ClrFunc.Initialize(FunctionCallbackInfo<v8::Value>* info)
   at initializeClrFunc(FunctionCallbackInfo<v8::Value>* info)
   at Nan.imp.?A0x8dda69af.FunctionCallbackWrapper(FunctionCallbackInfo<v8::Value>* info)
   at EdgeJs.Edge.NodeStartx86(Int32 argc, String[] argv)
   at EdgeJs.Edge.<>c__DisplayClass11_0.<Func>b__0()
   at System.Threading.ThreadHelper.ThreadStart_Context(Object state)
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.ThreadHelper.ThreadStart()

The text was updated successfully, but these errors were encountered:

mmanela · 2017-08-29T17:34:48Z

I turned on the EDGE_DEBUG environment variable and this is what it prints

Load edge native library from: C:\dev\EdgeVA_Repro\EdgeTest\bin\Debug\edge\x86\edge_nativeclr.node
edge::init
V8SynchronizationContext::Initialize
V8SynchronizationContext::Unref
CallbackHelper::Initialize
ClrFunc::Initialize MethodInfo wrapper
ClrFunc::Initialize Func<object,Task<object>> wrapper

Unhandled Exception: System.AccessViolationException: Attempted to read or write protected memory. This is often an indication that other memory is corrupt.
   at _atexit_helper(IntPtr func, UInt32* __pexit_list_size, (fnptr)** __ponexitend_e, (fnptr)** __ponexitbegin_e)
   at _atexit_m(IntPtr func)
   at ClrFunc.Initialize(Local<v8::Function>* , Func`2 func)
   at ClrFunc.Initialize(FunctionCallbackInfo<v8::Value>* info)
   at initializeClrFunc(FunctionCallbackInfo<v8::Value>* info)
   at Nan.imp.?A0x8dda69af.FunctionCallbackWrapper(FunctionCallbackInfo<v8::Value>* info)
   at EdgeJs.Edge.NodeStartx86(Int32 argc, String[] argv)
   at EdgeJs.Edge.<>c__DisplayClass11_0.<Func>b__0()
   at System.Threading.ThreadHelper.ThreadStart_Context(Object state)
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.ThreadHelper.ThreadStart()

agracio · 2017-09-04T17:11:48Z

I have merged #573 to edge-js fork and published it to npm: https://www.npmjs.com/package/edge-js. Please let me know if it fixes the issue.

mmanela · 2017-09-04T17:16:57Z

Thanks!, @agracio Can you also publish the nuget version also?

agracio · 2017-09-04T17:48:47Z

Unfortunately I haven't gotten around to creating nuget package, but you can grab pre-compiled binaries from the release page https://github.com/agracio/edge-js/releases/tag/v8.2.2.
Direct link: https://github.com/agracio/edge-js/releases/download/v8.2.2/edge.js.zip

mmanela · 2017-09-04T22:26:05Z

@agracio I am trying to validate with the zip you linked to but I am getting this error

>.\bin\Debug\EdgeTest.exe
module.js:598
  return process.dlopen(module, path._makeLong(filename));
                 ^

Error: %1 is not a valid Win32 application.
\\?\C:\dev\EdgeVA_Repro\EdgeTest\bin\Debug\edge\x86\edge_nativeclr.node
    at Object.Module._extensions..node (module.js:598:18)
    at Module.load (module.js:503:32)

agracio · 2017-09-04T22:29:47Z

Not sure what it means, do note that it was built using NodeJs 8.2.1. Perhaps try using npm module directly without relying on zip package.

mmanela · 2017-09-04T22:46:19Z

Hmm, got the same thing with NPM. Not sure if I am doing something wrong though.

agracio · 2017-09-04T22:51:35Z

Library build is working fine in both Linux and Windows test environments. Travis: https://travis-ci.org/agracio/edge-js. Appveyor: https://ci.appveyor.com/project/agracio/edge-js.
What version of NodeJs are you using?

mmanela · 2017-09-04T22:52:26Z

v8.4.0

agracio · 2017-09-04T22:59:16Z

Just tried creating a test NodeJs project, ran npm install edge-js and executed a basic scipt

var edge = require('edge-js');

var helloWorld = edge.func(function () {/*
    async (input) => { 
        return ".NET Welcomes " + input.ToString(); 
    }
*/});

helloWorld('JavaScript', function (error, result) {
    if (error) throw error;
    console.log(result);
});

Runs without any problems. Haven't tried to run NodeJs from .NET app though.

mmanela · 2017-09-09T19:55:56Z

Hmm, I wonder what the issue is. The version I consume directly from Nuget (8.2.1) works fine. Is there anything different that happens in the nuget packaging process?

agracio · 2017-09-09T20:02:19Z

To my knowledge I use the same process to build the package. Or at least thats what I think :)
I use tools/build_double.bat 8.2.1 command to build it.

mmanela · 2017-09-09T20:03:36Z

Could you try the nuget package and send it to me so I can try that? I tried myself but I am hitting lots of issues running build_double.

agracio · 2017-09-09T20:04:47Z

Not sure what you mean by trying nuget package and sending it to you :)
Could you share an example project so I can try it out?

mmanela · 2017-09-09T20:23:17Z

Here is my repro of the original issue.

And if you build and then run .\EdgeTest\bin\Debug\EdgeTest.exe you should see

Unhandled Exception: System.AccessViolationException: Attempted to read or write protected memory. This is often an indication that other memory is corrupt.
   at _atexit_helper(IntPtr func, UInt32* __pexit_list_size, (fnptr)** __ponexitend_e, (fnptr)** __ponexitbegin_e)
   at _atexit_m(IntPtr func)
   at ClrFunc.Initialize(Local<v8::Function>* , Func`

I tried then replacing with your dll and got the invalid application issue.

agracio · 2017-09-09T20:31:13Z

Yep, same thing happens on my PC too, something is wrong with the build process. Will try rebuilding.

mmanela · 2017-09-09T20:45:25Z

Well, at least I am not crazy ;)

agracio · 2017-09-10T15:41:21Z

No luck so far, tried rebuilding multiple times following these steps: https://github.com/agracio/edge-js#building-edgejs-nuget-package, but keep getting Error: %1 is not a valid Win32 application.
Have another idea in mind, but will take some time to test.

mmanela · 2017-09-15T03:55:38Z

@agracio What is your other idea?

agracio · 2017-09-15T11:17:39Z

Setting up a Windows 7 Azure VM with VS2015. Will try to build on it instead of my personal PC.
Not very optimistic to be honest since my PC successfully builds all the binaries in /lib/native/ folders, it is unlikely that different build setup would help.

mmanela · 2017-09-15T14:21:05Z

Any idea what the issue could be?

…

On Fri, Sep 15, 2017, 4:17 AM agracio ***@***.***> wrote: Setting up a Windows 7 Azure VM with VS2015. Will try to build using on it instead of my personal PC. Not very optimistic to be honest since my PC successfully builds all the binaries in /lib/native/ folders, it is unlikely that different build setup would help. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#586 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AASlGn7XKfbcUVOhF5txAzXT6m2PbhsFks5silzbgaJpZM4PD7bT> .

agracio · 2017-09-16T10:17:50Z

Ok, so here is where i got to so far.
First about your VS project.
I have noticed that you running it in x86 and it throws AccessViolationException when running in AppDomain mode. When I tried switching it to x64 it produces no output when running in 'AppDomain' mode. Let me know if this is correct or it's something specific to my PC.

Now about the binary build.
After hours of looking for a way to correctly compile it I finally spotted an error in build_double.bat and made some progress. Unfortunately x86 still produces unusable binary, however x64 binary does work.
It correctly runs in AppDomain mode but throws exception when trying to unload it.

Got this message: !!_!! This is cool
Node.js welcomes YES

Unhandled Exception: System.CannotUnloadAppDomainException: Error while unloading appdomain. (Exception from HRESULT: 0x80131015)
   at System.AppDomain.Unload(AppDomain domain)
   at EdgeTest.Program.RunExeInAppDomain(String assemblyFilename) in C:\Users\anton\Dev\EdgeJsRepro\EdgeTest\Program.cs:line 65
   at EdgeTest.Program.RunInAppDomain() in C:\Users\anton\Dev\EdgeJsRepro\EdgeTest\Program.cs:line 34
   at EdgeTest.Program.Main(String[] args) in C:\Users\anton\Dev\EdgeJsRepro\EdgeTest\Program.cs:line 22

I will continue to work on x86 binary but in the meantime let me know your thoughts on the above,

mmanela · 2017-09-16T21:20:44Z

Looking at docs it says that error can be caused by this:

If a thread does not abort, for example because it is executing unmanaged code, or because it is executing a finally block, then after a period of time a CannotUnloadAppDomainException is thrown in the thread that originally called Unload.

But not sure how that could happen since the code waits on the EdgeJs task before continuing.

   public void Run()
        {
            Test().Wait();
        }

agracio · 2017-09-16T22:39:54Z

So that actually explains the problem, unmanaged code is still executing. I noticed that in my Node.js -> .NET apps edge unmanaged code continues to run after the call is completed. You could handle exception or manage creating/unloading app domain elsewhere.

Whats important here is that #573 appears to solve AccessViolationException problem. As soon as I figure out how to compile x86 correctly I will post binaries.

mmanela · 2017-09-17T14:28:24Z

Interesting, how can c# know when the native work is done?

agracio · 2017-09-18T08:25:54Z

.NET cannot really have control over unmanaged resources. Unfortunately I work with large .NET applications that use unmanaged components and when you really need to terminate native component and free all the resources it is holding the only sure way it to terminate and restart .NET process itself.
I think you are misunderstanding the term 'native work is done' in this case :). Edge.js native code might have completed your request but that does not mean the unmanaged code itself has terminated. It could be still finishing internal processes and in that case you could try unloading AppDomain using a 'timeout' and handle the exception if it does not unload after given time. Something along these lines: https://stackoverflow.com/questions/4432141/how-to-properly-unload-an-appdomain-using-c.

mmanela · 2017-09-22T04:03:26Z

Cool sounds fine. Any idea what the x64 issue remaining is?

agracio · 2017-09-22T11:11:55Z

The issue is with x86 build, I got x64 working. Your solution is set to 32 bit and was using x86 binary.

mmanela · 2017-09-22T15:26:07Z

Ok, so there is not issue in the build? Then the PR is all ready to merge? Do we need to merge your build fixes into it?

agracio · 2017-09-23T09:01:23Z

This is not the only PR that is ready to be merged ;) The entire reason I maintain a fork is the fixes it has that have not been merged to edge. Some are PRs from this repo and some don't have PRs as I did not bother creating any more seeing as nothing is merged. This #566 PR for example is essential to my production apps.
As for nuget build process I am guessing @tjanczuk has his own way to build binaries, so my fix is not required.

mmanela · 2017-09-24T02:53:06Z

Ok, can you give me the updated zip of the dlls so I can at least unblock my project.

agracio · 2017-09-24T08:30:22Z

Latest binaries are attached to v.8.2.5 release. Only x64 binary works, so make sure your projects are not set to build at 32 bit like the test project you shared.

mmanela · 2017-09-24T08:40:05Z

Any idea what the issue is with x86? Why doesn't it work?

…

On Sun, Sep 24, 2017, 1:30 AM agracio ***@***.***> wrote: Latest binaries are attached to v.8.2.5 <https://github.com/agracio/edge-js/releases/tag/v8.2.5> release. Only x64 binary works, so make sure your projects are not set to build at 32 bit like the test project you shared. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#586 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AASlGv0NRRtZWU3bcIMjkd3b7SAvF1Myks5slhMmgaJpZM4PD7bT> .

agracio · 2017-09-24T08:47:11Z

Not really, something is wrong with the build process but I cannot figure it out. See if you have better luck. Start with this fix to build_double.bat and see if you can figure the rest out.

mmanela · 2017-09-24T17:02:31Z

Hmm, Ill try. Last time when I ran build_double, it failed on some other error. But will try again and see.

mmanela · 2017-09-24T19:40:54Z

Yea, this is what I get trying to build

creating config.gypi
creating config.mk
Traceback (most recent call last):
  File "configure", line 1415, in <module>
    run_gyp(gyp_args)
  File "tools\gyp_node.py", line 53, in run_gyp
    rc = gyp.main(args)
  File "tools\gyp\pylib\gyp\__init__.py", line 538, in main
    return gyp_main(args)
  File "tools\gyp\pylib\gyp\__init__.py", line 523, in gyp_main
    generator.GenerateOutput(flat_list, targets, data, params)
  File "tools\gyp\pylib\gyp\generator\msvs.py", line 2032, in GenerateOutput
    generator_flags))
  File "tools\gyp\pylib\gyp\generator\msvs.py", line 953, in _GenerateProject
    return _GenerateMSBuildProject(project, options, version, generator_flags)
  File "tools\gyp\pylib\gyp\generator\msvs.py", line 3378, in _GenerateMSBuildProject
    project_file_name)
  File "tools\gyp\pylib\gyp\generator\msvs.py", line 2714, in _GetMSBuildGlobalProperties
    _ConfigWindowsTargetPlatformVersion(configuration, version))
  File "tools\gyp\pylib\gyp\generator\msvs.py", line 311, in _ConfigWindowsTargetPlatformVersion
    return names[0]
IndexError: list index out of range
Failed to create vc project files.
Cannot build node.dll for 8.2.1-x86

agracio · 2017-09-24T20:29:12Z

Looks like error during gyp config rather than compile error. Are you able to build just edge by running something like build.bat release 8.2.1?

mmanela · 2017-09-24T21:31:06Z

That seems to work ok

  Generating code
  Finished generating code
  edge_coreclr.vcxproj -> C:\dev\edge\build\Release\\edge_coreclr.node
  edge_coreclr.vcxproj -> C:\dev\edge\build\Release\edge_coreclr.pdb (Full PDB)
gyp info ok
C:\dev\edge\tools\\..\lib\native\win32\x64\8.2.1
.\build\release\edge_coreclr.node
.\build\release\edge_nativeclr.node
        2 file(s) copied.
C:\dev\edge\tools\\..\lib\native\win32\x64\8.2.1\..\concrt140.dll
C:\dev\edge\tools\\..\lib\native\win32\x64\8.2.1\..\msvcp140.dll
C:\dev\edge\tools\\..\lib\native\win32\x64\8.2.1\..\vccorlib140.dll
C:\dev\edge\tools\\..\lib\native\win32\x64\8.2.1\..\vcruntime140.dll
        4 file(s) copied.
Success building edge.node release for node.js x64 v8.2.1

Michael-Tajmajer-Emrsn · 2017-11-02T23:52:30Z

The issue with the x86 build is due to build_double.bat:

"%SELF%\build\repl.exe" ./build/edge_nativeclr.vcxproj "%USERPROFILE%.node-gyp%1%2\node.lib" "%SELF%build\node-%1-%2\node.lib"

%2 is x86 -- but it needs to be ia32 for %USERPROFILE%.node-gyp%1%2\node.lib
and it needs to be x86 for "%SELF%build\node-%1-%2\node.lib"

I changed it to:
"%SELF%\build\repl.exe" ./build/edge_nativeclr.vcxproj "%USERPROFILE%.node-gyp%1%2\node.lib" "%SELF%build\node-%1-%3\node.lib"

and changed the calling lines to:
call :build_edge %1 ia32 x86
call :build_edge %1 x64 x64

After this, the win32 version ran fine.

As an aside, has anybody managed to get the 8.9.0 (LTS) version of node building?

I end up with a bunch of "cannot convert argument 1 from 'nullptr' to 'nullptr &&'" errors...

mmanela · 2017-11-07T00:37:32Z

@Michael-Tajmajer-Emrsn That is awesome. @agracio could you take his change and build updated binaries for me?

agracio · 2017-11-07T00:50:11Z

@Michael-Tajmajer-Emrsn thanks for this update, I've merged your PR and both x86 and x64 binaries are now working fine. Latest edge-js release includes both v8.9.0 LTS and new v9.0.0.
You need to set msvs_version to 2017 in order to build Node.js 8.3.+ https://github.com/agracio/edge-js/blob/master/tools/build.bat#L57.
@mmanela both x86 and x64 binaries are part of 9.0.0 release https://github.com/agracio/edge-js/releases/tag/v9.0.0.

agracio · 2017-12-23T17:41:04Z

Binaries are now also published to NuGet

mmanela changed the title ~~AccessViolationException when running inside of a XUnit test~~ AccessViolationException when running inside of a AppDomain Aug 28, 2017

mmanela mentioned this issue Sep 3, 2017

fix domain crash #573

Closed

agracio mentioned this issue Sep 18, 2017

throw System.AccessViolationException from edge.js 8.2.1 #581

Closed

agracio mentioned this issue Nov 14, 2017

System.AccessViolationException occurs the first time after I rebuild and restart a debug session in visual studio 2015 #399

Closed

agracio closed this as completed Jun 15, 2024

AccessViolationException when running inside of a AppDomain #586

AccessViolationException when running inside of a AppDomain #586

Comments

mmanela commented Aug 27, 2017 • edited Loading

mmanela commented Aug 29, 2017

agracio commented Sep 4, 2017

mmanela commented Sep 4, 2017 • edited Loading

agracio commented Sep 4, 2017 • edited Loading

mmanela commented Sep 4, 2017

agracio commented Sep 4, 2017

mmanela commented Sep 4, 2017

agracio commented Sep 4, 2017

mmanela commented Sep 4, 2017

agracio commented Sep 4, 2017

mmanela commented Sep 9, 2017

agracio commented Sep 9, 2017

mmanela commented Sep 9, 2017

agracio commented Sep 9, 2017

mmanela commented Sep 9, 2017

agracio commented Sep 9, 2017

mmanela commented Sep 9, 2017

agracio commented Sep 10, 2017

mmanela commented Sep 15, 2017

agracio commented Sep 15, 2017 • edited Loading

mmanela commented Sep 15, 2017 via email

agracio commented Sep 16, 2017

mmanela commented Sep 16, 2017 • edited Loading

agracio commented Sep 16, 2017

mmanela commented Sep 17, 2017

agracio commented Sep 18, 2017

mmanela commented Sep 22, 2017

agracio commented Sep 22, 2017

mmanela commented Sep 22, 2017

agracio commented Sep 23, 2017

mmanela commented Sep 24, 2017

agracio commented Sep 24, 2017

mmanela commented Sep 24, 2017 via email

agracio commented Sep 24, 2017

mmanela commented Sep 24, 2017

mmanela commented Sep 24, 2017

agracio commented Sep 24, 2017

mmanela commented Sep 24, 2017

Michael-Tajmajer-Emrsn commented Nov 2, 2017

mmanela commented Nov 7, 2017

agracio commented Nov 7, 2017

agracio commented Dec 23, 2017

mmanela commented Aug 27, 2017 •

edited

Loading

mmanela commented Sep 4, 2017 •

edited

Loading

agracio commented Sep 4, 2017 •

edited

Loading

agracio commented Sep 15, 2017 •

edited

Loading

mmanela commented Sep 16, 2017 •

edited

Loading