Crash when attaching a V8 debuger while in a host callback (wrong AppDomain?)

Sep 20, 2013 at 4:23 PM
Edited Sep 20, 2013 at 4:25 PM
NB: I used this tutorial to setup eclipse (https://github.com/joyent/node/wiki/Using-Eclipse-as-Node-Applications-Debugger) ; and selected "Standalone V8 VM" in "Run/Debug Configurations...".

The problem is triggered when attaching the debugger while ClearScript/V8 executes some code on the Host.

An interesting point is that the ClearScript.dll seems to get loaded a second time in another AppDomain when the debugger is attached.

The releases build crashes when getting the V8-debugger attached; but the debug gives an interesting exception message first :
Cannot pass a GCHandle across AppDomains
I had the following stack trace :
    mscorlib.dll!System.Runtime.InteropServices.GCHandle.GCHandle(System.IntPtr handle) + 0x23 bytes    
    mscorlib.dll!System.Runtime.InteropServices.GCHandle.FromIntPtr(System.IntPtr value) + 0x66 bytes   
>   ClearScript.dll!Microsoft.ClearScript.V8.V8ProxyHelpers.GetHostObject(void* pObject = 0x000000001b3a1328) Line 108 + 0x19 bytes C#
    ClearScript.dll!Microsoft.ClearScript.V8.V8ProxyHelpers.RemoveV8ObjectCacheEntry(void* pV8ObjectCache = 0x000000001b3a1328, void* pObject = 0x000000001b3a1330) Line 235 + 0xa bytes    C#
    ClearScriptV8-64.dll!HostObjectHelpers::RemoveV8ObjectCacheEntry(void* pvCache = 0x000000001B3A1328, void* pvObject = 0x000000001B3A1330) Line 286 + 0xf bytes  C++
    ClearScriptV8-64.dll!V8ContextImpl::DisposeWeakHandle(v8::Isolate * pIsolate=0x000000001f348a70, v8::Persistent<v8::Object> * phObject=0x000000002110db38, void * pvV8ObjectCache=0x000000001b3a1328)  Line 1208 + 0x20 bytes   C++
    v8-x64.dll!v8::internal::GlobalHandles::Node::PostGarbageCollectionProcessing(v8::internal::Isolate * isolate=0x000000001f348a70)  Line 269 + 0x17 bytes    C++
    v8-x64.dll!v8::internal::GlobalHandles::PostGarbageCollectionProcessing(v8::internal::GarbageCollector collector=MARK_COMPACTOR, v8::internal::GCTracer * tracer=0x000000002110dd90)  Line 672 + 0x1d bytes C++
    v8-x64.dll!v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector collector=MARK_COMPACTOR, v8::internal::GCTracer * tracer=0x000000002110dd90)  Line 1023 + 0x27 bytes    C++
    v8-x64.dll!v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace space=OLD_POINTER_SPACE, v8::internal::GarbageCollector collector=MARK_COMPACTOR, const char * gc_reason=0x000007fee368bab8, const char * collector_reason=0x000007fee3637288)  Line 687 + 0x19 bytes   C++
    v8-x64.dll!v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace space=OLD_POINTER_SPACE, const char * gc_reason=0x000007fee368bab8)  Line 545   C++
    v8-x64.dll!v8::internal::Heap::CollectAllGarbage(int flags=0, const char * gc_reason=0x000007fee368bab8)  Line 595  C++
    v8-x64.dll!v8::internal::Debug::CreateScriptCache()  Line 2537  C++
    v8-x64.dll!v8::internal::Debug::GetLoadedScripts()  Line 2580   C++
    v8-x64.dll!v8::internal::__RT_impl_Runtime_DebugGetLoadedScripts(v8::internal::Arguments args={...}, v8::internal::Isolate * isolate=0x000000001f348a70)  Line 12720    C++
    v8-x64.dll!v8::internal::Runtime_DebugGetLoadedScripts(int args_length=0, v8::internal::Object * * args_object=0x000000002110e198, v8::internal::Isolate * isolate=0x000000001f348a70)  Line 12712 + 0x4d bytes C++
Offending code :
        /// <summary>
        /// Long-lasting function
        /// </summary>
        private void WaitFunctionBadTime()
        {
            Thread.Sleep(10000);
        }

        /// <summary>
        /// Bad time for attaching the debugger
        /// </summary>
        [Test]
        public virtual void Test_Attach_Debugger_Bad_Time()
        {
            using (var engine = new V8ScriptEngine("Crash test debug script", V8ScriptEngineFlags.EnableDebugging, 9222))
            {
                engine.AddHostObject("waitfunction", new Action(() => WaitFunctionBadTime()));

                engine.Execute("Document0.js", @"function fun0() {}");
                engine.Execute("Document1.js", @"waitfunction();");
                engine.Execute("Document2.js", @"function stuff() {}
stuff();
");
            }
        }
I am a bit puzzled by the second dll-loading in another AppDomain ; is this really supposed to be a normal behaviour ?
Coordinator
Sep 20, 2013 at 5:50 PM
Edited Sep 20, 2013 at 5:50 PM
Hi julien_b,

Now that's an odd one. Thanks for reporting it!
Coordinator
Sep 20, 2013 at 6:24 PM
By the way, to help us reproduce this issue, can you tell us which version of ClearScript you're using, and whether you're using the default (tested) version of V8? Thanks!
Sep 21, 2013 at 6:42 PM
I am using clearscript 5.3.7 with default V8 version.

V8 and ClearScript are compiled in Debug mode (I had just cryptic crashes in Release mode so I tried with every dll in debug mode, and i hit the AppDomain issue every time).


From: ClearScript

By the way, to help us reproduce this issue, can you tell us which version of ClearScript you're using, and whether you're using the default (tested) version of V8? Thanks!

Coordinator
Sep 21, 2013 at 7:44 PM
Thanks julien_b. When you get a chance, please try the update we posted today.
Sep 23, 2013 at 9:06 AM
Update tested. It solves the "AppDomain crash" and the exit deadlock.

Now it deadlocks somewhere else :-)

(nb: I set a breakpoint with "debugger" instruction)
If I happen to step over (F6) or step into (F5) on the last instruction it deadlocks, leaving 8 threads with a similar stack waiting for a critical section owned by main runner thread.
They are waiting there :
void V8IsolateImpl::DispatchDebugMessages()
{
    if (++m_DebugMessageDispatchCount == 1)
    {
        auto wrThis = CreateWeakRef();
        Concurrency::create_task([wrThis]
        {
            auto spIsolate = wrThis.GetTarget();
            if (!spIsolate.IsEmpty())
            {
                auto pIsolateImpl = static_cast<V8IsolateImpl*>(spIsolate.GetRawPtr());
                pIsolateImpl->ProcessDebugMessages();
            }
        });
    }
}
with the following stack :
    ntdll.dll!NtWaitForSingleObject()  + 0xa bytes  
    ntdll.dll!RtlpWaitOnCriticalSection()  + 0xe8 bytes 
    ntdll.dll!RtlEnterCriticalSection()  - 0x4b75 bytes 
    v8-x64.dll!v8::internal::Win32Mutex::Lock()  Line 1723  C++
    v8-x64.dll!v8::internal::ThreadManager::Lock()  Line 218    C++
    v8-x64.dll!v8::Locker::Initialize(v8::Isolate * isolate=0x000000001f31e270)  Line 62    C++
    ClearScriptV8-64.dll!v8::Locker::Locker(v8::Isolate * isolate=0x000000001f31e270)  Line 5286 + 0x1d bytes   C++
    ClearScriptV8-64.dll!V8IsolateImpl::Scope::Scope(V8IsolateImpl * pIsolateImpl=0x000000001b0acd90)  Line 91 + 0x2d bytes C++
    ClearScriptV8-64.dll!V8IsolateImpl::ProcessDebugMessages()  Line 342 + 0x21 bytes   C++
>   ClearScriptV8-64.dll!V8IsolateImpl::DispatchDebugMessages()  Line 334   C++
    ClearScriptV8-64.dll!<lambda_dc9c8265224356d9d3c15d6ef2cbfd6f>::operator()()  Line 197 + 0xa bytes  C++
    ClearScriptV8-64.dll!std::_Callable_obj<<lambda_dc9c8265224356d9d3c15d6ef2cbfd6f>,0>::_ApplyX<void>()  Line 431 + 0x1b bytes    C++
    ClearScriptV8-64.dll!std::_Func_impl<std::_Callable_obj<<lambda_dc9c8265224356d9d3c15d6ef2cbfd6f>,0>,std::allocator<std::_Func_class<void,std::_Nil,std::_Nil,std::_Nil,std::_Nil,std::_Nil,std::_Nil,std::_Nil> >,void,std::_Nil,std::_Nil,std::_Nil,std::_Nil,std::_Nil,std::_Nil,std::_Nil>::_Do_call()  Line 239 + 0x1a bytes   C++
    ClearScriptV8-64.dll!std::_Func_class<void,std::_Nil,std::_Nil,std::_Nil,std::_Nil,std::_Nil,std::_Nil,std::_Nil>::operator()()  Line 514 + 0x33 bytes  C++
    ClearScriptV8-64.dll!<lambda_470e7c06e579e69f9c157af05ae21bf7>::operator()()  Line 2173 + 0x16 bytes    C++
    ClearScriptV8-64.dll!std::_Callable_obj<<lambda_470e7c06e579e69f9c157af05ae21bf7>,0>::_ApplyX<unsigned char>()  Line 431 + 0x1b bytes   C++
    ClearScriptV8-64.dll!std::_Func_impl<std::_Callable_obj<<lambda_470e7c06e579e69f9c157af05ae21bf7>,0>,std::allocator<std::_Func_class<unsigned char,std::_Nil,std::_Nil,std::_Nil,std::_Nil,std::_Nil,std::_Nil,std::_Nil> >,unsigned char,std::_Nil,std::_Nil,std::_Nil,std::_Nil,std::_Nil,std::_Nil,std::_Nil>::_Do_call()  Line 239 + 0x1a bytes C++
    ClearScriptV8-64.dll!std::_Func_class<unsigned char,std::_Nil,std::_Nil,std::_Nil,std::_Nil,std::_Nil,std::_Nil,std::_Nil>::operator()()  Line 514 + 0x33 bytes C++
    ClearScriptV8-64.dll!Concurrency::task<unsigned char>::_InitialTaskHandle<void,<lambda_dc9c8265224356d9d3c15d6ef2cbfd6f>,Concurrency::details::_TypeSelectorNoAsync>::_Init(Concurrency::details::_TypeSelectorNoAsync __formal={...})  Line 3191 + 0x5a bytes  C++
    ClearScriptV8-64.dll!Concurrency::task<unsigned char>::_InitialTaskHandle<void,<lambda_dc9c8265224356d9d3c15d6ef2cbfd6f>,Concurrency::details::_TypeSelectorNoAsync>::_Perform()  Line 3182 C++
    ClearScriptV8-64.dll!Concurrency::details::_PPLTaskHandle<unsigned char,Concurrency::task<unsigned char>::_InitialTaskHandle<void,<lambda_dc9c8265224356d9d3c15d6ef2cbfd6f>,Concurrency::details::_TypeSelectorNoAsync>,Concurrency::details::_UnrealizedChore>::operator()()  Line 1212 + 0xa bytes    C++
    ClearScriptV8-64.dll!Concurrency::details::_UnrealizedChore::_InvokeBridge<Concurrency::details::_PPLTaskHandle<unsigned char,Concurrency::task<unsigned char>::_InitialTaskHandle<void,<lambda_dc9c8265224356d9d3c15d6ef2cbfd6f>,Concurrency::details::_TypeSelectorNoAsync>,Concurrency::details::_UnrealizedChore> >(Concurrency::details::_PPLTaskHandle<unsigned char,Concurrency::task<unsigned char>::_InitialTaskHandle<void,<lambda_dc9c8265224356d9d3c15d6ef2cbfd6f>,Concurrency::details::_TypeSelectorNoAsync>,Concurrency::details::_UnrealizedChore> * _PChore=0x00000000007aa628)  Line 4467 C++
    msvcr110d.dll!Concurrency::details::_UnrealizedChore::_UnstructuredChoreWrapper(Concurrency::details::_UnrealizedChore * pChore=0x00000000007aa628)  Line 293 + 0x13 bytes  C++
    msvcr110d.dll!Concurrency::details::_UnrealizedChore::_Invoke()  Line 4423  C++
    msvcr110d.dll!Concurrency::details::WorkItem::Invoke()  Line 172    C++
    msvcr110d.dll!Concurrency::details::InternalContextBase::ExecuteChoreInline(Concurrency::details::WorkItem * pWork=0x000000001c55f870)  Line 1605   C++
    msvcr110d.dll!Concurrency::details::InternalContextBase::Dispatch(Concurrency::DispatchState * pDispatchState=0x000000001c55f908)  Line 1719    C++
    msvcr110d.dll!Concurrency::details::FreeThreadProxy::Dispatch()  Line 197   C++
    msvcr110d.dll!Concurrency::details::ThreadProxy::ThreadProxyMain(void * lpParameter=0x000000001b1eb070)  Line 171   C++
The main runner thread is hitting a breakpoint (probably automatic with F5/F6).. and waiting for a semaphore :
    ntdll.dll!NtWaitForSingleObject()  + 0xa bytes  
    KernelBase.dll!WaitForSingleObjectEx()  + 0x9c bytes    
>   v8-x64.dll!v8::internal::Win32Semaphore::Wait()  Line 1767  C++
    v8-x64.dll!v8::internal::Debugger::NotifyMessageHandler(v8::DebugEvent event=Break, v8::internal::Handle<v8::internal::JSObject> exec_state={...}, v8::internal::Handle<v8::internal::JSObject> event_data={...}, bool auto_continue=false)  Line 3166  C++
    v8-x64.dll!v8::internal::Debugger::ProcessDebugEvent(v8::DebugEvent event=Break, v8::internal::Handle<v8::internal::JSObject> event_data={...}, bool auto_continue=false)  Line 2974    C++
    v8-x64.dll!v8::internal::Debugger::OnDebugBreak(v8::internal::Handle<v8::internal::Object> break_points_hit={...}, bool auto_continue=false)  Line 2816 + 0x30 bytes    C++
    v8-x64.dll!v8::internal::Debug::Break(v8::internal::Arguments args={...})  Line 1024    C++
    v8-x64.dll!v8::internal::__RT_impl_Debug_Break(v8::internal::Arguments args={...}, v8::internal::Isolate * isolate=0x000000001f31e270)  Line 1094   C++
    v8-x64.dll!v8::internal::Debug_Break(int args_length=0x00000000, v8::internal::Object * * args_object=0x000000001fe4a8b8, v8::internal::Isolate * isolate=0x000000001f31e270)  Line 1092 + 0x4d bytes   C++
    [Managed to Native Transition]  
    ClearScriptV8-64.dll!Microsoft::ClearScript::V8::V8ContextProxyImpl::Execute(System::String^ gcDocumentName = "Document1.js", System::String^ gcCode = "debugger;
var a = 0;
var b = 1;

// whatever
function stuff() {
    var i = 111;
    for (i = 0; i < 10; i++)
    {
        a++;
    }
    fun0();
}

stuff();
", bool discard = false) Line 143 + 0xfe bytes  C++
    ClearScript.dll!Microsoft.ClearScript.V8.V8ScriptEngine.Execute.AnonymousMethod__11() Line 633 + 0x68 bytes C#
    ClearScript.dll!Microsoft.ClearScript.ScriptEngine.ScriptInvoke<object>(System.Func<object> func = {Method = Cannot evaluate expression because a native frame is on top of the call stack.}) Line 793 + 0xf bytes  C#
    ClearScript.dll!Microsoft.ClearScript.V8.V8ScriptEngine.BaseScriptInvoke<object>(System.Func<object> func = {Method = Cannot evaluate expression because a native frame is on top of the call stack.}) Line 408 + 0x4e bytes    C#
    ClearScript.dll!Microsoft.ClearScript.V8.V8ScriptEngine.ScriptInvoke<object>.AnonymousMethod__17() Line 677 + 0x7d bytes    C#
    ClearScriptV8-64.dll!Microsoft::ClearScript::V8::`anonymous namespace'::InvokeAction(void* pvActionRef = 0x000000001FE4B708) Line 75    C++
    [Native to Managed Transition]  
    ClearScriptV8-64.dll!V8ContextImpl::CallWithLock(void (void *)* pCallback=0x000007fee3e17b18, void * pvArg=0x000000001fe4b708)  Line 264 + 0xf bytes    C++
    [Managed to Native Transition]  
    ClearScriptV8-64.dll!Microsoft::ClearScript::V8::V8ContextProxyImpl::InvokeWithLock(System::Action^ gcAction = 0x0000000002be9d48) Line 101 + 0x5c bytes    C++
    ClearScript.dll!Microsoft.ClearScript.V8.V8ScriptEngine.ScriptInvoke<object>(System.Func<object> func = {Method = Cannot evaluate expression because a native frame is on top of the call stack.}) Line 677 + 0x5e bytes    C#
    ClearScript.dll!Microsoft.ClearScript.V8.V8ScriptEngine.Execute(string documentName = "Document1.js", string code = "\r\ndebugger;\r\nvar a = 0;\r\nvar b = 1;\r\n\r\n// whatever\r\nfunction stuff() {\r\n    var i = 111;\r\n    for (i = 0; i < 10; i++)\r\n    {\r\n        a++;\r\n    }\r\n    fun0();\r\n}\r\n\r\nstuff();\r\n", bool evaluate = false, bool discard = false) Line 615 + 0x72 bytes  C#
    ClearScript.dll!Microsoft.ClearScript.ScriptEngine.Execute(string documentName = "Document1.js", bool discard = false, string code = "\r\ndebugger;\r\nvar a = 0;\r\nvar b = 1;\r\n\r\n// whatever\r\nfunction stuff() {\r\n    var i = 111;\r\n    for (i = 0; i < 10; i++)\r\n    {\r\n        a++;\r\n    }\r\n    fun0();\r\n}\r\n\r\nstuff();\r\n") Line 483 + 0x2b bytes  C#
    ClearScript.dll!Microsoft.ClearScript.ScriptEngine.Execute(string documentName = "Document1.js", string code = "\r\ndebugger;\r\nvar a = 0;\r\nvar b = 1;\r\n\r\n// whatever\r\nfunction stuff() {\r\n    var i = 111;\r\n    for (i = 0; i < 10; i++)\r\n    {\r\n        a++;\r\n    }\r\n    fun0();\r\n}\r\n\r\nstuff();\r\n") Line 462 + 0x17 bytes    C#
    Xxxx.Javascript.Test.dll!Xxxx.Javascript.Test.TestJs.Test_Attach_Debugger() Line 728 + 0x2a bytes   C#
    [Native to Managed Transition]  
    nunit.core.dll!NUnit.Core.Reflect.InvokeMethod(...)
Offending code is shown in the stack but it is not specific to that code. (added a "debugger" instruction first to easily get into the debugger).
Coordinator
Sep 23, 2013 at 12:37 PM
Edited Sep 23, 2013 at 1:24 PM
Hi julien_b,

Unfortunately what you're seeing now is the effect of a V8 bug. Currently, if you "Step Over" the final line of a script, the V8 runtime and the Eclipse-based debugger get out of sync; the debugger believes the application is now running, but the V8 runtime encounters an internal error, reports it to the debugger, and remains stopped. Until this is fixed, you must use "Resume" to continue past the final line of a script.

Thanks for testing our fix! We'll incorporate it into ClearScript's next release. And thanks again for reporting that bug; it was a very subtle one that could cause both deadlocks and memory corruption.

Cheers!
Sep 23, 2013 at 2:50 PM
Ok so the bug is completely fixed. Thanks !