Memory consumption

Mar 23, 2014 at 10:15 AM
Edited Mar 23, 2014 at 10:15 AM
Hello,

I'm using clearscripts V8 facilities to process datasets in an ETL application. For each table I create a V8Runtime, for each row a V8Engine. For testing purposes I hand over no HostObjects right now, the Javascript-Code does nothing, and I don't read something back to the .NET side. After each script processing I dispose the V8Engine and trigger a CollectGarbage on the V8Runtime. After that I output the HeapInfo.

The HeapInfo suggests, that the memory-consumption of the v8 runtime remains more or less constant. But using Sysinternals Process-Explorer reveals, that the process working set as well as private bytes grow constantly to serveral GBs. When I'm short circuiting the part of my code which uses clearscript and therefore don't interact with V8 the memory-consumption of my .NET app is perfectly constant.

Has anyone experienced something similar and give me a hint to common pitfalls?
Thank you and andvance and best regards

Joachim
Coordinator
Mar 23, 2014 at 12:24 PM
Greetings, Joachim!

We run several tests before every release to check for memory leaks. For example, we run the SunSpider JavaScript benchmark in a loop for several hours. That test exposes some host objects and collects results, so it would seem to be a little more comprehensive than your current usage, yet we haven't encountered any runaway memory consumption.

Please give us more information about your application. For example, what ClearScript version are you using? What are you passing to the V8Runtime constructor and V8Runtime.CreateScriptEngine()? You mentioned that your script code currently does nothing, but what exactly does it look like? How many runtimes and engines are active at once?

If you could distill your application to a small code sample that exhibits the problem, that would be very helpful.

Thank you!
Mar 23, 2014 at 1:06 PM
Wow, reply on sundays. Thank you.
I don't assume a bug in clearscript, this is why I posted in the discussions forum not on the issue list.

I uploaded the project to bitbucket: https://bitbucket.org/jrosskopf/dbchangetrackingeventproducer.

The entry point to the script-processing is done in V8DbChangeToCypherProjector::ToCypherQueries(). I tried out there not to supply the "unitOfWork" to an empty version of "Application.js" script.

If you would rather like a stripped down version, I could try to do that. But sometimes IMHO the bigger picture is helpful.

Regards
Joachim
Coordinator
Mar 24, 2014 at 11:37 AM
Hi Joachim,

Thanks for posting your project. It looks like you're employing quite a sophisticated mix of technologies - exactly the sort of ClearScript usage we're hoping to enable.

We're taking a look, but in the meantime it would definitely be very helpful to have a small code sample that reproduces your symptoms. A test program with minimal external dependencies would be perfect.

Cheers!
Mar 27, 2014 at 7:58 AM
Hey,

I tried to reproduce with a simple combination of DataFlow and Clearscript. But I was not able to reproduce the memory leaking behavoir.
So I will try to lock down the problem any further.

On wednesday someone also reported a memory leak and you proposed a fix. Could that be somehow connected? Will you provide a nuget update with this patch included anytime soon, or is it better roll our own build?

Regards

Joachim
Coordinator
Mar 27, 2014 at 4:12 PM
Hi Joachim,

On wednesday someone also reported a memory leak and you proposed a fix. Could that be somehow connected?

To reproduce that memory leak the application has to (a) repeatedly instantiate and dispose script engines, and (b) invoke host methods from script code. You mentioned above that your issue can be reproduced without exposing host objects at all, so it appears to be unrelated.

Will you provide a nuget update with this patch included anytime soon, or is it better roll our own build?

We don't provide ClearScript binaries; the NuGet packages were all developed externally. However, we've tried to make building ClearScript as easy as possible. We recommend doing so, if only for the ability to integrate updates more quickly.

Good luck!
Mar 31, 2014 at 6:40 PM
Hello,

just to close this thread, I was able to find a solution, which does not produce a memory leak:
Instead of using AddHostObject, I used ScriptEngines Script property to hand over data to the script. In the script I return the data back and now use Evaluate with discard: true.

Thank you for your help!

Regards

Joachim