Using a Java URLClassLoader in CFMX Can Cause a Memory Leak

This is a bug in ColdFusion that can cause memory leaks when using a
java.net.URLClassLoader
to load external jar files.  Thus, this can cause memory leaks in
JavaLoader,
Transfer
and any other system that uses this technology.  However there is a
workaround for the issue.

To explain the problem, first we need to look at some key issues with a
URLClassLoader.  URLClassLoaders are notorious for causing memory leaks,
because, for them to be garbage collected, all instances to themselves
and the classes that they have created
need to be garbage collectible.

This means that if you access a class from a URLClassLoader and hold it
somewhere in memory, then the URLClassLoader can
never be garbage collected.

This is exactly what happens with ColdFusion.

When ColdFusion does some introspection and resolution of ColdFusion code
against a Java object, somewhere, deep inside its hidden internals, it keeps a
strong reference to the Class object that refers to that Java object.  This
means that when the JVM comes along to garbage collect the instance of the
URLClassLoader, it can't do it, because ColdFusion has a reference to a class
that it loaded somewhere inside.

So, the memory leak only ever actually happens when an instance of a
URLClassLoader is no longer available to ColdFusion, as it is never then garbage
collected by the JVM.

How does this translate to using JavaLoader? Well, a perfect example of this is
where you put an instance of JavaLoader in the application scope, because
generally it is used as a singleton.  JavaLoader (and anything that
subsequently uses JavaLoader) has an instance of a URLClassLoader inside
it.  However, when the application scope times out, the JavaLoader CFC may
well be garbage collected, but the URLClassLoader isn't, which can cause a
memory leak.

To note however, in production systems the leak is minimised in situations like
this, as it is often very rare that the application scope will ever time out.

So what is the workaround for this issue? To note, I have been pushing at Adobe
to get a hotfix out for CFMX to resolve this issue, but we can definitely still
use this technique now, without having to worry about memory leaks. 

Essentially, the memory leak only happens when the URLClassLoader is no longer
available to CF, i.e. an application scope times out, or something similar – so
we just need to make sure that it never, ever, times out.  How can we do
that? why, put it in the Server scope of
course!

Since variables in the Server scope never time out, we don't need to worry about
the URLClassLoader (or JavaLoader) being lost and then recreated, as it always
exists.  As long as you put it in the Server scope under a key no one will
ever need to utilise (I like a hard coded UUID myself)!  Hence we beat the
memory leak monster!

I have just committed a fix for Transfer that automatically puts the JavaLoader
it uses into the Server scope, so even if your TransferFactory times out, the
JavaLoader never will, which means there is no leak, and the RC2 for 0.6.3 will
have this as well.

Hopefully Adobe will put out a hotfix for this issue, but until then, put your
JavaLoaders in the Server scope.

Leave a Comment

Comments

  • Steve | April 11, 2007

    Unfortunately I’ve encountered the memory leak after using JavaLoader.cfc to iText classes directly after cfdocument proved to be a toothless tiger with more brawn than brain.. so I’m very happy to see this post. But are you saying that you have already server scoped the JavaLoader within the cfc? Or should we scope an original call to JavaLoader (currently we call it in a couple of different places as needed)?

  • Mark | April 11, 2007

    I am saying that whenever you use JavaLoader, make sure you store instances of JavaLoader in the server scope so it can never time out.

  • denny | June 24, 2007

    So, if you don’t put the javaloader in some scope (like session or application) you are ok? I.e., if you just use it in a request, it’s destroyed after the request is done, right?

    Or should we be putting a single instance in the server scope no matter what?

  • Mark | June 24, 2007

    @Denny – the JavaLoader isn’t garbage collected properly, so if it’s not put in the server scope, it will cause a memory leak.

  • denny | June 25, 2007

    Ah, so putting it in the request would be the worst possible! Thanks!

    To show a further lack of comprehension skills, what does this mean for the classes you load via the classloader? Hmm… at this point, I realize I’m going to go check out transfer to see what you did there. :)

    Gracias, Mark!

  • denny | July 19, 2007

    Once you put the javaloader in the server scope, can you re-init it with different classes/jars? What’s the best way to do so, if you know?

  • Mark | July 19, 2007

    Denny,

    During development you can re-init it with different classes, but note, that this will cause a memory leak, which means you will need to reset your coldfusion server every so often as your server may run out of memory.

  • Scott Mebberson | December 17, 2007

    I was just reading more about managing memory with the JavaLoader. Because of concurrency issues, I need a new instance of my Java class for every ‘transaction’ I make with it. I want to ensure that these instances are collected by the JVM garbage collection routines, so I’m doing a .create().init() every time I need it and storing that in session scope (I’m calling against an instance of JavaLoader which is stored in server scope). I’ve had quick look and I think the JavaLoader internals only ever use on urlClassLoader, and as such shouldn’t interfere with my instances being Garbage collected. Am I using it properly?

  • Mark | December 17, 2007

    Scott,

    That will be fine.

    The memory leak only ever comes from re-initialising the JavaLoader, not any of its subsequent instances that it creates.

  • Rémy Roy | December 20, 2007

    Is this still a problem with Coldfusion 8?

  • Mark | December 21, 2007

    Remy,

    I believe that they have fixed the bug that caused the memory leak in ColdFusion 8.

    Easy way to tell… create a page that creates a JavaLoader with a JAR file… refresh a bunch of times. If your memory never comes down, you know there is a memory leak, and should put the JavaLoader in the server scope.

  • cfSearching | December 21, 2007

    I hate to sounds dense :) .. but maybe you can clear up some confusion. JavaLoader v0.5 appears to add a java.net.URLClassLoader instance into the server scope under the key A0608BEC-0AEB-B46A-0E1E1EC5F3CE7C9C. Is this related to the fix you mentioned for Transfer and does this effect the need to store the javaloader in the server scope?

  • Mark | December 21, 2007

    @cfSearching –
    JavaLoader needs it’s own URLClassLoader to do some things, so it keeps a copy of one in the server scope, so that it is not a memory leak.

  • cfSearching | December 21, 2007

    Okay. So the javaloader does still need to be stored in the server scope.

    Thanks!

  • Rémy Roy | December 21, 2007

    Hello,

    As Mark said, I did the test with Coldfusion 8 and it seems like it’s not leaking even when it’s not stored and reuse from the server scope. I watched the jrun.exe process with the task manager while refreshing a page 50 times that instanciated the javaloader component and created a java object. The memory comsuption did go up but it went down after a while with the garbage collection routines.

    Here are my development settings:
    JavaLoader: 0.5
    ColdFusion: 8,0,0,176276
    Edition: Developer
    Java VM Vendor: Sun Microsystems Inc.
    Java VM Version: 1.6.0_01-b06
    Operating System: Windows 2003

  • Victor Rodriguez | February 7, 2008

    Mark,

    Can you (or somebody else) please give an example of putting the URLClassLoader in the Server scope?

    Or, alternatively, please call me at 720-864-5098 or 720-840-4761.

    Thanks!

    Victor

  • Victor Rodriguez | February 7, 2008

    Does this problem only apply if we are using Transfer? Or is it that the problem can be resolved by using Transfer?

  • Mark | February 7, 2008

    Victor –
    If you are using Javaloader:
    server.myjavaloaderformyapp = createObject("component", "javaloader.JavaLoader").init(paths);

    If you are using Transfer, this is a non issue, as it puts it’s own copy of JavaLoader in the server scope.

  • Victor Rodriguez | February 7, 2008

    Is this Java code or CF code?

  • Mark | February 7, 2008

    Regular ol’ cfscript…

  • Bilal | April 23, 2008

    I am facing the following error while running javaloader.cfc
    I would appreciate any help in this regard..

    com.compoundtheory.classloader.NetworkClassLoader

    The error occurred in C:InetpubwwwrootjavaloaderJavaLoader.cfc: line 59

    57 :
    58 : //classLoader = createObject("java", "com.compoundtheory.classloader0.NetworkClassLoader").init();
    59 : networkClassLoaderClass = getServerURLClassLoader().loadClass("com.compoundtheory.classloader.NetworkClassLoader");
    60 :
    61 : networkClassLoaderProxy = createJavaProxy(networkClassLoaderClass);

  • Mark | April 24, 2008

    Bilal – Send me an email via the contact form with all the relevant details.

  • Chandrakanta Kar | October 30, 2008

    com.compoundtheory.classloader.NetworkClassLoader class is available when we will use on CF8 but on CF 7 it is not there, I think some java library required to loadClass().

  • Chandrakanta Kar | October 30, 2008

    Please, have a look on the below link which contains an example
    http://javaloader.riaforge.org/index.cfm?event=action.download

  • Nelle | July 20, 2009

    On our server it still leaks under CF8 8,0,1,196946 – JVM 1.6.0_04 (JavaLoader 0.5) (loading bunch of Axis 2 jars)

  • Mark | July 20, 2009

    @Neil,
    Please forward all issues you are having with Javaloader to the forum:
    http://javaloader.riaforge.org/forums/forums.cfm?conferenceid=43246943-D2A4-7D02-727904196AD5D759

    Along with a detailed description of what you are doing.

  • Ben Nadel | August 24, 2010

    Someone just pointed me to this post. Sorry if I am totally missing something in the blog post and comments but, if your application times out, what is preventing all of the variables from being garbage collected? Is it a matter of a circular reference?

    I would think that if the Application times out, there’d be nothing left to cause a reference to a class that would prevent the URL Class Loader from being garbage collected itself?

  • Mark Mandel | August 24, 2010

    Ben,

    The important distinction to make here is not the *variables* that need to be garbage collected, but the instances of java.lang.Class that come from a ClassLoader need to be garbage collected before a ClassLoader can be also garbage collected.

    If there are strong references to the instance of java.lang.Class that a ClassLoader creates (which ColdFusion does do with ClassLoaders), then it can’t be garbage collected.

    Does that make more sense?

  • Ben Nadel | August 24, 2010

    @Mark,

    Hmmm, I think I just need to know more about garbage collecting that I need to know before I can wrap my head around this. If both the class loader and the class instances are no longer being referenced by the app, I figured the GC would know how to clean it all up. But clearly, it’s way more complex than that :D

    No worries.

  • Mark Mandel | August 24, 2010

    Maybe the important piece to the puzzle is that *as soon as* you use a Java Object loaded by a classloader in ColdFusion, CF caches the java.lang.Class behind it (and quite likely a lot of the Method objects as well) behind the scenes to speed up reflection calls.

    It’s not a question of what your app is doing, it’s a question of what CF does to enable CF->Java communication.

    Does that make more sense?

  • Ben Nadel | August 24, 2010

    @Mark, Ahhhh, that makes more sense. It never occurred to me that CF would be stepping in there. I assumed it was doing all kinds of caching for things it loads in its own class loader; it never occurred to me that it would even have the ability to optimize on an external class loader. Thanks!