I've just installed jolokia's .war version 0.80 on a test websphere application server v18.104.22.168 and a WebSphere admin complains about "memory leaks" and MBeans that are not unregistered properly when application stops.
I tried filling a bug report in github, but I'm a bit confused, so let's report this here.
Thanks for the report. In fact, the Jolokia agent uses two MBeans for configuration and history management. However, it does it very best to unregister them during shutdown or undeployment (HttpServlet.destroy())
I will investigate this issue. Well, I'm not a dedicated websphere expert, where on the Admin Console is is possible to detect Memory Leaks and unregistered MBeans ?
Sorry, I probably mis-understood you. It is probably not a memory detection feature of the admin console what you meant, but a 'real' administrator ;-)
Of course, I would be curious to know a bit more about the technique with which the Memory-Leak has been detected and kind of Memory Leak it is supposed to be (PermGen, Heap, ....)
* With Websphere 22.214.171.124.0 I could deploy, undeploy and deploy again the agent without problem. This indicates, that during undeployment the MBeans are properly unregistered, otherwise the second deployment would have failed since it registers the same MBeans again (on the same MBeanServer, which is a singleton for the JVM). This is true for start, stop, all kind of combinations.
* I used the new Tomcat 7 memory leak detector after several times of deployment and undeployment. Nothings shows up here, too. Also, we are using the agent in really large enviroments (500+ server), over a very long time. A memory leak didn't occur there, either.
So, I'm really curious about your findings, so that we can improve Jolokia even further.
Thanks for the pointer. I tried it also on Websphere 126.96.36.199.11 and I get a similar
result. However, there are some part of the report, which I don't buy.
I attached a screenshot of the result after deploying/undeploying jolokia
First of all it reports, that the MBean jolokia:type=Config still exists twice.
For the same MBeanServer (and Jolokia registers its MBean always on the same existing
MBeanServer), this cannot happen, since the name needs to be unique.
Also, as I said yesterday, Jolokia register the MBean with the same time each time it
starts. If such an MBean is already registered, Websphere would have croaked (I already saw
this sort of symptoms earlier).
Also, I wonder where the 'hanging' MemoryThread should come from. For sure, Jolokia
never obtained actively a reference to this Websphere thread.
So, my conclusion so far is: Either arit has some false detections or Websphere indeed has itself
a memory leak when dealing with custom MBeans.
However, I will try arit and Jolokia with different JEE Servers (jboss, weblogic) and will tell you the results.
Jolokia really doesn't something spectacular when it comes to MBean registration: It registers MBeans
on Servlet.init() and unregister them on Servlet.destroy(). Nothing uncommon.
Just even stranger, I just tried it with the jmx4perl agent. Look at
the screenshot, arit reports two MBeans registered twice each (with
the same name !). I can gurantee, that each MBean is registered
only once during startup.
In fact, it was a bug in WAS 7 which bites here. When registering an MBean via the MBeanRegistration-Interface mechanism (i.e. letting the MBean itself provide its name), then a NullPointerException is raised by WAS (that's the bug), but it already registered the MBean. Jolokia tries multiple MBeanServers for registering its own MBeans before giving up. So, after the Exception occured it assumes that the registration of the MBean has failed (which IMO is a fair assumption) and registers it with the second MBeanServer (in this case, the PlatformMBeanServer). Hence, there exists indeed 2 MBeans which identical names in the system (but for different MBeanServer). Arit was right.
The solution was to switch for WAS 7 to an explicite MBean registration with a given name. For WAS 6.1 I I still need the other indirect registration because only in this case, WAS doesn't add it server id as part of the MBean-Name (and I need a fixed MBean name so that I can predictably access this MBean).
In fact, WebSphere adds some IDs to the MBean name (process, node and cell name) so the Deployment Manager knows where to contact the MBean. So when registering, we must extract the name returned by WebSphere and use it when unregistering. (Note that I'm not an expert so I'm just reporting what I understood).