Strange IllegalStateException from Jetty in OSGi when CXF and Jolokia are installed

classic Classic list List threaded Threaded
4 messages Options
Ed Ed
Reply | Threaded
Open this post in threaded view
|

Strange IllegalStateException from Jetty in OSGi when CXF and Jolokia are installed

Hi Roland,

I'm not exactly sure who's issue this is, if it's a pax web/cxf/camel/jolokia issue, however, I thought I would first talk to you and see if I can understand how Jolokia is even involved here.

I have a karaf 3.0.3 container with camel/cxf/jolokia installed.  I have a camel route which is consuming a cxf soap webservice.

I can send messages to my webservice and it works just fine, client receives no errors.  However, in the log, for every message there are some pesky jetty warnings:

2015-07-01 14:04:58,738 | WARN  | tp1802860762-190 | Response             | 69 - org.eclipse.jetty.aggregate.jetty-all-server - 8.1.15.v20140411 | Committed before 401 null
2015-07-01 14:04:58,739 | WARN  | tp1802860762-190 | stractHttpConnection | 69 - org.eclipse.jetty.aggregate.jetty-all-server - 8.1.15.v20140411 | /cxf/NeustarEventListener
java.lang.IllegalStateException: Committed
        at org.eclipse.jetty.server.Response.resetBuffer(Response.java:1154)[69:org.eclipse.jetty.aggregate.jetty-all-server:8.1.15.v20140411]
        at org.eclipse.jetty.server.Response.sendError(Response.java:317)[69:org.eclipse.jetty.aggregate.jetty-all-server:8.1.15.v20140411]
        at org.eclipse.jetty.server.Response.sendError(Response.java:419)[69:org.eclipse.jetty.aggregate.jetty-all-server:8.1.15.v20140411]
        at javax.servlet.http.HttpServletResponseWrapper.sendError(HttpServletResponseWrapper.java:137)[63:org.apache.geronimo.specs.geronimo-servlet_3.0_spec:1.0.0]
        at org.jolokia.osgi.security.BasicAuthenticationHttpContext.handleSecurity(BasicAuthenticationHttpContext.java:49)[213:org.jolokia.osgi:1.3.1]
        at org.ops4j.pax.web.service.jetty.internal.HttpServiceServletHandler.doHandle(HttpServiceServletHandler.java:68)[78:org.ops4j.pax.web.pax-web-jetty:3.1.4]
        at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)[69:org.eclipse.jetty.aggregate.jetty-all-server:8.1.15.v20140411]
        at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557)[69:org.eclipse.jetty.aggregate.jetty-all-server:8.1.15.v20140411]
        at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)[69:org.eclipse.jetty.aggregate.jetty-all-server:8.1.15.v20140411]
        at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1086)[69:org.eclipse.jetty.aggregate.jetty-all-server:8.1.15.v20140411]
        at org.ops4j.pax.web.service.jetty.internal.HttpServiceContext.doHandle(HttpServiceContext.java:240)[78:org.ops4j.pax.web.pax-web-jetty:3.1.4]
        at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:429)[69:org.eclipse.jetty.aggregate.jetty-all-server:8.1.15.v20140411]
        at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)[69:org.eclipse.jetty.aggregate.jetty-all-server:8.1.15.v20140411]
        at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1020)[69:org.eclipse.jetty.aggregate.jetty-all-server:8.1.15.v20140411]
        at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)[69:org.eclipse.jetty.aggregate.jetty-all-server:8.1.15.v20140411]
        at org.ops4j.pax.web.service.jetty.internal.JettyServerHandlerCollection.handle(JettyServerHandlerCollection.java:93)[78:org.ops4j.pax.web.pax-web-jetty:3.1.4]
        at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)[69:org.eclipse.jetty.aggregate.jetty-all-server:8.1.15.v20140411]
        at org.eclipse.jetty.server.Server.handleAsync(Server.java:410)[69:org.eclipse.jetty.aggregate.jetty-all-server:8.1.15.v20140411]
        at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:519)[69:org.eclipse.jetty.aggregate.jetty-all-server:8.1.15.v20140411]
        at org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:982)[69:org.eclipse.jetty.aggregate.jetty-all-server:8.1.15.v20140411]
        at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:1043)[69:org.eclipse.jetty.aggregate.jetty-all-server:8.1.15.v20140411]
        at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:865)[69:org.eclipse.jetty.aggregate.jetty-all-server:8.1.15.v20140411]
        at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)[69:org.eclipse.jetty.aggregate.jetty-all-server:8.1.15.v20140411]
        at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82)[69:org.eclipse.jetty.aggregate.jetty-all-server:8.1.15.v20140411]
        at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:696)[69:org.eclipse.jetty.aggregate.jetty-all-server:8.1.15.v20140411]
        at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:53)[69:org.eclipse.jetty.aggregate.jetty-all-server:8.1.15.v20140411]
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)[69:org.eclipse.jetty.aggregate.jetty-all-server:8.1.15.v20140411]
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)[69:org.eclipse.jetty.aggregate.jetty-all-server:8.1.15.v20140411]
        at java.lang.Thread.run(Thread.java:745)[:1.8.0_45]

Oddly, nestled in that stack trace is a jolokia class, and i'm really having a hard time figuring out how it gets there.

Looking around the web, i found what appears to be someone else having this issue, but they didn't really address it, it looks like they just wrapped up jolokia to hide it:

https://tools.codice.org/jira/browse/DDF-451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

If I install jolokia without starting it (state=INSTALLED), things are all good, but as soon as I start it, the warnings appear, and stopping it does not remove the warnings.

It feels to me like something is happening when jolokia binds to the http service, but I really don't know what, and when I stop the bundle, and it unbinds, the problem remains.

The funny thing is none of the packages that Jolokia exports appear to be in use anywhere in the container, karaf tells me that nobody is importing any packages from the Jolokia bundle.

My best guess here, is maybe this a problem with pax?  when you register the service you are providing an HttpContext (in my case it's the BasicAuthenticationHttpContext, the same which shows up in the stack trace), and this seems to be the only way the framework would be introduced to this class.

Thoughts?

Ed

Ed Ed
Reply | Threaded
Open this post in threaded view
|

Re: Strange IllegalStateException from Jetty in OSGi when CXF and Jolokia are installed

I've been stepping through the pax-web code, and I have found something curious, going to start a discussion with those guys and see if they can explain, will follow up here
Ed Ed
Reply | Threaded
Open this post in threaded view
|

Re: Strange IllegalStateException from Jetty in OSGi when CXF and Jolokia are installed

An issue has been created for this with pax web: https://ops4j1.jira.com/browse/PAXWEB-863

roland roland
Reply | Threaded
Open this post in threaded view
|

Re: Strange IllegalStateException from Jetty in OSGi when CXF and Jolokia are installed

Thanks for finding out this issue. 'hope it will get fixed soon
... roland
... roland