Jolokia improvments

[ccamel]

Hi everybody,

We successfully use Jolokia to publish the managed beans of our services. Since the protocol used is based on HTTP and embraces the REST philosophy, I think it would be great to provide a description of published resources in the form of a WADL document. Particularly in response to a request of type OPTION. What do you think ?

I wonder also if you plan to offer others representations than json such as xml or html ? I understand that you are looking to make the library as small as possible but  these kind of additional (and probably optional) features would be greatly appreciated. At least, I think so.

Chris.

[roland]

Hi Chris,

Currently, there are no plans to design a WADL (or even to make the Jolokia protocol really REST-like). This doesn’t mean that we reject the idea to play a more gentle role in the SOA World. No, it is simply so that the focus is currently somewhere else.

Maybe it is worth to have a short look on the history of Jolokia. It started in 2009 as part of jmx4perl as  a result of the urgent need for coupling Nagios to JEE servers. The first agent was about 23k large albeit already rather powerful. As you have correctly observed, focus is (and was) on a small footprint and ease of installation. From this, Jolokia as well as it is protocol grow up with quite some unique features. So the protocol has been ‘historically grown’ to fit current needs.

Describing the current Jolokia protocol with a WADL descriptor is probably not of much use. As you might note, Jolokia supports HTTP GET and POST in parallel to serve the same kind of request, with POST requests being quite a bit more powerful. So, Jolokia doesn’t follow the REST philosophy of translating HTTP verbs into CRUD operations (GET for ‘read’, PUT for ‘update’, POST for ‘create’,...). For the GET request a WADL description would make sense somehow, but for POST, the description would be rather limited since it is not much more than posting one or more requests within the POST body and receiving one or more responses in the HTTP response. Both, requests and responses, are formatted as JSON, for which there is no official ‘schema’ yet (but there is http://json-schema.org/ which could be supported, too). However, contributions are always welcome and if you (or someone else) comes up with a useful WADL for the current protocol (or for a slightly changed protocol), I happily would integrate it into the agent so that this documents gets served for an OPTIONS request (or even a GET) request. Or if you want to roll out your own protocol, we could discuss about the proper abstraction within jolokia-core in order to plugin different protocol stacks (shouldn't be too hard).

And there is WS-Management as well. This full blown DMTF Standard is also well worth a consideration as an alternative to the current protocol.

But all of these would add to Jolokia’s footprint without adding much business value for most users. That’s the reason why tackling this will be postponed for some time and maybe in a future (post 1.0) version will result in multiple variants of agents for different purposes. No promises here, though.

Then, what’s next on the Jolokia roadmap ?

For 0.90, which will be released this weekend, the focus lies on a polished protocol (number are now treated as JSON numbers, not as strings) and extra hooks for the security mechanism.

For 1.0, which is expected in late summer this year, mostly a clean up and further harding is done. (So there might be some minor protocol changes between 0.90 and 1.0).

Next (or even for 1.0) we plan to work on a Jolokia-Dashboard, which will result in an extended agent containing a full blown Javascript application (hopefully not that large) for accessing MBean information and live graphing some metrics.

And we are listening to feature requests, too. Please use http://jolokia.idea.informer.com/ for adding your ideas and let the bubble up ;-)

[ccamel]

Hi Roland !

roland wrote

Currently, there are no plans to design a WADL (or even to make the Jolokia protocol really REST-like). This doesn’t mean that we reject the idea to play a more gentle role in the SOA World. No, it is simply so that the focus is currently somewhere else.

Ok. In fact, if I say it, this is because for all our applications that publish REST interfaces, we propose a resource for describing resources. As founder library for REST protocol, we use Jersey in our applications. Jersey automatically supports the methods HEAD and OPTIONS for all the declared resources, and manages well content negotiation to provide the best available representation. It's something you already know.

For our needs, WADL is interesting both for machine interfaces and human interfaces for the discovery of resources available (and how to access them). The HTML representation of the WADL document is suitable for users while the XML (or JSON) form is more appropriate for a machine.

Of course it makes sense only if the offered resources fully adhere to the REST philosophy. From what you say, it seems that this is not the case with Jolokia.

roland wrote

Describing the current Jolokia protocol with a WADL descriptor is probably not of much use. As you might note, Jolokia supports HTTP GET and POST in parallel to serve the same kind of request, with POST requests being quite a bit more powerful. So, Jolokia doesn’t follow the REST philosophy of translating HTTP verbs into CRUD operations (GET for ‘read’, PUT for ‘update’, POST for ‘create’,...).

Yes, you're right. I started to develop a resource monitoring before using Jolokia which tried to adhere to the REST principles:

- The GET method on the monitors resource returns the list of registred ObjectNames. Parameters can be use to express criteria: instanceOf, queryNames.
- The GET method on the monitors/{name} resource returns the description of the managed bean. Query parameters allows to switch content details on/off: attributes, operations, constructors...
- The GET method on the monitors/{name}/{attribute} returns the value of the attribute (or method)
- The POST method on the monitors/{name}/{method} allows to call a method with arguments.

For each verb, several representations are available: xml, json, and even (thanks to DynamicJasper) html, rdf, txt, csv, pdf, ms-excel, opendocument which is nice (from our users point of view)

But as this project meets our needs, I prefer to use a protocol supported by an open source project rather than developing a proprietary interface especially as this project provides us with a client and a bridge to Nagios.

roland wrote

Or if you want to roll out your own protocol, we could discuss about the proper abstraction within jolokia-core in order to plugin different protocol stacks (shouldn't be too hard).

What would be great is the ability to add other representations, but in a totally pluggable way and in different (and separated) maven artifacts. Thus, these representations will be optional and will not affect the users who do not need.

Do you think it is possible to make changes to make this feasible? (I can put this wish to http://jolokia.idea.informer.com/)

roland wrote

And there is WS-Management as well. This full blown DMTF Standard is also well worth a consideration as an alternative to the current protocol.

Interesting...

roland wrote

That’s the reason why tackling this will be postponed for some time and maybe in a future (post 1.0) version will result in multiple variants of agents for different purposes.

Yep.

roland wrote

Next (or even for 1.0) we plan to work on a Jolokia-Dashboard, which will result in an extended agent containing a full blown Javascript application (hopefully not that large) for accessing MBean information and live graphing some metrics.

Ah, that would be great !!! :)

Chris

[roland]

Hi Chris,

For our needs, WADL is interesting both for machine interfaces and human interfaces for the discovery of resources available (and how to access them). The HTML representation of the WADL document is suitable for users while the XML (or JSON) form is more appropriate for a machine.

Of course it makes sense only if the offered resources fully adhere to the REST philosophy. From what you say, it seems that this is not the case with Jolokia.

Agreed, having some sort of meta description of a service (might it be WADL or WSDL) is a good thing. Maybe a WADL (along with a JSON-Schema document for the data transfered) might be possible even for non 'pure' REST clients. I will have a second look at this soon.  

Yes, you're right. I started to develop a resource monitoring before using Jolokia which tried to adhere to the REST principles:

- The GET method on the monitors resource returns the list of registred ObjectNames. Parameters can be use to express criteria: instanceOf, queryNames.
- The GET method on the monitors/{name} resource returns the description of the managed bean. Query parameters allows to switch content details on/off: attributes, operations, constructors...
- The GET method on the monitors/{name}/{attribute} returns the value of the attribute (or method)
- The POST method on the monitors/{name}/{method} allows to call a method with arguments.

Maybe some remarks about my experiences with GET requests are ok at this point:

* I had really a hard time with ObjectNames used in the pathinfo part of a servlet request. Especially when the ObjectNames uses / as part of their properties. Simply URL quoting doesn't work here, since certain servlet container will do the unquoting *before* the request is given to the servlet because of security constraints. Therefore I introduced an escaping mechanism, which is confessedly quite clumsy. I will probably refactor this sort of escaping before the 1.0 release. In retrospective, it would have been better to concentrate on query parameters (and in fact the Agent can use a query parameter 'p' as alternative for the whole path).

* Proxy requests need the target server specified within the request. This is not that easy with pure GET requests and since JSR-160 credentials can be given here, too, this poses also an additional security risk (even when talking over SSL). Maybe one could use some HTTP-Headers for this purpose instead of the POST solution I've chosen for Jolokia

* Bulk request however are not possible with GET at all. There can be any number of Jolokia requests put into a single HTTP POST request, even mixed request types (read, write, exec, list, …). So at least for Bulk Requests I don't see a solution in REST land.

But as this project meets our needs, I prefer to use a protocol supported by an open source project rather than developing a proprietary interface especially as this project provides us with a client and a bridge to Nagios.

If you use Nagios, I really recommend to have a look at check_jmx4perl from the jmx4perl distribution. I takes full advantage of bulk request (resulting in so called 'multi checks') and is quite feature rich.

What would be great is the ability to add other representations, but in a totally pluggable way and in different (and separated) maven artifacts. Thus, these representations will be optional and will not affect the users who do not need.

Do you think it is possible to make changes to make this feasible?

I just had a close look at Jolokia's Sonar metric data and it should be really easy to extract the major functionality of Jolokia without referring to HTTP at all. The entry point is

org.jolokia.backend.BackendManager.handleRequest()

 which takes some JmxRequest object and returns a JSON answer. A different protocol implementation the could take its input, creates appropriate JmxRequest objects before feeding the BackendManager and then returns the obtains JSON Map into the appropriate result (XML or whatever, or even feeds POJOS with JAXB annotations for automatic response documentation generation or whatever ;-)

I will put this sort of abstraction into a 0.9x release, maybe even already 0.91 (the one after 0.90).

Thanks you for sharing your ideas and if you like to contribute some sort of real REST stack, the door is open ;-) (just fork Jolokia on GitHub)

I will notify you as soon as have splitted up jolokia-core in the two artifacts described above.

[roland]

Hi Chris,

I setup a branch at github (https://github.com/rhuss/jolokia/tree/custom-prot) where I extracted the
Jolokia protocol stack into an extra module into agent/jolokia/protocol/classic. It is only a
thin layer over the core, where the entry point is org.jolokia.backend.BackendManager.

From its JavaDoc:

Backendmanager for dispatching to various backends based on a given JmxRequest. This is the entry class for protocol stack handler, which take a Jolokia request in some format, creates JmxRequest objects from the input, calls this backend manager and returns its answer in an appropriate format.

A client of this class needs to provide the following input for the constructor:

* A configuration described in a map with ConfigKey enumeration values as keys and strings a values. ConfigKey described all possible value which can influence the operation of this backend manager. The client is responsible to extract request options (ConfigKey.isRequestConfig() == true) and pass them in via this Map.

* An implementation of a LogHandler used for logging within this BackendManager

* An optional Restrictor implementation for restricting access. If given, this manager will thrown an SecurityException if the request doesn't pass the restrictor. For restriction on the IP address and the HTTP method used the client of this class is responsible itself, since the backend manager is agnostic to the transport protocol. See PolicyRestrictor for an implementation of a restrictor which uses a policy file.

The single entry point of this backend manager is handleRequest(JmxRequest) which takes a prepared JmxRequest and returns a typeless JSONObject which can be directly used as return value for JSON based protocol stacks. A usage example for this class can be found in HttpHandler in the module jolokia-protocol-classic

I think, that's a good starting point for a custom stack implementation. You can expect this change to be included for 0.91.

I really would like to see some other protocol stacks beside the 'classic' one, so you are invited to dig into it and go for a 'real' REST client. Any code contribution and participation is highly appreciated ;-) And general feedback as well, or course.