Reflective XML-RPC
Dynamically invoke XML-based Remote Procedure Call
Summary
Java reflection offers a simple but effective way of hiding some of the complexity of remote procedure calls with XML-RPC (XML-based Remote Procedure Call). In this article, Stephan Maier shows how to wrap XML-RPC calls to a remote interface using the gadgets from the Reflection kit: TheProxy, theArray, andBeanInfoclasses. The article will also discuss various ramifications of the approach and the use of reflective methods in RMI (Remote Method Invocation). (3,800 words; February 7, 2005)
By Stephan Maier
ML-based Remote Procedure Call (XML-RPC) receives occasional attention as a simple protocol for remote procedure calls. It is straightforward to use, and easily available implementations such as Apache XML-RPC facilitate the protocol's use.
If your application is small or uses a limited number of remote procedures, you might prefer not to formally define the names of remote procedures and their signatures, but instead use XML-RPC in a straightforward way. Yet, if your application grows and the number of remote interfaces increases, you might find that the necessary conventions—remote methods and data objects—must be somehow fixed. In this article, I show how Java provides all you need to define remote interfaces and access remote methods: procedures and their signatures can be defined via Java interfaces, and remote procedure calls with XML-RPC can be wrapped such that both sides of a communication channel see only interfaces and suitable data objects.
This article also shows that when given Java interfaces describing the remote procedures and datastructures conforming to the JavaBeans specification, you can use the power of Java reflection as incorporated into the Reflection and JavaBeans packages to invoke remote methods transparently and convert between the data types of XML-RPC and Java with surprising ease.
Hiding complexity is good practice in itself. Needless to say, not all complexity can and should be hidden. With respect to distributed computing, this point has been famously made by Jim Waldo et al. in "A Note on Distributed Computing" (Sun Microsystems, November 1994). The framework presented here does not intend to hide the complexity of distributed computing, but it promises to reduce the pains involved in calling a remote procedure. For simplicity, I discuss only concurrent remote procedure calls and leave the asynchronous case to the zealous reader.
XML-RPC can be viewed as an oversimplification of RPC via SOAP. And by extension, the simple framework I discuss here must be regarded as a simplistic version of a SOAP engine, such as Axis. This article's main purpose is educational: I wish to show how reflection is employed to build a simple XML-RPC engine on top of existing XML-RPC frameworks. This may help you understand the inner workings of similar but vastly more complex engines for other protocols or how to apply reflection to solve different problems. A simple RPC engine can be used where a SOAP engine is clearly not feasible, such as with small applications that are not exposed via a Web server and where other forms of middleware are unavailable. Roy Miller's "XML-RPC in Java Programming" (developerWorks, January 2004) explains a useful example.
In this article, we use the Apache implementation of XML-RPC (Apache XML-RPC) to set up our framework. You do not need to know XML-RPC, nor do you need to understand the Apache XML-RPC framework, even though a basic understanding will help you appreciate what follows. This article focuses on the framework's precise inner workings, but does not make use of the protocol's details.
Avoiding conventions
Occasionally, I prefer unconventional programming. Having said this, I must immediately assure you that I am no iconoclast and do not reject good programming habits; quite the contrary. The word unconventional here means that I like to avoid conventions expressed in terms of strings scattered throughout the code that could also be defined via a programmatic API. Consider the following piece of code:
Listing 1. Invoking a remote procedure call
Vector paras = new Vector();
paras.add("Herbert");
Object result = client.execute("app.PersonHome.getName", paras);
Listing 1 illustrates how a remote procedure might be called using the Apache XML-RPC implementation. Observe that we need to know both the name of the procedure and the parameters we are allowed to pass to the method. We must also know the object type returned to us by the remote procedure call. Unless you have the implementation class available to check whether you have all the names (app.PersonHome and getName) and parameters right, you will need to look up these names and signatures, usually in some text file or some constant interface (an interface that provides constants for all required names). A suitably placed Javadoc might also be used. Observe that this sort of convention is rather error-prone because errors will show up only at runtime, not at compile time.
Now, in contrast, consider the following piece of code:
Listing 2. Invoking a remote procedure call
Person person = ((PersonHome)Invocator.getProxy(PersonHome.class)).getPerson("Herbert");
Here, we call a static method getProxy() on the class Invocator to retrieve an implementation of the interface PersonHome. On this interface, we can call the method getPerson() and, as a result, obtain a Person object.
Listing 2's code is much more economical than the code in Listing 1. In Listing 2, we can use a method defined on an interface, which neatly defines the available methods, their signatures, and the return types all in one place. Type-safety comes along free of charge, and the code is more readable because it is freed from redundant constructs such as the Vector class.
Furthermore, if you are using a sufficiently powerful IDE, code completion will list all available methods on PersonHome together with their signatures. Thus, we get IDE programming support on top of a type-safe remote method call.
I must admit that we cannot do without conventions. The one convention we must keep (unless we are prepared to accept considerable overhead and complications) is the assumption that all data objects conform to the JavaBeans specification. Simply stated, this means that object properties are exposed via getter/setter method pairs. This assumption's importance will become clear when I talk about converting XML-RPC datastructures into Java objects.
The demand for all data objects to be JavaBeans is a convention far superior to the conventions used in a straightforward XML-RPC application because it is a general convention. It is also a convention natural for all Java programmers. Towards the end of the article, I discuss XML-RPC's limitations and suggest other useful conventions that can help you live with those limitations.
The following sections walk you through an implementation of the Invocator class and a suitable version of a local server that provides the other end of our framework's communication channel.
Implementing Invocations
Let's first look at the method that provides an interface's implementation:
Listing 3. Creating the proxy
public static Object getProxy(Class ifType) {
if (!ifType.isInterface()) {
throw new AssertionError("Type must be an interface");
}
return Proxy.newProxyInstance(Invocator.class.getClassLoader(),
new Class[]{ifType}, new XMLRPCInvocationHandler(ifType));
}
The magic is hidden in a simple call to the method Proxy.newProxyInstance(). The class Proxy has been part of the Java Reflection package since Java 1.3. Via its method newProxyInstance(), a collection of interfaces can be implemented dynamically. Of course, the created proxy object does not know how to handle method invocations. Thus, it must pass invocations to a suitable handler—a task for the implementation of the java.lang.reflect.InvocationHandler interface. Here, I have chosen to call this implementation XMLRPCInvocationHandler. The InvocationHandler interface defines a single method, as shown in Listing 4.
Listing 4. InvocationHandler
public interface InvocationHandler {
public Object invoke(Object proxy, Method method, Object[] args) throws Throwable;
}
When a method is invoked on a proxy instance, the proxy passes that method and its parameters to the handler's invoke() method, while simultaneously identifying itself. Let's now look at our handler's implementation:
Listing 5. InvocationHandler
private static class XMLRPCInvocationHandler implements InvocationHandler {
private Class type;
public XMLRPCInvocationHandler(Class ifType) {
this.type = ifType;
}
public Object invoke(Object proxy, Method method, Object[] args) throws Throwable {
XmlRpcClient client = getClient(); // Get a reference to the client
Vector paras = null; // This will hold the list of parameters
if (args != null){
paras = new Vector();
for (int i = 0; i < args.length; i++) {
paras.add(ValueConverter.convertFromType(args[i]));
}
}
else{
paras = new Vector(); // The vector holding the parameters must not be null
}
Class retType = method.getReturnType();
Object ret = client.execute(type.getName() + '.' + method.getName(), paras);
return ValueConverter.convertToType(ret, retType);
}
}
On creation, an instance of XMLRPCInvocationHandler is given the class that defines the remote interface. We use this class only to get the remote interface's name, which, together with the method name available on method invocation, is part of the remote request. Observe that the remote method invocation is thus totally dynamic: we need neither invoke methods on a stub class nor require any knowledge from outside the interface.
The client is obtained from the method getClient():
Listing 6. Getting the client
protected static XmlRpcClient getClient() throws MalformedURLException {
return new XmlRpcClient("localhost", 8080);
}
Here, we are able to use Apache XML-RPC to get a client that handles the remote call for us. Observe that we return a client without consideration of the interface on which the method has been invoked. Needless to say, we could add considerable flexibility by allowing different service endpoints that depend on the interface.
The more important code for our present purposes is represented by the static methods invoked on the class ValueConverter. It is in these methods where reflection does its magic. We look at that code in the following section.
Converting from XML-RPC to Java and back
This section explains the core of our XML-RPC framework. The framework needs to do two things: It needs to convert a Java object into a datastructure understood by XML-RPC, and it needs to perform the reverse process of converting an XML-RPC datastructure into a Java object.
I start by showing how to convert a Java object into a datastructure understood by XML-RPC:
Listing 7. Java to XML-RPC
public static Object convertFromType(Object obj) throws IllegalArgumentException,
IllegalAccessException, InvocationTargetException, IntrospectionException {
if (obj == null) {
return null;
}
Class type = obj.getClass();
if (type.equals(Integer.class)
|| type.equals(Double.class)
|| type.equals(Boolean.class)
|| type.equals(String.class)
|| type.equals(Date.class)) {
return obj;
else if (type.isArray() && type.getComponentType().equals(byte.class)) {
return obj;
}
else if (type.isArray()) {
int length = Array.getLength(obj);
Vector res = new Vector();
for (int i = 0; i < length; i++) {
res.add(convertFromType(Array.get(obj, i)));
}
return res;
}
else {
Hashtable res = new Hashtable();
BeanInfo info = Introspector.getBeanInfo(type, Object.class);
PropertyDescriptor[] props = info.getPropertyDescriptors();
for (int i = 0; i < props.length; i++) {
String propName = props[i].getName();
Object value = null;
value = convertFromType(props[i].getReadMethod().invoke(obj, null));
if (value != null) res.put(propName, value);
}
return res;
}
}
To convert a Java object into a datastructure understood by XML-RPC, we must consider five cases, which are illustrated in the listing above:
- Null: If the object we need to convert is null, we just return null.
- Primitive type: If the object is of one of the primitive types (or their wrapping types)—int, double, Boolean, string, or date—then we can return the object itself, as XML-RPC understands these primitive types.
- base64: If the object is a byte array, it is understood to represent an instance of the base64 type. Again, we may simply return the array itself.
- Array: If the object is an array but not a byte array, we can use the utility class
Array, which comes with the Java Reflection package to first find the length of the array. We then use this length to loop over the array and, again, using theArrayutility, access the individual fields. Each array item is passed to theValueConverter, and the result is inserted into a vector. This vector represents the array to Apache XML-RPC. - Complex types: If the object is none of the above, we can assume it is a JavaBean, a basic assumption fundamental to the entire construction and the one convention we agreed on at the outset. We insert its attributes into a hashtable. To access the attributes, we use the introspective power of the JavaBeans framework: we use the utility class
Introspectorto get the bean information that comes encapsulated in aBeanInfoobject. In particular, we can loop over the bean's properties by accessing the array ofPropertyDescriptorobjects. From such a property descriptor, we retrieve the name of the property that will be the key into the hashtable. We get this key's value, i.e., the property value, by using the read method on the property descriptor.
Observe how easy it is to extract information from a bean with the JavaBeans framework. We need to know nothing about the type we want to convert, only that it is a bean. This assumption then is a necessary prerequisite for our framework to function faultlessly.
Let's now turn to the opposite transformation from XML-RPC structures to Java objects:
Listing 8. XML-RPC to Java
Implementing service handling Ramifications Limitations Controlling serialization Adding value Other languages Removing or replacing the XML-RPC implementation Remote Method Invocation Interface definition language Summary About the author
public static Object convertToType(Object object, Class type) throws IllegalArgumentException,
IllegalAccessException, InvocationTargetException, IntrospectionException, InstantiationException {
if (type.equals(int.class)
|| type.equals(double.class)
|| type.equals(boolean.class)
|| type.equals(String.class)
|| type.equals(Date.class)) {
return object;
}
else if (type.isArray() && type.getComponentType().equals(byte.class)) {
return object;
}
else if (type.isArray()) {
int length = ((Vector) object).size();
Class compType = type.getComponentType();
Object res = Array.newInstance(compType, length);
for (int i = 0; i < length; i++) {
Object value = ((Vector) object).get(i);
Array.set(res, i, convertToType(value, compType));
}
return res;
}
else {
Object res = type.newInstance();
BeanInfo info = Introspector.getBeanInfo(type, Object.class);
PropertyDescriptor[] props = info.getPropertyDescriptors();
for (int i = 0; i < props.length; i++) {
String propName = props[i].getName();
if (((Hashtable) object).containsKey(propName)) {
Class propType = props[i].getPropertyType();
props[i].getWriteMethod().
invoke(res, new Object[]
{ convertToType(((Hashtable) object).get(propName), propType)});
}
}
return res;
}
}
Converting to a Java type requires more knowledge than just the value that we wish to convert—we must also know which type to convert it to. This explains the second parameter in Listing 8's convertToType() method. Given the type's knowledge, we use the introspective power of Java to transform XML-RPC data types into Java types. The following list shows how conversion is completed for the various data types:
getComponentType(). Next, we use the utility class Array to create a new array with the given component type. Then we loop over the array and, using the Array utility again, set the individual fields, using the ValueConverter to get the right values for each array item. Observe that the datastructure we expect from the XML-RPC framework in the case of an array is a Vector.
Introspector to find the bean's property descriptors and, using the property descriptor, set the actual properties by accessing the write() method. Note that the framework hands us the properties stored in a hashtable. Of course, as the property's type may be complex, we must use the ValueConverter to obtain the correct Java object.
Armed with this understanding of data conversion, we can now look at how service handling is implemented.
Having explained how a remote service is invoked and what is involved in transforming between XML-RPC and Java, I now sketch the last piece of the puzzle: how to handle a request at a service endpoint.
Here is the complete code of the simple server I have implemented for this article's purpose:
Listing 9. Server
public class Server {
private WebServer webserver = null;
public void start() {
webserver = new WebServer(8080);
webserver.addHandler
(PersonHome.class.getName(),
new Handler(PersonHome.class,
new PersonHomeImpl()));
webserver.setParanoid(false);
webserver.start();
}
public void stop() {
webserver.shutdown();
webserver = null;
}
private static class Handler implements XmlRpcHandler {
private Object instance;
private Class type;
public Handler(Class ifType, Object impl) {
if (!ifType.isInterface()) {
throw new AssertionError("Type must be an interface");
}
if (!ifType.isAssignableFrom(impl.getClass())) {
throw new AssertionError("Handler must implement interface");
}
this.type = ifType;
this.instance = impl;
}
public Object execute(String method, Vector arguments) throws Exception {
String mName = method.substring(method.lastIndexOf('.') + 1);
Method[] methods = type.getMethods();
for (int i = 0; i < methods.length; i++) {
if (methods[i].getName().equals(mName)){
try {
Object[] args = new Object[arguments.size()];
for (int j = 0; j < args.length; j++) {
args[j] = ValueConverter.convertToType
(arguments.get(j), methods[i].getParameterTypes()[j]);
}
return ValueConverter.convertFromType(methods[i].invoke(instance,args));
}
catch (Exception e) {
if (e.getCause() instanceof XmlRpcException){
throw (XmlRpcException)e.getCause();
}
else{
throw new XmlRpcException(-1, e.getMessage());
}
}
}
}
throw new NoSuchMethodException(mName);
}
}
public static void main(String[] args){
Server server = new Server();
System.out.println("Starting server...");
server.start();
try {
Thread.sleep(30000);
}
catch (InterruptedException e) {
e.printStackTrace();
}
System.out.println("Stopping server...");
server.stop();
}
}
The key player is the class WebServer, which is from the Apache XML-RPC package. The code in boldface shows our main requirements: we must register a service handler. Such a handler is defined via a simple interface XmlRpcHandler, which, just like the proxy mechanism's InvocationHandler interface, has a method to which method invocation is delegated. Here, it is called execute(), with an implementation the same in spirit as InvocationHandler's. The most notable difference is that we need to register a handler that holds both the interface and its implementation. In the InvocationHandler implementation above, we do not need to provide an implementation of the service interface (in the form of a stub). In the server, however, we need to define which code is responsible for handling incoming requests. Finally, observe that we use the usual approach when invoking the service method by looping through the interface's methods to find the right method. Here, we cannot rely on standard introspection into JavaBeans because service methods are not likely to be mere setters and getters.
In this section, I briefly discuss a few ramifications that arise from the preceding discussion. I look at limitations of both the XML-RPC protocol and this article's framework, but I also consider opportunities introduced by this approach.
XML-RPC is a simple protocol and obviously cannot implement programmatic APIs for remote procedure calls that feature all aspects of an object-oriented system. Notably, such an API will not support the following:
Serialization is a process that happens behind the scenes. In particular, the framework proposed in this article finds properties to serialize automatically. Sometimes, however, you might wish to prevent certain properties from being serialized.
Suppose a Person object has a reference to various Address objects that differ in type. In particular, one of those addresses might be the mailing address, while others are significant in other contexts. You might wish to enhance your Person class with the Person.getMailingAddress() method, which returns the mailing address. Standard introspection will then see a new property, namely mailingAddress, and this property will be written during serialization with the entire list of addresses. In the best case, a corresponding Person.setMailingAddress() method will be written such that, regardless of the addresses' serialization order, the deserialization process will return an object identical to the one serialized. Of course, your methods should be written such that the serialization order does not matter, but even if you did write your methods correctly, somebody at the other end (who might be using a different language) might be unaware of your thinking, increasing the potential for problems. In any case, you would accept the overhead of serializing the mailing address twice.
But there is help. The Introspector can be told not to use reflection when looking for a class's properties and instead use given information. This information is found in a BeanInfo class, which must be named MyClassBeanInfo, if your class is called MyClass. This BeanInfo class must be either in the same package as MyClass or in one of the packages listed in BeanInfo's search path. This path can be set in the Introspector itself. When providing a BeanInfo class, you will usually just wish to offer the properties as follows:
Listing 10. BeanInfo Example 1
public class MyClassBeanInfo extends SimpleBeanInfo {
public PropertyDescriptor[] getPropertyDescriptors() {
try {
BeanInfo superInfo = Introspector.getBeanInfo(MyClass.class.getSuperclass());
List list = new ArrayList();
for (int i = 0; i < superInfo.getPropertyDescriptors().length; i++) {
list.add(superInfo.getPropertyDescriptors()[i]);
}
//
list.add(new PropertyDescriptor("myProperty", MyClass.class));
//
return (PropertyDescriptor[])list.toArray(new PropertyDescriptor[list.size()]);
} catch (IntrospectionException e) {
return null;
}
}
}
The method getPropertyDescriptors() must return the properties represented by property descriptors. First, add the properties of your class's superclass, then add the properties you wish to expose to your class, as shown in the bold section.
There is a serious drawback here: the above proposal implies a lot of hard coding, which you ideally want to avoid. More precisely, adding all the properties that should be serialized is probably more work than listing those that should not be considered. Of course, one approach is to use the Introspector to first get all properties via reflection by calling Introspector.getBeanInfo(MyClass.class, Introspector.IGNORE_ALL_BEANINFO). Then you apply a filter to the result you return. This approach might look like this:
Listing 11. BeanInfo Example 2
public class MyClassBeanInfo extends SimpleBeanInfo {
public PropertyDescriptor[] getPropertyDescriptors() {
try {
BeanInfo infoByReflection = Introspector.getBeanInfo(MyClass.class,
Introspector.IGNORE_ALL_BEANINFO); PropetyDescriptor allProperies =
infoByReflection.getPropertyDescriptors();
return filter(allProperies);
} catch (IntrospectionException e) {
return null;
}
}
protected PropertyDescriptor[] filter(PropertyDescriptor[] props){
// Remove properties which must not be exposed
}
}
A better way is to build a framework on some form of interface definition language (IDL), which allows you to generate beans and extend the properties and methods by hand if you need to. The generator will be responsible for providing BeanInfo classes that filter out just the properties defined in the IDL. Continue reading for an example of such a language.
As we have hidden the actual transport mechanism, it is easy to add information to messages sent and received. Suppose we are required to pass session information with each remote method invocation. This information could be added in the invocator and the handler as a first argument (wrapping all necessary information into a suitable bean). At the other end, this information would be removed from the vector of parameters and handled separately from the method invocation. Extending the code available from Resources in this direction may be a useful way to play around with the framework.
Weaknesses can be considered strengths, provided you look at them correctly. XML-RPC's simplicity leads to the limitations described above. However, XML-RPC implementations are now available for many languages such as Ruby, Python, or functional languages such as Haskell. Not all of these languages support inheritance as understood in object-oriented languages, and not all allow method overloading. Some languages, such as Haskell, have flexible list types, which, from a Java perspective, fall somewhere between arrays and lists. Hence, the inherent limitations of XML-RPC make it a suitable candidate for communication across language boundaries.
When XML-RPC is chosen for bridging the gap between Java and some other language, you can still use the framework presented here, but you will be able to use it only for the Java side of the communication channel. However, you could extend the framework to cover other languages. For instance, you could rewrite the framework in another language and then add support for the transformation of Java interfaces and data objects into corresponding objects in the other language. Another approach, which I have already hinted at above, is to write a compiler that turns a suitable form of IDL into code for the various languages, Java among them. I give an example of this approach below.
Needless to say, such approaches for extending this article's framework will be more involved than the framework itself, but they will work along similar lines.
A productive system might prefer to avoid the use of an intermediate XML-RPC framework and instead transfer the XML data of XML-RPC straight into suitable objects. You might consider abstracting calls to the XML-RPC framework by hiding them behind suitable interfaces that can be implemented for various XML-RPC implementations. As I have seen no need to do so in our work, I have not implemented this functionality. Again, you are invited to adapt the framework as suits your needs.
With J2SE 1.5, RMI will also use the proxy mechanism under the hood. Using the rmic compiler to generate stub classes is no longer necessary (unless you wish to interoperate with older versions). Thus, if a generated stub class cannot be loaded, the remote object's stub will be an instance of java.lang.reflect.Proxy.
An obvious way to remove some of the pains involved in observing bean conventions and the various restrictions imposed by XML-RPC, which I have discussed above, is to avoid writing interfaces and beans and instead generate them with a suitable IDL. Such a language might look as follows:
Listing 12. IDL
module partner;
exception NoPartnerException < 123 : "No partner found" >;
struct Partner {
int id;
string name;
int age;
date birthday;
};
interface PartnerHome {
Partner getPartner(int id) throws NoPartnerException;
Partner[] findPartner(string name, date bday) throws NoPartnerException;
};
Writing a parser and code generator based on such an IDL offers an easy way to facilitate cross-language communication.
In this article, I have shown how the power of Java reflection can be used to transparently wrap the complexity of remote method invocation via XML-RPC. I have placed particular emphasis on often overlooked mechanisms that have been incorporated in the Proxy, Array, and Introspector classes. Based on these utilities, a simple middleware framework for remote method invocation has been constructed that can be readily adapted to various needs.
Stephan Maier holds a Ph.D. in mathematics and has been involved in software development for more than five years. He has been a teacher and coach of state-of-the-art technology for most of his career. Apart from programming, he enjoys singing and sports. Currently, he is working on a compiler that turns a simple form of IDL into suitable versions of datastructures and remote interfaces for languages such as Java, Ruby, or Python, where the underlying protocol for remote calls is XML-RPC.
Resources
http://www-106.ibm.com/developerworks/xml/library/ws-tip-roundtrip2.html
http://www-106.ibm.com/developerworks/library/j-xmlrpc.html
http://www-106.ibm.com/developerworks/library/ws-tip-coding.html?ca=dnx-420
http://research.sun.com/techrep/1994/smli_tr-94-29.pdf
http://www.xmlrpc.com
http://ws.apache.org/xmlrpc
http://www.javaworld.com/javaworld/jw-11-2001/jw-1102-codegen.html
http://www.javaworld.com/channel_content/jw-rmi-index.shtml
http://www.javaworld.com/channel_content/jw-xml-index.shtml
http://www.javaworld.com/channel_content/jw-javabeans-index.shtml
