Thank you all for the Java flavored advent of 2012!

Thank you everybody!

This concludes the series of posts for this years Java Advent Calendar. Thank you everybody for writing, reading and commenting on the articles. Thanks to Transylvania JUG for proving the inspiration for this project and a big thank you to all the contributors:

Below you can find a calendar of all our articles. Until next December!

Sunday Monday Tuesday Wednesday Thursday Friday Saturday
1 (Re)Start me up!
2 Java Management Extensions 3 Multi-threading in Java Swing with SwingWorker 4 Java 7 – The Language Grows Up 5 JAX-RS Bean Validation Error Message Internationalization 6 Quercus on Google App Engine – 2.0 7 Hand and Finger Detection using JavaCV 8 Of Hacking Enums and Modifying “final static” Fields
9 Changes to String.substring in Java 7 10 Java EE 6 Web Profile. On the cloud. Easy. 11 Using Builder Pattern in JUnit tests 12 Escaping from the JVM heap for memory intensive applications 13 Java Sound – Make A Noise 14 Functional Java collections 15 AChartEngine – A Charting Library for Android Applications
16 Cleaning Up After Yourself – Java Style 17 Java – far sight look at JDK 8 18
NetRexx
Kawa
19 Java Runtime options 20 Ensuring the order of execution for tasks 21
JVM Classloaders
Pleading for Java
22 Weak, Weaker, Weakest
23 Waiting for the right moment 24 Java – The 2012 Review and Future Predictions

Update: Here is a nice advent calendar about numbers. See you in a year!

Java – The 2012 Review and Future Predictions

Hi all,
It’s a real privilege to close off the Java Advent calendar for 2012. It’s a wonderful initiative and I (like many of you) eagerly checked my feed every morning for the next addition. A big Thank You! has to go to +Attila-Mihaly Balazs and all of the other authors for rounding off 2012 in some style.
This post will focus on the events big and small that occurred in 2012 and also take a look at some future predictions for 2013. Some of the predictions will be honest guesses, others…. well lets just say that my Diabolical side will have taken over :-).
So without further adieu lets look at the year that was 2012 for Java…..

2012 – A Year in Review


2012 was a rocking year for Java, the JVM and the community. James Governer (RedMonk analyst) stated that “2012 was the dawning of a 2nd age for Java”.

Java enters the cloud (for real this time)


Java/JVM based cloud offerings became a serious reality in 2012 with a host of new PAAS and IAAS offerings. Cloudbees, JElastic, Heroku, Joyent, Oracle are just five of the large number of offerings out there now.

What does that mean for you as a developer? Well, it means lots of choice and the ability to try out this space very cheaply. I highly recommend that you try some of these providers out over the holidays (it takes minutes to set-up a free account) and see what all of the fuss is about.


Counter to this however is a lack of standardisation in this space and although JEE8 promises to change this (assuming the vendors get on board) – for the next few years you’ll need to be careful about being locked into a particular platform. If you’re a bit more serious about having agnostic services/code running on the various offerings then I can recommend looking at the jClouds API to assist you.

It’s fair to say that many of the offerings are still feeling their way in terms of getting the most out of the JVM. In particular multi-tenancy is an issue, as is Garbage Collection and performance on a virtualised environment.  Companies such as Waratek and jClarity (Disclaimer: I’m their CTO) now offer solutions  to alleviate those gaps.


The Java community thrives


The community continues to thrive despite many main stream tech media reports of “developers leaving the Java platform” or “Java is dead”. There are more Java User Groups (JUGs) than ever before, consisting of ~400,000 developers world wide. Notably, one of them, the London Java Community won several awards including the Duke’s Choice award and JCP Member of the Year (along with SouJava – the major Brazilian JUG).

The conference circuit is bursting at the seams with large, sold out in advance, world-class Java conferences such as JFokus, Devoxx and of course JavaOne. In addition to this the host of regional conferences that often pack in an audience of over 1000 people all continued to do well.

Oracle’s Java Magazine was launched and has grown to over 100,000 subscribers. Stalwarts like JaxEnter, Coderanch and the Javaposse continue to grow in audience sizes.

OpenJDK


Further OpenJDK reforms happened over 2012 and a new scorecard is now in place for the wider community to give feedback on governance, openness and transparency. 2012 also saw a record number of individuals and organisations joining OpenJDK. In particular, the port to the ARM processor and support for running Java on graphic cards (Project Sumatra) were highlights this year.

Java Community Process (JCP)


The Java Community Process (JCP), Java’s standards body also continued its revival with record numbers of new sign-ups and a hotly contested election. As well as dealing with the important business of trademarks, IP and licensing for Java, a re-focus on the technical aspects for Java Specification Requests (JSRs) occurred. In particular the new Adopt a JSR programme is being strongly supported by the JCP.


Java and the JVM


The JVM continues to improve rapidly through OpenJDK – the number of Java Enhancement Proposals (JEPs) going into Java 8 is enormous. Jigsaw dropping out was a disappointing but given the lack of broader vendor support and the vast amount of technical work required, it was the correct decision.

JEE / Spring


JEE7 is moving along nicely (and will be out soon), bringing Java developers a standard way to deal with the modern web (JSON, Web Sockets, etc). Of course many developers are already using the SpringSource suite of APIs but it’s good to see advancement in the underlying specs.

Rapid Web Development


Java/JVM based rapid web development frameworks are finally gaining the recognition they deserve. Frameworks like JBoss’s SEAM, Spring Roo, Grails, Play etc all give Java developers parity with the Rails and Django crowd.

Mechanical Sympathy


A major focus of 2012 was on Mechanical Sympathy (as coined by Martin Thompson in his blog). The tide has turned, and we now have to contend with having multi-core machines and virtualised O/S’s. Java developers have had to start thinking about how Java and the JVM interacts with the underlying platform and hardware.

Performance companies like jClarity are building tooling to help developers understand this complex space, but it certainly doesn’t hurt to get those hardware manuals off the shelf again!

2013 – Future predictions


It’s always fun to gaze into the crystal ball and here are my predictions for 2013!

Java 8 will get delivered on time


Java 8 with Nashorn, Lambda, plus a port to the ARM processor will open up loads of new opportunities for developers working on the leading edge of web and mobile tech. I anticipate rapid adoption of Java 8 (much faster than 7).

However, the lack of JVMs present on iOS and Android devices will continue to curtail adoption there.

Commercial Java in the cloud


2013 will be the year of commercial Java/JVM in the cloud – many of the kinks will get ironed out with regards to mutli-tenancy and memory management and a rich SAAS ecosystem will start to form.


The organisations that enable enterprises to get their in house Java apps out onto the cloud will be the big commercial winners.

We’ll also see some consolidation in this space as the larger vendors snap up smaller ones that have proven technology.

OpenJDK

OpenJDK will continue to truly open up with a public issue tracker based on JIRA, a distributed build farm available to developers and a far superior code review and patch system put in place.

Oracle, IBM and other major vendors have also backed initiatives to bring their in house test suites out into the open, donating them to the project for the good of all. 


JVM languages and polyglot

There will be a resurgence in Groovy thanks to its new static compilation capability and improved IDE tooling. Grails in particular will look like an even more attractive rapid development framework as it will offer decent performance for midrange web apps. 

Scala will continue to be hyped but will only be used successfully by small focused teams.  Clojure will continue to be popular for small niche areas.  Java will still outgrow them all in terms of real numbers and percentage growth.


A random prediction is that JRuby may well entice over Rails developers that are looking to take advantage of the JVM’s performance and scalability.



So that’s it from me, it was an amazing 2012 and I look forward to spending another year working with many of you and watching the rest making dreams into reality!

Cheers,
Martijn (@karianna – CTO of jClarity – aka “The Diabolical Developer”)

Meta: this post is part of the Java Advent Calendar and is licensed under the Creative Commons 3.0 Attribution license. If you like it, please spread the word by sharing, tweeting, FB, G+ and so on! Want to write for the blog? We are looking for contributors to fill all 24 slot and would love to have your contribution! Contact Attila Balazs to contribute!

Waiting for the right moment – in integration testing

When you have to test multi-threaded programs, there is always the need to wait until the system arrives at a particular state, at which point the test can verify that the proper state has been reached.

The usual way to do it is to insert a “probe” in the system which will signal a synchronization primitive (like a Semaphore) and the test waits until the semaphore gets signaled or a timeout passes. (Two things which you should never do – but are a frequent mistake – is to insert sleeps into your code – because they slow you down and are fragile – or to use the Object.wait method without looping around it – because you might get spurious wakeups which will result in spurious, hard to diagnose and very frustrating test failures).

This is all nice and good (although a little verbose – at least until the Java 8 lambdas arrive), but what if the second thread calls a third thread and doesn’t wait for it to finish, but in the test we want to wait for it? A concrete example would be: an integration test which verifies that a system composed out of a client which communicates trough a messaging middleware with a datagrid properly writes the data to the datagrid? Of course we will use a mock middleware and a mock datagrid, thus the startup/shutdown and processing will be very fast, but they would be still asynchronous (suppose that we can’t make it synchronous because the production one isn’t and the code is written such that it relies on this fact).

The situation is described visually in the sequence graph below: we have the test running on T0 and we would like for it to wait until the task on T3 has finished before it checks the state the system arrived to.

We can achieve this using a small modification to our execution framework (which probably is some kind of Executor). Given the following interface:


public interface ActivityCollector {
void before();
void after();
}

We would call before() at the moment a task is enqueued for execution and after() after it has executed (these will usually occur on different threads). If we now consider that before increments a counter and after decrements it, we can just wait for the counter to become zero (with proper synchronization) at which point we know that all the tasks were processed by our system. You can find an Executor which implements this here. In production you can of course use an implementation of the interface which does nothing, thus removing any performance overhead.

Now lets look at the interface which defines how we wait for the “processed” condition:

interface ActivityWatcher {
void await(long time, TimeUnit timeUnit);
}

Two personal design choices used here were: only provide a way to wait for a specific time and no longer (if the test takes too long that’s probably a performance regression one needs to take a look at) and to use unchecked exceptions to make testing code shorter.

A final feature would be to collect exceptions during the execution of the tasks and abort immediately if there is an exception somewhere rather than timing out. This means that we modify our interface as follows:

public interface ActivityCollector {
void before();
void after();
void collectException(Throwable t);
}

And the code wrapping the execution would be something like the following:

try {
command.run();
} catch (Throwable t) {
activityCollector.collectException(t);
throw t;
} finally {
activityCollector.after();
}

You can find an implementation of ActivityWatcher/ActivityCollector here (they are quite linked, thus the one class implementing them both). Happy testing!

A couple of caveats:

  • This requires some modification to your production code, so it might not be the best solution (for example you can try creating synchronous mocks of your subsystems and do testing that way).
  • This solution is not well suited for cases where Timers are involved because there will be times when “no tasks are waiting”, but in fact a task is waiting in a timer. You can work around this by using a custom timer which calls “before” when scheduling and “after” at the finish of the task.
  • The same issue can come up if you are using network communication for more authenticity (even if it is inside of the same process): there will be a moment when no tasks are scheduled because they are serialized in the OSs network buffer.
  • The ActivityCollector is a single point of synchronization. As such it might decrease performance and it might hide concurrency bugs. There are more complicated ways to implement it which avoids some of the synchronization overhead (like using a ConcurrentLinkedQueue), but you can’t eliminate it completely.

PS. This example is based on an IBM article I can’t seem to find (dear lazyweb: if somebody finds it, please leave a comment – before/after were called tick/tock in it) as well as work by my colleagues. My only role was to write it up and synthesize it.

Weak, Weaker, Weakest, Harnessing The Garbage Collector With Specialist References

When and when not to use specialist references in Java

Weak, Soft and Phantom references are dangerous and powerful. If they are used the wrong way they can destroy JVM performance; however, when used the correct way they can substantially enhance performance and program clarity.

Weak and Soft references are the more obvious of the three. They are pretty much the same thing actually! The idea is simply that they be used to access an object but will not prevent that object being reclaimed by the garbage collector:


Object y=new Object();
// y is a hard reference to the object
// and so that object cannot be reclaimed.

Obejct x=WeakReference<Object>(y);
// now x is a weak reference to the object
// (not to y - as y is just a variable).
// The object still cannot be reclaimed
// because y is still a hard reference to it.

y=null;
// now there is only a weak reference to
//the object, it is eligible for garbage collection.

if(x.get()==null){
System.out.println("The object has gone away");
}else{
System.out.println("The object is " + x.get().toString());
}

Have you spotted the deliberate mistake? It is an easy one to miss and it will probably not show up in unit testing. It is exactly the sort of issue which makes me say:

Only Use Weak/Soft References If You Absolutely Have To
And Probably Not Even Then.
When the JVM is under memory pressure it might reclaim the object between the first and second invocations of the get method in the weak reference. This will result in the program throwing a null pointer exception when the toString method is invoked on null. The correct form for the code is:


Object x=x.get();
// Now we have null xor a hard reference to
// the object
if(z==null){
System.out.println("The object has gone away");
}else{
System.out.println("The object is " + z.toString());
}

So they are mad, bad and dangerous to be with; why do we want them?

We have not fully touched on why they are really, really dangerous yet. To do that we need to see why we might want them and why we might need them. There are two common situations in which weak and soft references might seem like a good idea (we will look at the difference between soft and weak in a little bit). The first of these is in some form of RAM cache.

It works like this: We have some data, for example customer details, which is stored in a database. We keep looking it up and that is slow. What we can do is cache that data in RAM. However, eventually the RAM will fill up with names and addresses and the JVM throw an OutOfMemoryError. The solution is to store the names and addresses in objects which are only weakly reachable. Something like this:


ConcurrentHasMap>String,WeakReference>CustomerInfo<< cache=new ConcurrentHashMap><();
...
CustomerInfo currentCustomer=cache.get(customerName);
if(currentCustomer==null){
currentCustomer=reloadCachesEntry(customerName);
}

This innocent little pattern is quite capable of bringing a monster sized JVM to its knees. The pattern is using the JVM’s garbage collector to manage an in-memory cache. The garbage collector was never designed to do that. The pattern abuses the garbage collector by filling up the memory with weakly reachable objects which run the JVM out of heap space. When the JVM gets low in memory, it has to traverse all the reference, weak, soft and otherwise, in its heap and reclaim RAM. This is expensive and shows up as a processing cost. It is even worse on very big JVM instances with a lot of processor cores because the garbage collector may well end up having to perform a ‘stop the world’ full cycle and hence reduce performance down to single core levels!

I am not saying in memory cache technology is a bad idea. Oh no – it is a great idea. However, just throwing it against the garbage collector and not expecting trouble is a very poor design choice.

Weak vs Soft what is the difference? Well, there is much really. On some JVMs (the client hostspot JVM for example – but that might change at any time) weak reference are marked for preferential garbage collection. In other words, the garbage collector should make more effort to reclaim memory from the object graph to which they refer (and no soft or hard references refer) than for other memory. Soft references do not have this idea to them. However, this is just an optional idea on some JVMs and cannot be relied upon, and it is a bad idea anyway. I would suggest using either soft or weak references all the time and stick with it. Pick which ever you like the sound of. I prefer the name WeakReference, so tend to use that.

There is one other difference; an object which is referenced to by a soft reference and a weak reference, but not a hard reference, can have the situation where it can still be acquired from the .get() method of the weak reference but not that of the soft reference. The reverse is not possible not the other way around. Code that relies on this behaviour is probably wrong headed.

Good uses for weak references do exist. What weak references are great for it keeping track of objects which are being used else where. An example is from Sonic Field (an audio processing package). In this example, ‘slots’ in files contain audio data and are associated with objects in memory. This model does not use the weak references to refer to in-memory copies of the data. In memory objects use the slots. Weak references are used to allow the file management system to reuse slots.

The code using slots does not need (and should not need to) be concerned with the management of disk space. It is the concern of the file manager to do that. The file manager has weak references to the objects using the slots. When a new slot is requested, the file manager checks for any existing slots referred to via weak references which have been reclaimed (and hence return null from the get method). If it finds such a reference, it can reuse the slot.

Automatic notification of reclamation

Sometimes we might want to be told when a weak or soft (or the other sort – phantom) reference has been reclaimed. This can be done via the en-queuing system. We can do this using a reference queue:


WeakReference(T referent, ReferenceQueue<? super T> q)

We do something like this:


ReferenceQueue junkQ = new ReferenceQueue<>();
....
WeakReference<FileSlot> mySlot=new WeakReference<>(aSlot);
....
// In a different thread - make sure it is daemon!
WeakReference<FileSlot> isDead;
while(true){
isDead = junkQ.remove();
// Take some action based on the fact it is dead
// But - it might not be dead - see end of post :(
...
}

But, remember, by the time weak reference ends on the junkQ calling .get() on it will return null. If you will have to store information to allow what ever action you are interesting it to happen somewhere else (like a ConcurrentHashMap where the reference is the key),

So What Is A Phantom Reference?

Phantom references are the one sort which, when you need them, you really need them. But on the face of it, they seem utterly useless. You see, whenever you invoke .get() on a phantom reference, you always get null back. It is not possible to use a phantom reference to get to the object to which it refers – ever. Well – that is not quite true. We can achieve this via JNI sometimes but we should never do so.

Consider the situation where you allocate native memory in JNI associated with a Java object. This is the sort of model which the DirectBuffers in the noi package of the JDK use. It is something I have used repeatedly in large commercial projects.

So, how do we reclaim that native memory? In the case of file like systems, it is possible to say that the memory is not reclaimed until the file is closed. This places the responsibility of resource management on the shoulders of the programmer; which is exactly what the programmer expects for things like files. However, for lighter weight objects, we programmers do not like to have to think about resource management – the garbage collector is there to do it for us.

We could place code in a finalizer which calls into the JNI code to reclaim the memory. This is bad (as in lethal) because JVMs make almost guarantee that they will call finalizers. So, don’t do that! But, phantom references come to the rescue! First we need to understand ‘phantom reachable’: A phantom reference will only become enqueued if the thing to which it refers cannot be reach via any other sort of reference (hard, weak or soft). At this point the phantom reference can be enqueued. If the object had a finalizer, then it will either have been ignored or run; but it will not have ‘brought the object back to life’. Phantom reachable objects are ‘safe’ for JNI native code (or any other code) to reclaim resources against.

So our code with phantom references can look like this:


ReferenceQueue<FileSlot> junkQ = new ReferenceQueue<>();
....
Phantom<FileSlot> mySlot=new Phantom<>(aSlot);
....
// In a different thread - make sure it is daemon!
Phantom<FileSlot> isDead;
while(true){
isDead=junkQ.remove();
long handle=lookUpHandle(isDead);
cleanNativeMemory(handle);
}

In this pattern we keep a handle which the native code can use to find and reclaim resources in a structure (another hashmap probably) in Java. When we are absolutely sure that Java object cannot be brought back to life – it is phantom reachable (i.e. a ghost – we can then safely reclaim the native resource. If your code does other ‘naughty’ things using sun.misc.unsafe (for example) this trick might be of use as well. For a full example which uses this technique – check out this post.

One final thought about phantom references. It is technically possible to implement the same pattern as above using weak references. However, that is not the purpose of weak references and such a pattern would be abusing them. Phantom references makes an absolute guarantee that an object really is dead and so resource can be reclaimed. For just one example, it is theoretically possible for a weak reference to be enqueued and then the object be brought back to life by its finalizer because the finalization queue is running slower than the weak reference queue. This sort of edge case horror story cannot happen with phantom references.

There is one little problem, which is a weakness of the JVM design. That is that the JNI global weak reference type has an undefined relationship with phantom references. Some people suggest that you can use a global weak reference even to get to am object even when it is enqueued as a phantom reference. This is a quirk of one particular implementation of the JVM and should never be used.

By: Dr Alexander J Turner: I would like to thank Attila for contacting me and organising this interesting project. Feel free to pop over to my blog at Nerds-Central at any time 🙂

Meta: this post is part of the Java Advent Calendar and is licensed under the Creative Commons 3.0 Attribution license. If you like it, please spread the word by sharing, tweeting, FB, G+ and so on! Want to write for the blog? We are looking for contributors to fill all 24 slot and would love to have your contribution! Contact Attila Balazs to contribute!

Under the JVM hood – Classloaders

By Simon Maple, @sjmapleZeroTurnaround Technical Evangelist

Classloaders are a low level and often ignored aspect of the Java language among many developers. At ZeroTurnaround, our developers have had to live, breathe, eat, drink and almost get intimate with classloaders to produce the JRebel technology which interacts at a classloader level to provide live runtime class reloading, avoiding lengthy rebuilds/repackaging/redeploying cycles. Here are some of the things we’ve learnt around classloaders including some debugging tips which will hopefully save you time and potential headdesking in the future.

A classloader is just a plain java object

Yes, it’s nothing clever, well other than the system classloader in the JVM, a classloader is just a java object! It’s an abstract class, ClassLoader, which can be implemented by a class you create. Here is the API:

public abstract class ClassLoader {

public Class loadClass(String name);

protected Class defineClass(byte[] b);

public URL getResource(String name);

public Enumeration getResources(String name);

public ClassLoader getParent()

}

Looks pretty straightforward, right? Let’s take a look method by method. The central method is loadClass which just takes a String class name and returns you the actual Class object. This is the method which if you’ve used classloaders before is probably the most familiar as it’s the most used in day to day coding. defineClass is a final method in the JVM that takes a byte array from a file or a location on the network and produces the same outcome, a Class object.

A classloader can also find resources from a classpath. It works in a similar way to the loadClass method. There are a couple of methods, getResource and getResources, which return a URL or an Enumeration of URLs which point to the resource which represents the name passed as input to the method.

Every classloader has a parent; getParent returns the classloaders parent, which is not Java inheritance related, rather a linked list style connection. We will look into this in a little more depth later on.

Classloaders are lazy, so classes are only ever loaded when they are requested at runtime. Classes are loaded by the resource which invokes the class, so a class, at runtime, could be loaded by multiple classloaders depending on where they are referenced from and which classloader loaded the classes which referen… oops, I’ve gone cross-eyed! Let’s look at some code.

public class A {

public void doSmth() {

B b = new B();

b.doSmthElse();

}

}

Here we have class A calling the constructor of class B within the doSmth of it’s methods.  Under the covers this is what is happening

A.class.getClassLoader().loadClass(“B”);

The classloader which originally loaded class A is invoked to load the class B.

Classloaders are hierarchical, but like children, they don’t always ask their parents

Every classloader has a parent classloader. When a classloader is asked for a class, it will typically go straight to the parent classloader first calling loadClass which may in turn ask it’s parent and so on. If two classloaders with the same parent are asked to load the same class, it would only be done once, by the parent. It gets very troublesome when two classloaders load the same class separately, as this can cause problems which we’ll look at later.

When the JEE spec was designed, the web classloader was designed to work the opposite way – great. Let’s take a look at the figure below as our example.  
 


Module WAR1 has its own classloader and prefers to load classes itself rather than delegate to it’s parent, the classloader scoped by App1.ear. This means different WAR modules, like WAR1 and WAR2 cannot see each others classes. The App1.ear module has its own classloader and is parent to the WAR1 and WAR2 classloaders.  The App1.ear classloader is used by the WAR1 and WAR2 classloaders when they needs to delegate a request up the hierarchy i.e. a class is required outside of the WAR classloader scope. Effectively the WAR classes override the EAR classes where both exist. Finally the EAR classloader’s parent is the container classloader.  The EAR classloader will delegate requests to the container classloader, but it does not do it in the same way as the WAR classloader, as the EAR classloader will actually prefer to delegate up rather than prefer local classes. As you can see this is getting quite hairy and is different to the plain JSE class loading behaviour.

The flat classpath

We talked about how the system classloader looks to the classpath to find classes that have been requested. This classpath could include directories or JAR files and the order which they are looked through is actually dependant on the JVM you are using. There may be multiple copies or versions of the class you require on the classpath, but you will always get the first instance of the class found on the classpath.  It’s essentially just a list of resources, which is why it’s referred to as flat. As a result the classpath list can often be relatively slow to iterate through when looking for a resource.

Problems can occur when applications who are using the same classpath want to use different versions of a class, lets use Hibernate as an example. When two versions of Hibernate JARs exist on the classpath, one version cannot be higher up the classpath for one application than it is for the other, which means both will have to use the same version. One way around this is to bloat the application (WAR) with all the libraries necessary, so that they use their local resources, but this then leads to big applications which are hard to maintain. Welcome to JAR hell! OSGi provides a solution here as it allows versioning of JAR files, or bundles, which results in a mechanism to allow wiring to particular versions of JAR files avoiding the flat classpath problems.

How do I debug my class loading errors?

NoClassDefFoundError/ClassNotFoundException/ClassNoDefFoundException?

 

So, you’ve got an error/exception like the ones above. Well, does the class actually exist? Don’t bother looking in your IDE, as that’s where you compiled your class, it must be there otherwise you’ll get a compile time exception. This is a runtime exception so it’s in the runtime we want to look for the class which it says we’re missing… but where do you start? Consider the following piece of code…

Arrays.toString((((URLClassLoader) Test.class.getClassLoader())
.getURLs()));

This code returns an array list of all jars and directories on the classpath of the classloader the class Test is using. So now we can see if the JAR or location our mystery class should exist in is actually on the classpath. If it does not exist, add it! If it does exist, check the JAR/directory to make sure your class actually exists in that location and add it if it’s missing. These are the two typical problems which result in this error case.

NoSuchMethodError/NoSuchFieldError/AbstractMethodError/IllegalAccessError?

 

Now it’s getting interesting! These are all subclasses of the IncompatibleClassChangeError. We know the classloader has found the class we want (by name), but clearly it hasn’t found the right version. Here we have a class called Test which is making an invocation to another class, Util, but BANG – We get an exception! Lets look at the next snippet of code to debug:

Test.class.getClassLoader().getResource(Util.class.getName()
.replace('.', '/') + ".class");

We’re calling getResource on the classloader of class Test. This returns us the URL of the Util resource. Notice we’ve replaced the ‘.’ with a ‘/’ and added a ‘.class’ at the end of the String. This changes the package and classname of the class we’re looking for (from the perspective of the classloader) into a directory structure and filename on the filesystem – neat. This will show us the exact class we have loaded and we can make sure it’s the correct version. We can use javap -private on the class at a command prompt to see the byte code and check which methods and fields actually exist. You can easily see the structure of the class and validate whether it’s you or the Java runtime which is going crazy! Believe me, at one stage or another you’ll question both, and nearly every time it will be you! :o)

LinkageError/ClassCastException/IllegalAccessError

 

These can occur if two different classloaders load the same class and they try to interact… ouch! Yes, it’s now getting a bit hairy. This can cause problems as we do not know if they will load the classes from the same place. How can this happen? Lets look at the following code, still in the Test class:

Factory.instance().sayHello();

The code looks pretty clean and safe, and it’s not clear how an error could emerge from this line. We’re calling a static factory method to get us an instance of the Test class and are invoking a method on it. Lets look at this supporting image to show the reason why an exception is being thrown.


Here we can see a web classloader (which loaded the Test class) will prefer local classes, so when it makes reference to a class, it will be loaded by the web classloader, if possible. Fairly straightforward so far.  The Test class uses the Factory class to get hold of an instance of the Util class which is fairly typical practice in Java, but the Factory class doesn’t exist in the WAR as it is an external library.  This is no problem as the web classloader can delegate to the shared classloader, which can see the Factory class. Note that the shared classloader is now loading it’s own version of the Util class as when the Factory instantiates the class, it uses the shared classloader (as shown in the first example earlier). The Factory class returns the Util object (created by the shared classloader) back to the WAR, which then tries to use the class, and effectively cast the class to a potentially different version of the same class (the Util class visible to the web classloader). BOOM!

We can run the same code as before from within both places (The Factory.instance() method and the Test class) to see where each of our Util classes are being loaded from.

Test.class.getClassLoader().getResource(Util.class.getName()
.replace('.', '/') + ".class"));

Hopefully this has given you an insight into the world of classloading, and instead of not understanding the classloader, you can now appreciate it with a hint of fear and uncertainty! Thanks for reading and making it to the end. We’d all like to wish you a Merry Christmas and a happy new year from ZeroTurnaround!  Happy coding!
 
Meta: this post is part of the Java Advent Calendar and is licensed under the Creative Commons 3.0 Attribution license. If you like it, please spread the word by sharing, tweeting, FB, G+ and so on! Want to write for the blog? We are looking for contributors to fill all 24 slot and would love to have your contribution! Contact Attila Balazs to contribute!