How and Why is Unsafe used in Java?

Overview

sun.misc.Unsafe has been in Java from at least as far back as Java 1.4 (2004).  In Java 9, Unsafe will be hidden along with many other, for-internal-use classes. to improve the maintainability of the JVM.  While it is still unclear exactly what will replace Unsafe, and I suspect it will be more than one thing which replaces it, it raises the question, why is it used at all?

Doing things which the Java language doesn’t allow but are still useful.

Java doesn’t allow many of the tricks which are available to lower level languages.  For most developers this is very good thing, and it not only saves you from yourself, it also saves you from your co-workers.  It also makes it easier to import open source code because you know there is limits to how much damage they can do.  Or at least there is limits to how much you can do accidentally. If you try hard enough you can still do damage.

But why would you even try, you might wonder?  When building libraries many (but not all) of the methods in Unsafe are useful and in some cases, there is no other way to do the same thing without using JNI, which is even more dangerous and you lose the “compile once, run anywhere”

Deserialization of objects

When deserializing or building an object using a framework, you make the assumption you want to reconstitute an object which existed before.  You expect that you will use reflection to either call the setters of the class, or more likely set the internal fields directly, even the final fields.  The problem is you want to create an instance of an object, but you don’t really need a constructor as this is likely to only make things more difficult and have side effects.
public class A implements Serializable {
private final int num;
public A(int num) {
System.out.println("Hello Mum");
this.num = num;
}

public int getNum() {
return num;
}
}

In this class, you should be able to rebuild and set the final field, but if you have to call a constructor and it might do things which don’t have anything to do with deserialization.  For these reasons many libraries use Unsafe to create instances without calling a constructor

Unsafe unsafe = getUnsafe();
Class aClass = A.class;
A a = (A) unsafe.allocateInstance(aClass);

Calling allocateInstance avoids the need to call the appropriate constructor, when we don’t need one.

Thread safe access to direct memory

Another use for Unsafe is thread safe access to off heap memory.  ByteBuffer gives you safe access to off heap or direct memory, however it doesn’t have any thread safe operations.  This is particularly useful if you want to share data between processes.

import sun.misc.Unsafe;
import sun.nio.ch.DirectBuffer;

import java.io.File;
import java.io.IOException;
import java.io.RandomAccessFile;
import java.lang.reflect.Field;
import java.nio.MappedByteBuffer;
import java.nio.channels.FileChannel;

public class PingPongMapMain {
public static void main(String... args) throws IOException {
boolean odd;
switch (args.length < 1 ? "usage" : args[0].toLowerCase()) {
case "odd":
odd = true;
break;
case "even":
odd = false;
break;
default:
System.err.println("Usage: java PingPongMain [odd|even]");
return; }
int runs = 10000000;
long start = 0;
System.out.println("Waiting for the other odd/even");
File counters = new File(System.getProperty("java.io.tmpdir"), "counters.deleteme"); counters.deleteOnExit();

try (FileChannel fc = new RandomAccessFile(counters, "rw").getChannel()) {
MappedByteBuffer mbb = fc.map(FileChannel.MapMode.READ_WRITE, 0, 1024);
long address = ((DirectBuffer) mbb).address();
for (int i = -1; i < runs; i++) {
for (; ; ) {
long value = UNSAFE.getLongVolatile(null, address);
boolean isOdd = (value & 1) != 0;
if (isOdd != odd)
// wait for the other side.
continue;
// make the change atomic, just in case there is more than one odd/even process
if (UNSAFE.compareAndSwapLong(null, address, value, value + 1))
break;
}
if (i == 0) {
System.out.println("Started");
start = System.nanoTime();
}
}
}
System.out.printf("... Finished, average ping/pong took %,d ns%n",
(System.nanoTime() - start) / runs);
}

static final Unsafe UNSAFE;

static {
try {
Field theUnsafe = Unsafe.class.getDeclaredField("theUnsafe");
theUnsafe.setAccessible(true);
UNSAFE = (Unsafe) theUnsafe.get(null);
} catch (Exception e) {
throw new AssertionError(e);
}
}
}

When you run this in two programs, one with odd and the other with even. You can see that each process is changing data via  persisted shared memory.

In each program it maps the same are of the disks cache into the process.  There is actually only one copy of the file in memory.  This means the memory can be shared, provided you use thread safe operations such as the volatile and CAS operations.

The output on an i7-3970X is

Waiting for the other odd/even
Started
… Finished, average ping/pong took 83 ns

That is 83 ns round trip time between two processes. When you consider System V IPC takes around 2,500 ns and IPC volatile instead of persisted, that is pretty quick.

Is using Unsafe suitable for work?

I wouldn’t recommend you use Unsafe directly.  It requires far more testing than natural Java development.  For this reason I suggest you use a library where it’s usage has been tested already.  If you wan to use Unsafe yourself, I suggest you thoughly test it’s usage in a stand alone library.  This limits how Unsafe is used in your application and give syou a safer, Unsafe.

Conclusion

It is interesting that Unsafe exists in Java, and you might to play with it at home.  It has some work applications especially in writing low level libraries, but in general it is better to use a library which uses Unsafe which has been tested than use it directly yourself.

About the Author.

Peter Lawrey has the most Java answers on StackOverflow. He is the founder of the Performance Java User’s Group, and lead developer of Chronicle Queue and Chronicle Map, two libraries which use Unsafe to share persisted data between processes.

Developers want to be heard

How often have you been in this situation?

You’re in a meeting with the team and you’re all discussing the implementation of a new feature. The group seems to be converging on a design, but there’s something about it that feels off, some sort of “smell”.  You point this out to the team, perhaps outlining the specific areas that make you uncomfortable. Maybe you even have an alternative solution. The team lets you have your say, but assures you their solution is The Way.

Or what about this?

A tech lead asks you to fix a bug, and as you work on your implementation you bounce ideas around periodically just to make sure you’re on the right track. Things seem to be OK, until it comes to getting your code merged. Now it becomes clear that your implementation is not what the lead had in mind, and there’s a frustrating process of back-and-forth while you explain and defend your design decisions whilst trying to incorporate the feedback. At the end, the solution doesn’t feel like your work, and you’re not entirely sure what was wrong with your initial implementation – it fixed the problem, passed the tests, and met the criteria you personally think are important (readability / scalability / performance / stability / time-to-implementation, whatever it is that you value).

When you speak to women developers, you often hear “I feel like I have to work really hard to convince people about my ideas” or “it’s taken me a long time to prove my worth” or “I still don’t know how to be seen as a full member of the team”.

And you hear these a lot from women because we ask women a lot what they don’t like about their work, since we’re (correctly) concerned as an industry about the lack of female developers and the alarming rate at which they leave technical roles.

However, if you ask any developer you’ll hear something similar.  Even very senior, very experienced (very white, very male) developers have a lot of frustration trying to convince others that their ideas have value.

It’s not just a Problem With Women.

I’ve been wondering if our problem is that we don’t listen.  When it comes to exchanging technical ideas, I think overall we’re not good at really listening to each other.  At the very least, I think we’re bad at making people feel heard.

Let’s think about this for a bit: if we don’t listen to developers, if we don’t help them to understand why they’re wrong, or work together to incorporate all ideas into a super-idea that’s the best solution, developers will become frustrated.  We’re knowledge workers, what we bring to the table is our brains, our ideas, our solutions.  If these are persistently not valued, we could go one of two ways:

  1. Do it our way anyway. We still think we’re right, we haven’t been convinced that our idea is not correct, or that someone else’s is correct (maybe because we didn’t listen to them? Maybe because no-one took the time to listen to us and explain why we were wrong? Maybe because we were right and no-one was listening?).
  2. Leave. We might join a team where we feel more valued, or we might leave development all together.  At least as a business analyst, as a project manager, as a tester, people have to listen to us: by their very definition the output of those jobs is an input to the development team. 

Option one leads to rogue code in our application, often not checked by anyone else in the team let alone understood by them, because of course we were not allowed to implement this.  So it’s done in secret.  If it works, at worst no-one notices.  And at best? You’re held up as a hero for actually Getting Something Done. This can’t be right, we’re rewarding the rebel behaviour, not encouraging honest discussion and making people feel included.

Option two leads to the team (and maybe the industry) losing a developer. Sometimes you might argue “Good Riddance”.  But there’s such a skills shortage, it’s so hard (and expensive) to hire developers, and you must have seen something in that developer to hire them in the first place, that surely it’s cheaper, better, to make them feel welcome, wanted, valued?

What can we do to listen to each other?

  • Retrospectives. Done right, these give the team a safe place to discuss things, to ask questions, to suggest improvements.  It’s not necessarily a place to talk about code or design, but it is a good place to raise issues like the ones above, and to suggest ways to address these problems.
  • You could schedule sessions for sharing technical ideas: maybe regular brown bags to help people understand the technologies or existing designs; maybe sessions where the architecture or design or principals of a particular area are explored and explained; maybe space and time for those who feel unheard to explain in more detail where they’re coming from, principals that are important to them.  It’s important that these sessions are developer-lead, so that everyone has an opportunity to share their ideas.
  • Pair programming. When you’re sat together, working together, there’s a flow of ideas, information, designs, experience. It’s not necessarily a case of a more senior person mentoring a less experienced developer, all of us have different skills and value different qualities in our implementation – for example, one of you could be obsessive about the tests, where the other really cares about readability of the code. When you implement something in a pair, you feel ownership of that code but you feel less personally attached to the code – you created it, but you created it from the best of both of you, you had to listen to each other to come to a conclusion and implement it. And if an even better idea comes along, great, it just improves the code. You’re constantly learning from the other people you work with, and can see the effect of them learning from you.
  • We should value, and coach, more skills than simply technology skills. I don’t know why we still seem to have this idea that developers are just typists communing with the computer – the best developers work well in teams and communicate effectively with the business and users; the best leaders make everyone in their team more productive.  In successful organisations, sales people are trained in skills like active listening, like dealing with objections.  More development teams should focus on improving these sorts of communication skills as a productivity tool. 

I’m sure there are loads more options, I just thought of these in ten minutes.  If you read any books aimed at business people, or at growing your career, there are many tried and tested methods for making people feel heard, for playing nicely with others.

So we should work harder to listen to each other. Next time you’re discussing something with your team, or with your boss, try and listen to what they’re saying – ask them to clarify things you don’t understand (you won’t look stupid, and developers love explaining things), and repeat back what you do understand. Request the same respect in return – if you feel your ideas aren’t being heard, make sure you sit down with someone to talk over your ideas or your doubts in more detail, and be firm in making sure the team or that person is hearing what you think you’re saying.  We may be wrong, they may be right, but we need to understand why we’re wrong, or we’ll never learn.

If we all start listening a bit more, maybe we’ll be a bit happier.

This post is part of the Java Advent Calendar and is licensed under the Creative Commons 3.0 Attribution license. If you like it, please spread the word by sharing, tweeting, FB, G+ and so on!