A beginner’s guide to Java agents

While Java beginners quickly learn typing public static void main to run their applications, even seasoned developers often do not know about the JVM’s support of two additional entry points to a Java process: the premain and the agentmain methods. Both methods allow so-called Java agents to contribute to an existing Java program while residing in their own jar file even without being explicit linked by the main application. Doing so, it is possible to develop, release and publish Java agents entirely separate from the application that is hosting them while still running them in the same Java process.

The simplest Java agent is running prior to the actual application, for example to execute some dynamic setup. An agent could for instance install a specific SecurityManager or configure system properties programmatically. A less useful agent that still serves as a good introductory demo would be the following class that simply prints a line to the console before passing control to the actual application’s main method:

package sample;
public class SimpleAgent<?> {
  public static void premain(String argument) {
    System.out.println("Hello " + argument);
  }
}

To use this class as a Java agent, it needs to be packed in a jar file. Other than with regular Java programs, it is not possible to load classes of a Java agent from a folder. In addition, it is required to specify a manifest entry that references the class containing the premain method:

Premain-Class: sample.SimpleAgent

With this setup, a Java agent can now be added on the command line by pointing to the file system location of the bundled agent and by optionally adding a single argument after an equality sign as in:

java -javaagent:/location/of/agent.jar=World some.random.Program

The execution of the main method in some.random.Program will now be preceded by a print out of Hello World where the second word is the provided argument.

The instrumentation Api

If preemptive code execution was the only capability of Java agents, they would of course only be of little use. In reality, most Java agents are useful only because of the Instrumentation API which can be requested by a Java agent by adding a second parameter of type Instrumentation to the agent’s entry point method. The instrumentation API offers access to lower level functionality that is provided by the JVM which is exclusive to Java agents and that is never provided to regular Java programs. As its centerpiece, the instrumentation API allows for the modification of Java classes before or even after they were loaded.

Any compiled Java class is stored as a .class file which is presented to a Java agent as byte array whenever the class is loaded for the first time. The agent is notified by registering one or multiple ClassFileTransformers into the instrumentation API which are notified for any class that is loaded by a ClassLoader of the current JVM process:

package sample;
public class ClassLoadingAgent {
  public static void premain(String argument, 
                             Instrumentation instrumentation) {
    instrumentation.addTransformer(new ClassFileTransformer() {
      @Override
       public byte[] transform(Module module, 
                               ClassLoader loader, 
                               String name, 
                               Class<?> typeIfLoaded, 
                               ProtectionDomain domain, 
                               byte[] buffer) {
         System.out.println("Class was loaded: " + name);
         return null;
       }
    });
  }
}

In the above example, the agent remains inoperational by returning null from the transformer what aborts the transformation process but only prints a message with the name of the most recently loaded class to the console. But by transforming the byte array that is provided by the buffer parameter, the agent could change the behavior of any class before it is loaded.

Transforming a compiled Java class might sound like a complex task. But fortunately, the Java Virtual Machine Specification (JVMS) details the meaning of every byte that represents a class file. To modify the behavior of a method, one would therefore identify the offset of the method’s code and then add so-called Java byte code instructions to that method to represent the desired changed behavior. Typically, such a transformation is not applied manually but by using a bytecode processor, most famously the ASM library which splits a class file into its components. This way, it becomes possible to look at fields, methods and annotations in isolation what allows for applying more targeted transformations and saves some bookkeeping.

Distraction-free agent development

While ASM makes class file transformation safer and less complicated, it still relies on a good understanding of bytecode and its characteristics by the library’s user. Other libraries however, often based on ASM, allow to express bytecode transformations on a higher level what makes such understanding circumstantial. An example for such a library is Byte Buddy which is developed and maintained by the author of this article. Byte Buddy aims to map bytecode transformations to concepts that are already known to most Java developers in order to make agent development more approachable.

For writing Java agents, Byte Buddy offers the AgentBuilder API which creates and registers a ClassFileTransformer under the covers. Instead of registering a ClassFileTransformer directly, Byte Buddy allows specifying an ElementMatcher to first identify types that are of interest. For each matched type, one or multiple transformations can then be specified. Byte Buddy then translates these instruction into a performant implementation of a transformer that can be installed into the instrumentation API. As an example, the following code recreates the previous non-operational transformer in Byte Buddy’s API:

package sample;
public class ByteBuddySampleAgent {
  public static void premain(String argument, 
                             Instrumentation instrumentation) {
    new AgentBuilder.Default()
      .type(ElementMatchers.any())
      .transform((DynamicType.Builder<?> builder, 
                  TypeDescription type, 
                  ClassLoader loader, 
                  JavaModule module) -> {
         System.out.println("Class was loaded: " + name);
         return builder;
      }).installOn(instrumentation);
  }
}

It should be mentioned that in contrast to the previous example, Byte Buddy will transform all discovered types without applying changes what is less efficient then ignoring those unwanted types altogether. Also, it will ignore classes of the Java core library by default if not specified differently. But in essence, the same effect is achieved such that, such that a simple agent using Byte Buddy can be demonstrated using the above code. 

Measuring execution time with Byte Buddy advice

Instead of exposing class files as byte arrays, Byte Buddy attempts to weave or link regular Java code into instrumented classes. This way, developers of Java agents do not need to produce bytecode directly but can rather rely on the Java programming language and its existing tools to which they already have a relationship to. For Java agents written using Byte Buddy, behavior is often expressed by advice classes where annotated methods describe the behavior that is added to the beginning and the end of existing methods. As an example, the following advice class serves as a template where a method’s execution time is printed to the console:

public class TimeMeasurementAdvice {
  @Advice.OnMethodEnter
  public static long enter() {
    return System.currentTimeMillis();
  }
  @Advice.OnMethodExit(onThrowable = Throwable.class)
  public static void exit(@Advice.Enter long start, 
                          @Advice.Origin String origin) {
     long executionTime = System.currentTimeMillis() - start;
    System.out.println(origin + " took " + executionTime 
                           + " to execute");
  }
}

In the above advice class, the enter method simply records the current timestamp and returns it for making it available at the end of the method. As indicated, enter advice is executed before the actual method body. At the method’s end, the exit advice is applied where the recorded value is subtracted from the current timestamp to determine the method’s execution time. This execution time is then printed to the console.

To make use of the advice, it needs to be applied within the transformer that remained inoperational in the previous example. To avoid printing the runtime for any method, we condition the advice’s application to a custom, runtime-retained annotation MeasureTime that application developers can add to their classes.

package sample;
public class ByteBuddyTimeMeasuringAgent {
  public static void premain(String argument, 
                             Instrumentation instrumentation) {
    Advice advice = Advice.to(TimeMeasurementAdvice.class);
    new AgentBuilder.Default()
      .type(ElementMatchers.isAnnotatedBy(MeasureTime.class))
      .transform((DynamicType.Builder<?> builder, 
                  TypeDescription type, 
                  ClassLoader loader, 
                  JavaModule module) -> {
         return builder.visit(advice.on(ElementMatchers.isMethod());
      }).installOn(instrumentation);
  }
}

Given the application of the above agent, all method execution times are now printed to the console if a class is annotated by MeasureTime. In reality, it would of course make more sense to collect such metrics in a more structured manner but after already having achieved a print-out, this is no longer a complex task to accomplish.

Dynamic agent attachment and class redefinition

Until Java 8, this was possible thanks to utilities stored in a JDK’s tools.jar which can be found in the JDK’s installation folder. Since Java 9, this jar was dissolved into the jdk.attach module which is now available on any regular JDK distribution. Using the contained tooling API, it is possible to attach a jar file to a JVM with a given process id using the following code:

VirtualMachine vm = VirtualMachine.attach(processId);
try {
  vm.loadAgent("/location/of/agent.jar");
} finally {
  vm.detach();
}

When the above API is invoked, the JVM will locate the process with the given id and execute the agents agentmain method in a dedicated thread within that remote virtual machine. Additionally, such agents might request the right to retransform classes in their manifest to change the code of classes that were already loaded:

Agentmain-Class: sample.SimpleAgent
Can-Retransform-Classes: true

Given these manifest entries, the agent can now request that any loaded class is considered for retransformation such that the previous ClassFileTransformer can be registered with an additional boolean argument, indicating a requirement to be notified upon a retransformation attempt:

package sample;
public class ClassReloadingAgent {
  public static void agentmain(String argument, 
                               Instrumentation instrumentation) {
    instrumentation.addTransformer(new ClassFileTransformer() {
      @Override
       public byte[] transform(Module module, 
                               ClassLoader loader, 
                               String name, 
                               Class<?> typeIfLoaded, 
                               ProtectionDomain domain, 
                               byte[] buffer) {
          if (typeIfLoaded == null) {
           System.out.println("Class was loaded: " + name);
         } else {
           System.out.println("Class was re-loaded: " + name);
         }
         return null;
       }
    }, true);
    instrumentation.retransformClasses(
        instrumentation.getAllLoadedClasses());
  }
}

To indicate that a class already was loaded, the instance of the loaded class is now presented to the transformer which would be null for a class that has not been loaded prior. At the end of the above example, the instrumentation API is requested to fetch all loaded classes to submit any such class for retransformation what triggers the execution of the transformer. As before, the class file transformer is implemented to be non-operational for the purpose of demonstrating the working of the instrumentation API.

Of course, Byte Buddy also covers this form of transformation in its API by registering a retransformation strategy in which case, Byte Buddy will also consider all classes for retransformation. Doing so, the previous time-measuring agent can be adjusted to also consider loaded classes if it was attached dynamically:

package sample;
public class ByteBuddyTimeMeasuringRetransformingAgent {
  public static void agentmain(String argument, 
                               Instrumentation instrumentation) {
    Advice advice = Advice.to(TimeMeasurementAdvice.class);
    new AgentBuilder.Default()
       .with(AgentBuilder.RetransformationStrategy.RETRANSFORMATION)
       .disableClassFormatChanges()
      .type(ElementMatchers.isAnnotatedBy(MeasureTime.class))
      .transform((DynamicType.Builder<?> builder, 
                  TypeDescription type, 
                  ClassLoader loader, 
                  JavaModule module) -> {
         return builder.visit(advice.on(ElementMatchers.isMethod());
      }).installOn(instrumentation);
  }
}

As a final convenience, Byte Buddy also offers an API for attaching to a JVM that abstracts over JVM versions and vendors to make the attachment process as simple as possible. Given a process id, Byte Buddy can attache an agent to a JVM by executing a single line of code:

ByteBuddyAgent.attach(processId, "/location/of/agent.jar");

Furthermore, it is even possible to attach to the very same virtual machine process that is currently running what is especially convenient when testing agents:

Instrumentation instrumentation = ByteBuddyAgent.install();

This functionality is available as its own artifact byte-buddy-agent and should make it trivial to try out a custom agent for yourself as owing an instance of Instrumentation makes it possible to simply invoke a premain or agentmain method directly, for example from a unit test, and without any additional setup.

Author: Rafael Winterhalter

Rafael works as a software consultant in Oslo, Norway. He is a proponent of static typing and a JVM enthusiast with particular interest in code instrumentation, concurrency and functional programming. Rafael blogs about software development, regularly presents at conferences and was pronounced a JavaOne Rock Star. When coding outside of his work place, he contributes to a wide range of open source projects and often works on Byte Buddy, a library for simple runtime code generation for the Java virtual machine. For his work, Rafael received a Duke’s Choice award, an Oracle groundbreaker award and was elected a Java Champion.

Twitter Github