JIT Compiler, Inlining and Escape Analysis

Just-in-time (JIT)

Just-in-time (JIT) compiler is the brain of the Java Virtual Machine. Nothing in the JVM affects performance more than the JIT compiler.

For a moment let’s step back and see examples of compiled and non compiled languages.

Languages like Go, C and C++ are called compiled languages because their programs are distributed as binary (compiled) code, which is targeted to a particular CPU.

On the other hand languages like PHP and Perl, are interpreted. The same program source code can be run on any CPU as long as the machine has the interpreter. The interpreter translates each line of the program into binary code as that line is executed.

Java attempts to find a middle ground here. Java applications are compiled, but instead of being compiled into a specific binary for a specific CPU, they are compiled into a bytecode. This gives Java the platform independence of an interpreted language. But Java doesn’t stop here.

In a typical program, only a small sections of the code is executed frequently, and the performance of an application depends primarily on how fast those sections of code are executed. These critical sections are known as the hot spots of the application.
The more times JVM executes a particular code section, the more information it has about it. This allows the JVM to make smart/optimized decisions and compile small hot code into a CPU specific binary. This process is called Just in time compilation (JIT).

Now let’s run a small program and observe JIT compilation.

public class App {
  public static void main(String[] args) {
    long sumOfEvens = 0;
    for(int i = 0; i < 100000; i++) {
      if(isEven(i)) {
        sumOfEvens += i;
      }
    }
    System.out.println(sumOfEvens);
  }

  public static boolean isEven(int number) {
    return number % 2 == 0;
  }
}


#### Run
javac App.java && \
java -server \
     -XX:-TieredCompilation \
     -XX:+PrintCompilation \
              - XX:CompileThreshold=100000 App


#### Output
87    1             App::isEven (16 bytes)
2499950000

Output tells us that isEven method is compiled. I intentionally disabled TieredCompilation to get only the most frequently compiled code.

JIT compiled code will give a great performance boost to your application. Want to check it ? Write a simple benchmark code.

Inlining

Inlining is one of the most important optimizations that JIT compiler makes. Inlining replaces a method call with the body of the method to avoid the overhead of method invocation.

Let’s run the same program again and this time observe inlining.

#### Run
javac App.java && \
java -server \
     -XX:+UnlockDiagnosticVMOptions \
     -XX:+PrintInlining \
     -XX:-TieredCompilation App

#### Output
@ 12   App::isEven (16 bytes)   inline (hot)
2499950000

Inlining again will give a great performance boost to your application.

Escape Analysis

Escape analysis is a technique by which the JIT Compiler can analyze the scope of a new object’s uses and decide whether to allocate it on the Java heap or (Wrong: on the method stack) [Update] handle object members directly (scalar replacement)[/Update]. It also eliminates locks for all non-globally escaping objects

Let’s run a small program and observe garbage collection.

public class App {
  public static void main(String[] args) {
    long sumOfArea = 0;
    for(int i = 0; i < 10000000; i++) {
      Rectangle rect = new Rectangle(i+5, i+10);
      sumOfArea += rect.getArea();
    }
    System.out.println(sumOfArea);
  }

  static class Rectangle {
    private int height;
    private int width;

    public Rectangle(int height, int width) {
      this.height = height;
      this.width = width;
    }

    public int getArea() {
      return height * width;
    }
  }
}

In this example Rectangle objects are created and available only within a loop, they are characterised as NoEscape and can handle object members directly (scalar replacement) instead of allocating objects in heap. Specifically, this means that no garbage collection will happen.

Let’s run the program without EscapeAnalysis.

#### Run
javac App.java && \
java -server \
     -verbose:gc \
     -XX:-DoEscapeAnalysis App

#### Output
[GC (Allocation Failure)  65536K->472K(251392K), 0.0007449 secs]
[GC (Allocation Failure)  66008K->440K(251392K), 0.0008727 secs]
[GC (Allocation Failure)  65976K->424K(251392K), 0.0005484 secs]
16818403770368

As you can see GC kicked-in. Allocation Failure means no more space is left in young generation to allocate objects. So, it is normal cause of young GC.

This time let’s run it with EscapeAnalysis.

#### Run
javac App.java && \
java -server \
    -verbose:gc \
    -XX:+DoEscapeAnalysis App

#### Output
16818403770368

No GC happened this time. Which basically means creating short lived and narrow scoped objects is not necessarily introducing garbage.

DoEscapeAnalysis option is enabled by default. Note that only Java HotSpot Server VM supports this option.

As a consequence, we all should avoid premature optimization, focus on writing more readable/maintainable code and let JVM do it’s job.

Functional vs Imperative Programming. Fibonacci, Prime and Factorial in Java 8

There are multiple programming styles/paradigms, but two well-known ones are Imperative and Functional.

Imperative programming is the most dominant paradigm as nearly all mainstream languages (C++, Java, C#) have been promoting it. But in the last few years functional programming started to gain attention. One of the main driving factors is that simply all new computers are shipped with 4, 8, 16 or more cores and it’s very difficult to write a parallel program in imperative style to utilize all cores. Functional style moves this difficultness to the runtime level and frees developers from hard and error-prone work.

Wait! So what’s the difference between these two styles.

Imperative programming is a paradigm where you tell how exactly and which exact statements machine/runtime should execute to achieve desired result.

Functional programming is a form of declarative programming paradigm where you tell what you would like to achieve and machine/runtime determines the best way how to do it.

Functional style moves the how part to the runtime level and helps developers focus on the what part. By abstracting the how part we can write more maintainable and scalable software.

To handle the challenges introduced by multicore machines and to remain attractive for developers Java 8 introduced functional paradigm next to imperative one.

Enough theory, let’s implement few programming challenges in Imperative and Functional style using Java and see the difference.

Fibonacci Sequence Imperative vs Functional (The Fibonacci Sequence is the series of numbers: 1, 1, 2, 3, 5, 8, 13, 21, 34, … The next number is found by adding up the two numbers before it.)

Fibonacci Sequence in iterative and imperative style

public static int fibonacci(int number) {
  int fib1 = 1;
  int fib2 = 1;
  int fibonacci = fib1;
  for (int i = 2; i < number; i++) {
    fibonacci = fib1 + fib2;
    fib1 = fib2;
    fib2 = fibonacci;
  }
  return fibonacci;
}

for(int i = 1; i  <= 10; i++) {
  System.out.print(fibonacci(i) +" ");
}
// Output: 1 1 2 3 5 8 13 21 34 55 

As you can see here we are focusing a lot on how (iteration, state) rather that what we want to achieve.

Fibonacci Sequence in iterative and functional style

IntStream fibonacciStream = Stream.iterate(
    new int[]{1, 1},
    fib -> new int[] {fib[1], fib[0] + fib[1]}
  ).mapToInt(fib -> fib[0]);

fibonacciStream.limit(10).forEach(fib ->  
    System.out.print(fib + " "));
// Output: 1 1 2 3 5 8 13 21 34 55 

In contrast, you can see here we are focusing on what we want to achieve.

Prime Numbers Imperative vs Functional (A prime number is a natural number greater than 1 that has no positive divisors other than 1 and itself.)

Prime Number in imperative style

public boolean isPrime(long number) {  
  for(long i = 2; i <= Math.sqrt(number); i++) {  
    if(number % i == 0) return false;  
  }  
  return number > 1;  
}
isPrime(9220000000000000039L) // Output: true

Again here we are focusing a lot on how (iteration, state).

Prime Number in functional style

public boolean isPrime(long number) {  
  return number > 1 &&  
    LongStream
     .rangeClosed(2, (long) Math.sqrt(number))  
     .noneMatch(index -> number % index == 0);
}
isPrime(9220000000000000039L) // Output: true

Here again we are focusing on what we want to achieve. The functional style helped us to abstract away the process of explicitly iterating over the range of numbers.

You might now think, hmmm, is this all we can have …. ? Let’s see how can we use all our cores (gain parallelism) in functional style.

public boolean isPrime(long number) {  
  return number > 1 &&  
    LongStream
    .rangeClosed(2, (long) Math.sqrt(number))
    .parallel()  
    .noneMatch(index -> number % index == 0);
}
isPrime(9220000000000000039L) // Output: true

That’s it! We just added .parallel() to the stream. You can see how library/runtime handles complexity for us.

Factorial Imperative vs Functional ( The factorial of n is the product of all positive integers less than or equal to n.)

Factorial in iterative and imperative style

public long factorial(int n) {
  long product = 1;
  for ( int i = 1; i <= n; i++ ) {
    product *= i;
  }
  return product;
}
factorial(5) // Output: 120

Factorial in iterative and functional style

public long factorial(int n) {
 return LongStream
   .rangeClosed(1, n)
   .reduce((a, b) -> a *   b)
   .getAsLong();
}
factorial(5) // Output: 120

It’s worth repeating that by abstracting the how part we can write more maintainable and scalable software.

To see all the functional goodies introduced by Java 8 check out the following Lambda Expressions, Method References and Streams guide. Continue reading Functional vs Imperative Programming. Fibonacci, Prime and Factorial in Java 8