Inspecting HotSpot JVM Options

Introduction

A method for inspecting the comprehensive set of HotSpot JVM options and some examples comparing different outputs.


Background

The Oracle JVM provides an enormous spectrum of options to control the runtime JVM. These options are given a criminally short and non-comprehensive discussion at the OTN here. OTN divides these options into three categories: Behavioural, Performance and Debugging. This is a useful abstraction but some examples Oracle gives in each category can be misleading due to the overlap between Behavioural and Performance.

The JVM options themselves can be controlled in a number of ways:

• via the command line on JVM startup.
  
• via JMX for certain options where this is allowed.
  
• indirectly via the command line by a super-option which then sets other options.
  
• automatically by the JVM. The JVM has ergonomic capability to detect features of the host and set options accordingly.

Using the first two methods we are explicitly setting options ourselves and can easily track what value each option has (true, false, 20, 100 etc). These last two methods for setting options can cause developers the most confusion, as we have less insight and therefore less understanding over what options are being set. This puts us in an unfavourable position when explaining the runtime behaviour of our applications. To dump out every JVM option and its value there is an option to do this hidden away in the JVM source code: 

-XX:+PrintFlagsFinal


Usage

Let's see this in action! First let's run the following:


$ jdk1.6.0_23/bin/java -XX:+UnlockDiagnosticVMOptions -XX:+PrintFlagsFinal -version

 (I have added -XX:+UnlockDiagnosticVMOptions as this unlocks more options for viewing and control. Also I have added -version at the end so Java doesn't complain that it has nothing to run and to supplement the results).

The sysout is quite large so I've saved it here. Firstly you may be surprised at the sheer quantity of JVM options available: 717! That's a lot of configuration! Before we dive into the options lets look at our host specification and the output of the -version option. The host has 8 dual core 64bit AMD Opteron processors running at 3.2GHz so 16 cores in total with 64GB of RAM. The Java -version output shows:

java version "1.6.0_23" 
Java(TM) SE Runtime Environment (build 1.6.0_23-b05) 
Java HotSpot(TM) Server VM (build 19.0-b09, mixed mode)

this tells us we are running Java 6 update 23 using the Server version of the HotSpot VM in 32bit mode. Now let's look closer at the output. For each option we can see the following columns:

1. Datatype. bool for booleans, uintx for unsigned integers etc
   
2. Name
   
3. Assignment. = means this is the default. := means this option was modified from the default value by command line or ergonomics.
   
4. Value
   
5. Category. There are two large categories diagnostic or product. (I find this a better grouping than Behavioural, Performance and Debugging.) and also manageable if this option can be set via JMX. C1 relates to the JIT compiler used in the Client JVM. C2 means the same but for the Server JVM and I suspect pd means Platform Dependent as most pd options are Solaris only.

Let's look closer at the modified options which have the := assignment:

uintx InitialHeapSize                     := 67108864         {product} 
uintx MaxHeapSize                         := 1073741824       {product} 
uintx ParallelGCThreads                   := 13               {product} 
bool PrintFlagsFinal                     := true             {product} 
bool UnlockDiagnosticVMOptions           := true             {diagnostic} 
bool UseParallelGC                       := true             {product}

apart from the two flags which I set myself on the command line we can see Java ergonomics has given the heap a sensible range of 64MB to 1GB. The fairly old Parallel Garbage Collector has been chosen with an enormous 13 threads spawned to do the collection. Straightaway we've got a lot more insight into how our JVM is configured and a few ideas for improvements.


Client vs Server JVM Analysis 

We know from looking at the -version output Java automatically chose the server JVM instead of the client JVM (Java automatically chooses server if it detects 64bit processors). But how do we know what options are set because Java chose the Server JVM? What options were set because of our host specification? Lets run the following commands and review the output:


$ jdk1.6.0_23/bin/java -client -XX:+UnlockDiagnosticVMOptions -XX:+PrintFlagsFinal -version
$ jdk1.6.0_23/bin/java -server -XX:+UnlockDiagnosticVMOptions -XX:+PrintFlagsFinal -version

I have saved the client output here and the diff client server here. Firstly lets look at the output from the Client JVM; it has 661 options, 56 fewer than the Server version. Now let's look solely at the Client options which have been modified, excluding the two options I set on the command line there are only two values which were changed:

uintx InitialHeapSize                     := 16777216         {product} 
uintx MaxHeapSize                         := 268435456        {product}

so Java has chosen some fairly sensible values for the Client JVM: 16MB initial heap size with a max size of 256MB. There are many entries in the diff file mostly by virtue of the extra options available to the Server JVM or options which relate to memory simply having higher values. Here are the more interesting differences:

<  intx CICompilerCount                      = 1                {product} 
>  intx CICompilerCount                      = 2                {product} 
<  intx CompileThreshold                     = 1500             {pd product} 
>  intx CompileThreshold                     = 10000            {pd product}

 The Client VM waits only 1,500 invocations before compiling a method to native code and uses only one thread for compilation vs the Server which waits for 10,000 invocations and uses two threads.

>  bool DoEscapeAnalysis                     = true             {C2 product}

 In this build the Server VM has escape analysis enabled by default whereas the Client VM cannot enable escape analysis at all. I don't think this is sensible on Oracle's part as whilst escape analysis can improve performance for scientific computing (where many primitives are created within a method and then discarded once a final value has been computed) for other applications escape analysis can lower performance when arguments do escape their methods and are better left on the heap. Slipping in such a change transparently can also be problematic as developers upgrading to a new JVM may see performance degradation in their applications when no visible command line option has changed. Developers should benchmark accordingly when upgrading JVM builds.

<  bool RewriteBytecodes                     = false            {pd product} 
<  bool RewriteFrequentPairs                 = false            {pd product} 
>  bool RewriteBytecodes                     = true             {pd product} 
>  bool RewriteFrequentPairs                 = true             {pd product}

 An optimisation to rewrite two bytecode operations as one operation.


Super Option Activation

As mentioned earlier some options are super-options and will enable other options. Let's take a look at a common super option: -XX:+AggressiveOpts

The two commands are:

jdk1.6.0_23/bin/java -server -XX:+UnlockDiagnosticVMOptions -XX:+PrintFlagsFinal -version
jdk1.6.0_23/bin/java -server -XX:+AggressiveOpts -XX:+UnlockDiagnosticVMOptions -XX:+PrintFlagsFinal -version

here are all the differences:


<  bool AggressiveOpts                       = false            {product} 
>  bool AggressiveOpts                      := true             {product} 
<  intx AutoBoxCacheMax                      = 128              {C2 product} 
>  intx AutoBoxCacheMax                      = 20000            {C2 product} 
<  intx BiasedLockingStartupDelay            = 4000             {product} 
>  intx BiasedLockingStartupDelay            = 500              {product} 
<  bool EliminateAutoBox                     = false            {C2 diagnostic} 
>  bool EliminateAutoBox                     = true             {C2 diagnostic} 
<  bool OptimizeFill                         = false            {C2 product} 
>  bool OptimizeFill                         = true             {C2 product} 
<  bool OptimizeStringConcat                 = false            {C2 product} 
>  bool OptimizeStringConcat                 = true             {C2 product}

So a number of options have been enable and some others have been given more aggressive values. We know -XX:+AggressiveOpts changes with each JVM build. Let's look at an older build:

(NB. Java6u20 is the oldest build to support this form of option dumping)

jdk1.6.0_20/bin/java -server -XX:+UnlockDiagnosticVMOptions -XX:+PrintFlagsFinal -version
jdk1.6.0_20/bin/java -server -XX:+AggressiveOpts -XX:+UnlockDiagnosticVMOptions -XX:+PrintFlagsFinal


<  bool AggressiveOpts                       = false            {product} 
>  bool AggressiveOpts                      := true             {product} 
<  intx AutoBoxCacheMax                      = 128              {C2 product} 
>  intx AutoBoxCacheMax                      = 20000            {C2 product} 
<  intx BiasedLockingStartupDelay            = 4000             {product} 
>  intx BiasedLockingStartupDelay            = 500              {product} 
<  bool DoEscapeAnalysis                     = false            {C2 product} 
>  bool DoEscapeAnalysis                     = true             {C2 product} 
<  bool EliminateAutoBox                     = false            {C2 diagnostic} 
>  bool EliminateAutoBox                     = true             {C2 diagnostic} 
<  bool UseLoopPredicate                     = false            {C2 product} 
>  bool UseLoopPredicate                     = true             {C2 product}

We can see in 6u20 DoEscapeAnalysis was an AggressiveOpt but in the new build 6u23 DoEscapeAnalysis became the default (as did UseLoopPredicate). Along the same lines OptimizeFill and OptimizeStringConcat may become default values in future builds.


Conclusion

There is enormous scope for coarse and fine grained control of JVM behaviour. The option -XX:+PrintFlagsFinal allows comprehensive reporting of the options and their values. The available options vary by build and JVM type (server or client). Recording and auditing this output is an important step in any Java benchmarking or continuous monitoring exercise. Oracle can surreptitiously enable options in new Java builds which may cause inexplicable changes in performance and behaviour of existing applications.