Sunday, August 14, 2011

Leaving Blogger

All, after a few years of publishing my blog on the Blogger service provided by Google, I've decided to move to a paid blog with my own domain name hosted by Wordpress. Whilst Blogger is a great free blogging service, lately I have been reading about Google deleting people's accounts, for non-fee paying users Google has little accountability and it is difficult - if at all possible - to restore accounts later.

Please point your browsers here for my regular RSS digest and new and original tehnical articles.

Friday, July 1, 2011

Friday, June 10, 2011

RSS Digest: Week Ending 10-Jun-2011


Java SE 6 Update 26 is released!

Moving to OpenJDK as the official Java SE 7 Reference Implementation


Google Publishes C++, Go, Java and Scala Performance Benchmarks

Windows 8: could it be more than lipstick on a pig?

Apple Introduces iCloud

New Version of iOS Includes Notification Center, iMessage, Newsstand, Twitter Integration Among 200 New Features

Iomega Home Media Network Hard Drive Cloud Edition 2TB

Will 10 GigE Finally Go Big in HPC?

Friday, May 27, 2011

Friday, April 15, 2011

Friday, April 8, 2011

RSS Digest: Week Ending 08-Apr-2011


DZone Meets The Well-Grounded Java Developers

Next step on the road...

Java founder Gosling joins Google

Cliff Click in "A JVM Does What?"


My Bash Profile - Part V: Prompt

My vimrc


Informatica, HP, and Mellanox/Voltaire Benchmark Report

AMD: quad-core Llano outshines Sandy Bridge

Evaluating Multi-core for HPC

Dell XPS 15 (2011) review

Samsung SyncMaster C27A750 review

Intel unveils next-generation Xeons with ten cores

Friday, March 25, 2011

RSS Digest: Week Ending 25-Mar-2011


The Performance Paradox of the JVM: Why More Hardware Means More Failures

Java Tip: Find process id of running Java process

The impact of Garbage Collection on Java performance


My Bash Profile - Part IV: Functions


May’s Law and Parallel Software

Apple iPad 2 review

Satellite snaps extent of Japan's post-quake blackout

New Intel Turbo Boost Technology

Nintendo 3DS: the essential guide

Firefox 4 finally ready to roll

iPad 2 Arrives in 25 More Countries This Friday

CUDAfy - General Purpose GPU Development on .NET

Friday, March 18, 2011

Friday, March 4, 2011

RSS Digest: Week Ending 04-Mar-2011


Java Bytecode Fundamentals: Using Objects and Calling Methods

Performance tuning @ JFokus

The Azul Garbage Collector

Nondeterministic Floating-Point Conversions in Java

Java Synchronization (Mutual Exclusion) Benchmark

JavaOne and Oracle Develop 2010

JVM Language Summit 2010


Thunderbolt: The Fastest Data Connection to Your PC Just Arrived

The essential tools for IT pros in 2011

How long does it take to make a context switch?

Apple iPad 2 review: first look

Apple staffer: wait for iPad 3


I'm a romantic, says Xavi, heartbeat of Barcelona and Spain

Friday, January 28, 2011

RSS Digest: Week Ending 28-Jan-2011


The Future Of Java

Looking Ahead to Java SE 7 and 8: A Discussion with Oracle’s Java Language Architect, Brian Goetz


TIOBE Programming Community Index for January 2011


A Day in the Life of an Emerging Market Trader

Financial crisis was 'avoidable', says official US report

Friday, January 21, 2011

Friday, January 7, 2011

Inspecting HotSpot JVM Options


A method for inspecting the comprehensive set of HotSpot JVM options and some examples comparing different outputs.


The Oracle JVM provides an enormous spectrum of options to control the runtime JVM. These options are given a criminally short and non-comprehensive discussion at the OTN here. OTN divides these options into three categories: Behavioural, Performance and Debugging. This is a useful abstraction but some examples Oracle gives in each category can be misleading due to the overlap between Behavioural and Performance.

The JVM options themselves can be controlled in a number of ways:
  • via the command line on JVM startup.
  • via JMX for certain options where this is allowed.
  • indirectly via the command line by a super-option which then sets other options.
  • automatically by the JVM. The JVM has ergonomic capability to detect features of the host and set options accordingly.
Using the first two methods we are explicitly setting options ourselves and can easily track what value each option has (true, false, 20, 100 etc). These last two methods for setting options can cause developers the most confusion, as we have less insight and therefore less understanding over what options are being set. This puts us in an unfavourable position when explaining the runtime behaviour of our applications. To dump out every JVM option and its value there is an option to do this hidden away in the JVM source code: 



Let's see this in action! First let's run the following:

$ jdk1.6.0_23/bin/java -XX:+UnlockDiagnosticVMOptions -XX:+PrintFlagsFinal -version

 (I have added -XX:+UnlockDiagnosticVMOptions as this unlocks more options for viewing and control. Also I have added -version at the end so Java doesn't complain that it has nothing to run and to supplement the results).

The sysout is quite large so I've saved it here. Firstly you may be surprised at the sheer quantity of JVM options available: 717! That's a lot of configuration! Before we dive into the options lets look at our host specification and the output of the -version option. The host has 8 dual core 64bit AMD Opteron processors running at 3.2GHz so 16 cores in total with 64GB of RAM. The Java -version output shows:

java version "1.6.0_23" 
Java(TM) SE Runtime Environment (build 1.6.0_23-b05) 
Java HotSpot(TM) Server VM (build 19.0-b09, mixed mode)

this tells us we are running Java 6 update 23 using the Server version of the HotSpot VM in 32bit mode. Now let's look closer at the output. For each option we can see the following columns:
  1. Datatype. bool for booleans, uintx for unsigned integers etc
  2. Name
  3. Assignment. = means this is the default. := means this option was modified from the default value by command line or ergonomics.
  4. Value
  5. Category. There are two large categories diagnostic or product. (I find this a better grouping than Behavioural, Performance and Debugging.) and also manageable if this option can be set via JMX. C1 relates to the JIT compiler used in the Client JVM. C2 means the same but for the Server JVM and I suspect pd means Platform Dependent as most pd options are Solaris only.

Let's look closer at the modified options which have the := assignment:

uintx InitialHeapSize                     := 67108864         {product} 
uintx MaxHeapSize                         := 1073741824       {product} 
uintx ParallelGCThreads                   := 13               {product} 
bool PrintFlagsFinal                     := true             {product} 
bool UnlockDiagnosticVMOptions           := true             {diagnostic} 
bool UseParallelGC                       := true             {product}

apart from the two flags which I set myself on the command line we can see Java ergonomics has given the heap a sensible range of 64MB to 1GB. The fairly old Parallel Garbage Collector has been chosen with an enormous 13 threads spawned to do the collection. Straightaway we've got a lot more insight into how our JVM is configured and a few ideas for improvements.

Client vs Server JVM Analysis 

We know from looking at the -version output Java automatically chose the server JVM instead of the client JVM (Java automatically chooses server if it detects 64bit processors). But how do we know what options are set because Java chose the Server JVM? What options were set because of our host specification? Lets run the following commands and review the output:

$ jdk1.6.0_23/bin/java -client -XX:+UnlockDiagnosticVMOptions -XX:+PrintFlagsFinal -version
$ jdk1.6.0_23/bin/java -server -XX:+UnlockDiagnosticVMOptions -XX:+PrintFlagsFinal -version

I have saved the client output here and the diff client server here. Firstly lets look at the output from the Client JVM; it has 661 options, 56 fewer than the Server version. Now let's look solely at the Client options which have been modified, excluding the two options I set on the command line there are only two values which were changed:

uintx InitialHeapSize                     := 16777216         {product} 
uintx MaxHeapSize                         := 268435456        {product}

so Java has chosen some fairly sensible values for the Client JVM: 16MB initial heap size with a max size of 256MB. There are many entries in the diff file mostly by virtue of the extra options available to the Server JVM or options which relate to memory simply having higher values. Here are the more interesting differences:

<  intx CICompilerCount                      = 1                {product} 
>  intx CICompilerCount                      = 2                {product} 
<  intx CompileThreshold                     = 1500             {pd product} 
>  intx CompileThreshold                     = 10000            {pd product}

 The Client VM waits only 1,500 invocations before compiling a method to native code and uses only one thread for compilation vs the Server which waits for 10,000 invocations and uses two threads.

>  bool DoEscapeAnalysis                     = true             {C2 product}

 In this build the Server VM has escape analysis enabled by default whereas the Client VM cannot enable escape analysis at all. I don't think this is sensible on Oracle's part as whilst escape analysis can improve performance for scientific computing (where many primitives are created within a method and then discarded once a final value has been computed) for other applications escape analysis can lower performance when arguments do escape their methods and are better left on the heap. Slipping in such a change transparently can also be problematic as developers upgrading to a new JVM may see performance degradation in their applications when no visible command line option has changed. Developers should benchmark accordingly when upgrading JVM builds.

<  bool RewriteBytecodes                     = false            {pd product} 
<  bool RewriteFrequentPairs                 = false            {pd product} 
>  bool RewriteBytecodes                     = true             {pd product} 
>  bool RewriteFrequentPairs                 = true             {pd product}

 An optimisation to rewrite two bytecode operations as one operation.

Super Option Activation

As mentioned earlier some options are super-options and will enable other options. Let's take a look at a common super option: -XX:+AggressiveOpts

The two commands are:

jdk1.6.0_23/bin/java -server -XX:+UnlockDiagnosticVMOptions -XX:+PrintFlagsFinal -version
jdk1.6.0_23/bin/java -server -XX:+AggressiveOpts -XX:+UnlockDiagnosticVMOptions -XX:+PrintFlagsFinal -version

here are all the differences:

<  bool AggressiveOpts                       = false            {product} 
>  bool AggressiveOpts                      := true             {product} 
<  intx AutoBoxCacheMax                      = 128              {C2 product} 
>  intx AutoBoxCacheMax                      = 20000            {C2 product} 
<  intx BiasedLockingStartupDelay            = 4000             {product} 
>  intx BiasedLockingStartupDelay            = 500              {product} 
<  bool EliminateAutoBox                     = false            {C2 diagnostic} 
>  bool EliminateAutoBox                     = true             {C2 diagnostic} 
<  bool OptimizeFill                         = false            {C2 product} 
>  bool OptimizeFill                         = true             {C2 product} 
<  bool OptimizeStringConcat                 = false            {C2 product} 
>  bool OptimizeStringConcat                 = true             {C2 product}

So a number of options have been enable and some others have been given more aggressive values. We know -XX:+AggressiveOpts changes with each JVM build. Let's look at an older build:

(NB. Java6u20 is the oldest build to support this form of option dumping)

jdk1.6.0_20/bin/java -server -XX:+UnlockDiagnosticVMOptions -XX:+PrintFlagsFinal -version
jdk1.6.0_20/bin/java -server -XX:+AggressiveOpts -XX:+UnlockDiagnosticVMOptions -XX:+PrintFlagsFinal

<  bool AggressiveOpts                       = false            {product} 
>  bool AggressiveOpts                      := true             {product} 
<  intx AutoBoxCacheMax                      = 128              {C2 product} 
>  intx AutoBoxCacheMax                      = 20000            {C2 product} 
<  intx BiasedLockingStartupDelay            = 4000             {product} 
>  intx BiasedLockingStartupDelay            = 500              {product} 
<  bool DoEscapeAnalysis                     = false            {C2 product} 
>  bool DoEscapeAnalysis                     = true             {C2 product} 
<  bool EliminateAutoBox                     = false            {C2 diagnostic} 
>  bool EliminateAutoBox                     = true             {C2 diagnostic} 
<  bool UseLoopPredicate                     = false            {C2 product} 
>  bool UseLoopPredicate                     = true             {C2 product}

We can see in 6u20 DoEscapeAnalysis was an AggressiveOpt but in the new build 6u23 DoEscapeAnalysis became the default (as did UseLoopPredicate). Along the same lines OptimizeFill and OptimizeStringConcat may become default values in future builds.


There is enormous scope for coarse and fine grained control of JVM behaviour. The option -XX:+PrintFlagsFinal allows comprehensive reporting of the options and their values. The available options vary by build and JVM type (server or client). Recording and auditing this output is an important step in any Java benchmarking or continuous monitoring exercise. Oracle can surreptitiously enable options in new Java builds which may cause inexplicable changes in performance and behaviour of existing applications.

RSS Digest: Week Ending 07-Jan-2011


Java Bytecode Fundamentals


What's new in Linux 2.6.37


2nd UK GPU Computing Conference, December 13-14 2010, Cambridge

CES: Sony's 3D visor - a hit, or garish headgear?


Quantnet's Best-selling Books of 2010

Bloomberg: 100 Richest Hedge Funds

Tuesday, January 4, 2011

RSS Digest: Week Ending 31-Dec-2010


Java concurrency bug patterns for multicore systems

Josh Bloch on Java and Programming

Java SE & Java EE Performance: A Lot Of Garbage

A Comparative Study of JVM Implementations with SPECjvm2008

Azul's Pauseless Garbage Collector

NUMA-Aware Java Heaps for Server Applications

Help for the NUMA Weary

Java Knobs: How to Change Performance of the Sun JVM

Do Java 6 threading optimizations actually work?


Scientific Computing on Heterogeneous Architectures

Google Body Browser
(requires a WebGL browser)

The disposable academic - Why doing a PhD is often a waste of time

Sick of reading? Five cool things you can do with your Amazon Kindle

Intel's Sandy Bridge Architecture Exposed