Java On Raspberry Pi Performance

My objective is to performance test basic JVM functions of different JVMs on the Raspberry Pi hardware to see which JVM is fastest overall. For this I have created a few different tests that exercise specific operations, such as floating-point math, file IO, etc. The results are not relevant compared to other hardware. The results metrics are really not important either other than as they compare to the same test on a different JVM.

VMs Tested

There's currently a few different JVMs available for Raspberry Pi. I tested these:

• Oracle ejre1.7.0_06

java version "1.7.0_06"
Java(TM) SE Embedded Runtime Environment (build 1.7.0_06-b24, headless)
Java HotSpot(TM) Embedded Client VM (build 23.2-b09, mixed mode)

• java-7-openjdk-armel with -zero

java version "1.7.0_03"
OpenJDK Runtime Environment (IcedTea7 2.1.3) (7u3-2.1.3-1)
OpenJDK Zero VM (build 22.0-b10, mixed mode)

• with -cacao

java version "1.7.0_03"
IcedTea Runtime Environment (IcedTea7 2.1.3) (7u3-2.1.3-1)
CACAO (build 1.1.0pre2, compiled mode)

• java-7-openjdk-armel with -jamvm

java version "1.7.0_03"
OpenJDK Runtime Environment (IcedTea7 2.1.3) (7u3-2.1.3-1)
JamVM (build 1.6.0-devel, inline-threaded interpreter with stack-caching)

• The control was a MacBook Pro OSX Darwin Kernel Version 10.8.0
  • 2.4 GHz Intel Core 2 Duo
  • All tests we run with -d32 to force 32-bit mode

java version "1.6.0_37"
Java(TM) SE Runtime Environment (build 1.6.0_37-b06-434-10M3909)
Java HotSpot(TM) 64-Bit Server VM (build 20.12-b01-434, mixed mode)

Results Summary

• Oracle's VM was the fastest in the most categories. Unfortunately they're not providing a JDK yet with a compiler.
• Jam VM looks promising for startup speed
• Zero was the slowest in most categories, but it's also the only OpenJDK VM that appears to be complete
• Zero and Jamm were significantly slower in numerical calculations than the other 2
• IO performance was the most dramatic win for the Oracle VM. IO is so much slower on the all the OpenJDK VMs that seriously impacts Resin performance
• There was no clear best choice after these tests because Oracle doesn't provide a compiler, Zero is so slow, and the other 2 VMs are incomplete.
• I am going to further test Cacao to see how Resin runs since this is OpenJDK VM appears to offer the best overall performance

Startup Time

Tests HelloWorld is a basic print Hello World from a main method. This tests basic startup time. ResinVersion requests Resin version. It tests basic Resin startup time. ResinStatus requests status from a running Resin instance. The call fails as no instance was running, but that's not important. It tests realistic Resin startup and network connection. Results JamVM is clearly the fastest to startup but the margin is not huge. I believe this is due to the smaller size of this JVM. However the caveat is both Cacao and JamVM report warning when running both Resin tests. JamVM reports this: jmm_GetBoolAttribute: Unknown attribute 24 jmm_GetBoolAttribute: Unknown attribute 25 jmm_GetInputArgumentArray Cacao reports this: LOG: [0x400ec000] jmm_GetMemoryManagers: FIX ME! LOG: [0x400ec000] jmm_GetMemoryPools: FIX ME! LOG: [0x400ec000] jmm_GetBoolAttribute: Unknown attribute 24 LOG: [0x400ec000] jmm_GetBoolAttribute: Unknown attribute 25 LOG: [0x400ec000] jmm_GetInputArgumentArray: IMPLEMENT ME! These all look like warning associated with attempting to read MBean attributes from MBeans Resin is expecting to exist. Resin gleans a lot of info about OS and Memory Pools from JMX. I'll have to do more thorough Resin tests to see how adversely this effects Resin.

General Operations

Tests Exceptions tests throwing and catching exception Hashing tests inserting into a map and retrieving from a map Reflection tests looking up a class by name and creating a new instance with Reflection Strings tests String creation and concatenation Results There's nothing outstanding here, although Zero VM is showing up slow again which correlates with my initial real-world tests of Resin.

Math

Tests IntMath lots of calculations using int LongMath lots of calculations using long DoubleMath lots of calculations using double Results Clearly there's significant numerical operation slowness on Zero and Caco. The difference is significant. Although it's worth mentioning all of these are really small numbers, so for anything other than constant intensive calculation it should not matter.

IO

Tests IO tests performed file create, write, read, delete Results Out of all the tests I ran, I think this is the most significant. Resin does quite a lot of IO and these difference here is not small. I believe IO delays are the primary cause for the Resin slowness I'm experiencing.