Java On Raspberry Pi Performance

From Resin 4.0 Wiki

(Difference between revisions)
Jump to: navigation, search
 
(4 intermediate revisions by one user not shown)
Line 1: Line 1:
−
My objective is to performance test various basic JVM functions of different JVMs on the Raspberry Pi hardware to see which JVM is fastest.  For this I have created a few different tests that exercise specific operations, such as floating-point math or file IO.  The results are not relevant compared to other hardware.  The results metrics are really not important either other than as they compare to the same test on a different JVM.
+
{{Embedded}} {{Raspi}}
 +
 
 +
My objective is to performance test basic JVM functions of different JVMs on the Raspberry Pi hardware to see which JVM is fastest overall.  For this I have created a few different tests that exercise specific operations, such as floating-point math, file IO, etc.  The results are not relevant compared to other hardware.  The results metrics are really not important either other than as they compare to the same test on a different JVM.
 +
 
 +
* '''Also see [[Paul's Raspberry Pi Journal]]'''
 +
 
  
 
= VMs Tested =
 
= VMs Tested =
  
−
There's currently a few different JVMs available for Raspberry Pi.  I tested these
+
There's currently a few different JVMs available for Raspberry Pi.  I tested these:
 +
 
 +
* [http://www.oracle.com/technetwork/java/javase/downloads/index.html Oracle Java SE Development Kit 7u10 (Linux ARM v6/v7 Soft Float ABI)]
 +
 
 +
java version "1.7.0_10"
 +
Java(TM) SE Runtime Environment (build 1.7.0_10-b18)
 +
Java HotSpot(TM) Client VM (build 23.6-b04, mixed mode)
  
−
* Oracle ejre1.7.0_06
+
* [http://www.oracle.com/technetwork/java/embedded/downloads/javase/index.html Oracle Java SE Embedded version 6 Update 38 (ARMv6/7 Linux - Headless EABI, VFP, SoftFP ABI, Little Endian)]
  
 
  java version "1.7.0_06"
 
  java version "1.7.0_06"
Line 11: Line 22:
 
  Java HotSpot(TM) Embedded Client VM (build 23.2-b09, mixed mode)
 
  Java HotSpot(TM) Embedded Client VM (build 23.2-b09, mixed mode)
  
−
* java-7-openjdk-armel with -zero
+
* [http://icedtea.classpath.org/wiki/ZeroSharkFaq java-7-openjdk-armel with -zero]
  
 
  java version "1.7.0_03"
 
  java version "1.7.0_03"
Line 17: Line 28:
 
  OpenJDK Zero VM (build 22.0-b10, mixed mode)
 
  OpenJDK Zero VM (build 22.0-b10, mixed mode)
  
−
* java-7-openjdk-armel with -cacao
+
* [http://www.cacaojvm.org/ java-7-openjdk-armel with -cacao]
  
 
  java version "1.7.0_03"
 
  java version "1.7.0_03"
Line 23: Line 34:
 
  CACAO (build 1.1.0pre2, compiled mode)
 
  CACAO (build 1.1.0pre2, compiled mode)
  
−
* java-7-openjdk-armel with -jamvm
+
* [http://jamvm.sourceforge.net/ java-7-openjdk-armel with -jamvm]
  
 
  java version "1.7.0_03"
 
  java version "1.7.0_03"
 
  OpenJDK Runtime Environment (IcedTea7 2.1.3) (7u3-2.1.3-1)
 
  OpenJDK Runtime Environment (IcedTea7 2.1.3) (7u3-2.1.3-1)
 
  JamVM (build 1.6.0-devel, inline-threaded interpreter with stack-caching)
 
  JamVM (build 1.6.0-devel, inline-threaded interpreter with stack-caching)
 +
 +
* CONTROL was a MacBook Pro OSX Darwin Kernel Version 10.8.0
 +
** 2.4 GHz Intel Core 2 Duo
 +
** All tests we run with -d32 to force 32-bit mode
 +
 +
java version "1.6.0_37"
 +
Java(TM) SE Runtime Environment (build 1.6.0_37-b06-434-10M3909)
 +
Java HotSpot(TM) 64-Bit Server VM (build 20.12-b01-434, mixed mode)
 +
 +
= Results Summary =
 +
 +
* '''Oracle JDK VM looks to be the clear winner in term of overall performance'''
 +
* Oracle's VMs are fastest in the most categories.
 +
* The JDK _10 VM is slightly faster in most categories than the Embedded _06 JRE
 +
* Jam VM has impressive startup speed but nothing else
 +
* Zero was the slowest in most categories, but it's also the only OpenJDK VM that appears to be complete
 +
* Zero and Jamm were significantly slower in numerical calculations than the other 2
 +
* IO performance was the most dramatic win for the Oracle VMs.  IO is so much slower on the all the OpenJDK VMs that it signifigantly impacts Resin performance
 +
* Cacao was clearly the fastest OpenJDK VM, although it's not a "complete" JVM and throws some warnings
 +
* I am going to further test Cacao to see how Resin runs on it
  
 
= Startup Time =
 
= Startup Time =
 +
 +
<table border="0" width="100%" cellpadding="0" cellspacing="0">
 +
<tr>
 +
<td valign="top" width="650" align="left">[[Image:Pi-startup640.png]]</td>
 +
<td valign="top" align="left">
 +
 +
=== Tests ===
  
 
* '''HelloWorld''' is a basic print Hello World from a main method.  This tests basic startup time.
 
* '''HelloWorld''' is a basic print Hello World from a main method.  This tests basic startup time.
Line 35: Line 73:
 
* '''ResinStatus''' requests status from a running Resin instance.  The call fails as no instance was running, but that's not important.  It tests realistic Resin startup and network connection.
 
* '''ResinStatus''' requests status from a running Resin instance.  The call fails as no instance was running, but that's not important.  It tests realistic Resin startup and network connection.
  
−
JamVM is clearly the fastest to startup.  I believe this is due to the compactness.  However the caveat is both Cacao and JamVM report warning when running both Resin tests.   
+
=== Results ===
 +
 
 +
JamVM is clearly the fastest to startup but the margin is not huge.  I believe this is due to the smaller size of this JVM.  However the caveat is both Cacao and JamVM report warning when running both Resin tests.   
  
 
JamVM reports this:
 
JamVM reports this:
Line 52: Line 92:
  
 
These all look like warning associated with attempting to read MBean attributes from MBeans Resin is expecting to exist.  Resin gleans a lot of info about OS and Memory Pools from JMX.  I'll have to do more thorough Resin tests to see how adversely this effects Resin.   
 
These all look like warning associated with attempting to read MBean attributes from MBeans Resin is expecting to exist.  Resin gleans a lot of info about OS and Memory Pools from JMX.  I'll have to do more thorough Resin tests to see how adversely this effects Resin.   
 +
</td>
 +
</tr>
 +
</table>
 +
 +
= General Operations =
 +
 +
<table border="0" width="100%" cellpadding="0" cellspacing="0">
 +
<tr>
 +
<td valign="top" width="650" align="left">[[Image:Pi-operations640.png]]</td>
 +
<td valign="top" align="left">
 +
 +
=== Tests ===
 +
 +
* '''Exceptions''' tests throwing and catching exception
 +
* '''Hashing''' tests inserting into a map and retrieving from a map
 +
* '''Reflection''' tests looking up a class by name and creating a new instance with Reflection
 +
* '''Strings''' tests String creation and concatenation
 +
 +
=== Results ===
 +
 +
There's nothing extremely outstanding here, although Zero VM is showing up slow again which correlates with my initial real-world tests of Resin.  Cacao is looking good so far compared to the other 2 OpenJDK VMs.
 +
 +
</td>
 +
</tr>
 +
</table>
 +
 +
= Math =
 +
 +
<table border="0" width="100%" cellpadding="0" cellspacing="0">
 +
<tr>
 +
<td valign="top" width="650" align="left">[[Image:Pi-math640.png]]</td>
 +
<td valign="top" align="left">
 +
 +
=== Tests ===
 +
 +
* '''IntMath''' lots of calculations using int
 +
* '''LongMath''' lots of calculations using long
 +
* '''DoubleMath''' lots of calculations using double
 +
 +
=== Results ===
 +
 +
Clearly there's significant numerical operation slowness on Zero and JamVM and the difference is significant.  Again Cacao compares well with Oracle.  It's worth mentioning here that all of these are really small number.  For anything other than constant intensive numerical calculation, these results will not make much impact overall. 
 +
 +
</td>
 +
</tr>
 +
</table>
  
−
[[Image:Pi-startup.png]]
+
= IO =
  
−
== Generation Operations ==
+
<table border="0" width="100%" cellpadding="0" cellspacing="0">
 +
<tr>
 +
<td valign="top" width="650" align="left">[[Image:Pi-io640.png]]</td>
 +
<td valign="top" align="left">
  
−
[[Image:Pi-operations.png]]
+
=== Tests ===
  
−
== Math ==
+
* '''IO''' tests performed file create, write, read, delete
  
−
[[Image:Pi-math.png]]
+
=== Results ===
  
−
== IO ==
+
Out of all the tests I ran, I think this is the most significant.  Resin does quite a lot of IO and these difference here is not small.  This are big numbers also, in the order of seconds rather than milliseconds as in the case of the numerical calculations above.  I believe IO delays are the primary cause for the Resin slowness I'm experiencing.
  
−
[[Image:Pi-io.png]]
+
</td>
 +
</tr>
 +
</table>

Latest revision as of 00:00, 22 December 2012

Embed-48.pngPie48.png

My objective is to performance test basic JVM functions of different JVMs on the Raspberry Pi hardware to see which JVM is fastest overall. For this I have created a few different tests that exercise specific operations, such as floating-point math, file IO, etc. The results are not relevant compared to other hardware. The results metrics are really not important either other than as they compare to the same test on a different JVM.


Contents

[edit] VMs Tested

There's currently a few different JVMs available for Raspberry Pi. I tested these: 

java version "1.7.0_10"
Java(TM) SE Runtime Environment (build 1.7.0_10-b18)
Java HotSpot(TM) Client VM (build 23.6-b04, mixed mode)

java version "1.7.0_06"
Java(TM) SE Embedded Runtime Environment (build 1.7.0_06-b24, headless)
Java HotSpot(TM) Embedded Client VM (build 23.2-b09, mixed mode)

java version "1.7.0_03"
OpenJDK Runtime Environment (IcedTea7 2.1.3) (7u3-2.1.3-1)
OpenJDK Zero VM (build 22.0-b10, mixed mode)

java version "1.7.0_03"
IcedTea Runtime Environment (IcedTea7 2.1.3) (7u3-2.1.3-1)
CACAO (build 1.1.0pre2, compiled mode)

java version "1.7.0_03"
OpenJDK Runtime Environment (IcedTea7 2.1.3) (7u3-2.1.3-1)
JamVM (build 1.6.0-devel, inline-threaded interpreter with stack-caching)
  • CONTROL was a MacBook Pro OSX Darwin Kernel Version 10.8.0
    • 2.4 GHz Intel Core 2 Duo
    • All tests we run with -d32 to force 32-bit mode 
java version "1.6.0_37"
Java(TM) SE Runtime Environment (build 1.6.0_37-b06-434-10M3909)
Java HotSpot(TM) 64-Bit Server VM (build 20.12-b01-434, mixed mode)

[edit] Results Summary

  • Oracle JDK VM looks to be the clear winner in term of overall performance
  • Oracle's VMs are fastest in the most categories.
  • The JDK _10 VM is slightly faster in most categories than the Embedded _06 JRE
  • Jam VM has impressive startup speed but nothing else
  • Zero was the slowest in most categories, but it's also the only OpenJDK VM that appears to be complete
  • Zero and Jamm were significantly slower in numerical calculations than the other 2
  • IO performance was the most dramatic win for the Oracle VMs. IO is so much slower on the all the OpenJDK VMs that it signifigantly impacts Resin performance
  • Cacao was clearly the fastest OpenJDK VM, although it's not a "complete" JVM and throws some warnings
  • I am going to further test Cacao to see how Resin runs on it

[edit] Startup Time

Pi-startup640.png

[edit] Tests

  • HelloWorld is a basic print Hello World from a main method. This tests basic startup time.
  • ResinVersion requests Resin version. It tests basic Resin startup time.
  • ResinStatus requests status from a running Resin instance. The call fails as no instance was running, but that's not important. It tests realistic Resin startup and network connection.

[edit] Results

JamVM is clearly the fastest to startup but the margin is not huge. I believe this is due to the smaller size of this JVM. However the caveat is both Cacao and JamVM report warning when running both Resin tests.

JamVM reports this: 

jmm_GetBoolAttribute: Unknown attribute 24
jmm_GetBoolAttribute: Unknown attribute 25
jmm_GetInputArgumentArray

Cacao reports this: 

LOG: [0x400ec000] jmm_GetMemoryManagers: FIX ME!
LOG: [0x400ec000] jmm_GetMemoryPools: FIX ME!
LOG: [0x400ec000] jmm_GetBoolAttribute: Unknown attribute 24
LOG: [0x400ec000] jmm_GetBoolAttribute: Unknown attribute 25
LOG: [0x400ec000] jmm_GetInputArgumentArray: IMPLEMENT ME!

These all look like warning associated with attempting to read MBean attributes from MBeans Resin is expecting to exist. Resin gleans a lot of info about OS and Memory Pools from JMX. I'll have to do more thorough Resin tests to see how adversely this effects Resin.

[edit] General Operations

Pi-operations640.png

[edit] Tests

  • Exceptions tests throwing and catching exception
  • Hashing tests inserting into a map and retrieving from a map
  • Reflection tests looking up a class by name and creating a new instance with Reflection
  • Strings tests String creation and concatenation

[edit] Results

There's nothing extremely outstanding here, although Zero VM is showing up slow again which correlates with my initial real-world tests of Resin. Cacao is looking good so far compared to the other 2 OpenJDK VMs.

[edit] Math

Pi-math640.png

[edit] Tests

  • IntMath lots of calculations using int
  • LongMath lots of calculations using long
  • DoubleMath lots of calculations using double

[edit] Results

Clearly there's significant numerical operation slowness on Zero and JamVM and the difference is significant. Again Cacao compares well with Oracle. It's worth mentioning here that all of these are really small number. For anything other than constant intensive numerical calculation, these results will not make much impact overall.

[edit] IO

Pi-io640.png

[edit] Tests

  • IO tests performed file create, write, read, delete

[edit] Results

Out of all the tests I ran, I think this is the most significant. Resin does quite a lot of IO and these difference here is not small. This are big numbers also, in the order of seconds rather than milliseconds as in the case of the numerical calculations above. I believe IO delays are the primary cause for the Resin slowness I'm experiencing.

Retrieved from "http://wiki4.caucho.com/index.php?title=Java_On_Raspberry_Pi_Performance&oldid=969"
Personal tools
TOOLBOX
LANGUAGES