The units in SciMark 2.0 are Mflops (Millions of floating
point operations per second.) Some of the kernels excercise
transcendental functions (e.g.
as integer operations, so the Mflop count is only approximate.
However, the same Mflop count is used consistently, so that
comparisons are still valid.
Note that the original version of SciMark used a different metric, a unitless ratio of performance compared to a baseline platform, hence incompatible with SciMark 2.0 scores.
Yes. With version 2.0, the actual Java source code is available for download.
The Java applet and commandline version of SciMark check the LU and FFT results to make sure they are reasonable. If any of these fail, a bogus score of 0.0 is reported. This is in place to discourage posting scores which may have cheated on numerical computations.
If your results are 0.0, download the commandline verison and run it. This should report which kernel is failing. Please submit your results. These systems have somehow violated the JVM floating point model, and it is important to document this behavior.
To isolate the memory-hierarchy effects and focus on efficiency of JVM execution engine. The default problem sizes were designed to fit on most hardware caches.
Yes, a large version of SciMark 2.0 is available which uses bigger out-of-cache problem sizes. (This version is available for off-line execution only; see the download page more details.)
Most often, the underlying Java environment does not have a JIT (Just-In-Time) compiler. Make sure you have a recent version of the browser or Java envrironment in which SciMark is being run.
Currently, PCs have some of the better execution environments (and hence most of the high scores) but various projects are developing even better JITs/compilers for Java.
Sun Microsystem's JDK 1.2, with the compile line: "javac -O".
Some optimizations are done by hand, such as multidimensional array aliasing and loop unrolling, but the source codes could be further optimized. (Eventually, most of these should be performed by a good optimizing compiler.)
Because of background threads in Java environments, it is not unusual to see a +/- 5% variation in timing results. While this does not dramatically change the overall conclusions, keep this in mind when trying to split hairs. Typically, you should report the highest SciMark number, unless the performance is varying wildly --- this suggests you may have background processes interfering.
No. SciMark 2.0 focuses on single-processor performance. Multithreaded kernels may be included in future versions.
Commercial equipment and software is referred to on these pages in order to precisely document experimental measurements. Mention of these products does not imply recommendation or endorsement by the National Institute of Standards and Technology, nor does it imply that the products so identified are necessarily the best available for that purpose.