[Home] [Version 2] [Run it Now] [Download] [Results] [FAQ] [About] [Resources] [Credits]

Frequently Asked Questions

• What units are SciMark scores in?

  The units in SciMark 2.0 are Mflops (Millions of floating point operations per second.) Some of the kernels excercise transcendental functions (e.g.sin, cos) as well as integer operations, so the Mflop count is only approximate. However, the same Mflop count is used consistently, so that comparisons are still valid.

  Note that the original version of SciMark used a different metric, a unitless ratio of performance compared to a baseline platform, hence incompatible with SciMark 2.0 scores.

• Is the source code for SciMark available?

  Yes. With version 2.0, the actual Java source code is available for download.

• My SciMark results are 0.0 Mflops. What happened?

  The Java applet and commandline version of SciMark check the LU and FFT results to make sure they are reasonable. If any of these fail, a bogus score of 0.0 is reported. This is in place to discourage posting scores which may have cheated on numerical computations.

  If your results are 0.0, download the commandline verison and run it. This should report which kernel is failing. Please submit your results. These systems have somehow violated the JVM floating point model, and it is important to document this behavior.

• Why are the (default) problem sizes so small?

  To isolate the memory-hierarchy effects and focus on efficiency of JVM execution engine. The default problem sizes were designed to fit on most hardware caches.

• Is there a version which which uses larger problem sizes?

  Yes, a large version of SciMark 2.0 is available which uses bigger out-of-cache problem sizes. (This version is available for off-line execution only; see the download page more details.)

• Why is my system so slow?

  Most often, the underlying Java environment does not have a JIT (Just-In-Time) compiler. Make sure you have a recent version of the browser or Java envrironment in which SciMark is being run.

  Currently, PCs have some of the better execution environments (and hence most of the high scores) but various projects are developing even better JITs/compilers for Java.

• What compiler was used to generate the SciMark bytecodes?

  Sun Microsystem's JDK 1.2, with the compile line: "javac -O".

• Can the original source code be further optimizied?

  Some optimizations are done by hand, such as multidimensional array aliasing and loop unrolling, but the source codes could be further optimized. (Eventually, most of these should be performed by a good optimizing compiler.)

• My timing results vary from run to run. Which numbers do I report?

  Because of background threads in Java environments, it is not unusual to see a +/- 5% variation in timing results. While this does not dramatically change the overall conclusions, keep this in mind when trying to split hairs. Typically, you should report the highest SciMark number, unless the performance is varying wildly --- this suggests you may have background processes interfering.

• Are the SciMark kernels multithreaded?

  No. SciMark 2.0 focuses on single-processor performance. Multithreaded kernels may be included in future versions.

Commercial equipment and software is referred to on these pages in order to precisely document experimental measurements. Mention of these products does not imply recommendation or endorsement by the National Institute of Standards and Technology, nor does it imply that the products so identified are necessarily the best available for that purpose.