On Tue, 2013-07-30 at 13:39 +0100, David Matthews wrote:
A major difficulty with measuring timing when the runs are only a few seconds is that garbage-collection can have a significant effect. Depending on what has happened before the run starts it is possible that a GC may happen during the run, distorting the figures. With short runs there will probably be only a single GC if it happens at all. Runs of only a few seconds are a useful indication but to get consistent figures it's probably necessary for a run to take several minutes and involve multiple garbage collections.
It may be worth calling "PolyML.fullGC ()" immediately before short benchmarks. Together with suitable memory parameters, this could perhaps avoid GC during short runs.
Best, Tjark