On Fri, Jul 26, 2013 at 6:07 AM, Jesus Aransay < jesus-maria.aransay at unirioja.es> wrote:
The matrices used for the benchmarks are in a zip file (9MB compressed, 160MB uncompressed) in the link "Download the benchmarks" in the previous website. Uncompressing the folder three different folders appear (RAT, REAL, Z2). In each of these folders there is a "Readme" file where we describe how we carried out the computations (we would be very glad to hear if there is a more proper way, both in MLton and Poly/ML).
Jesus,
Thanks for the link to the benchmarks (and sorry I didn't find them by following the links in your paper).
I wasn't expecting your benchmarks to "load" the matrices by including a (very large) SML expression in the program. For MLton, at least, this explains the large "processing" times --- which are really "compile" times. MLton isn't doing very well with this very large constant data; actually, it seems that one of the intermediate-representation type checkers is very slow on this kind of large constant data, as are a couple of optimization passes. I was expecting that the matrices would be loaded by opening and parsing a text or binary file, in which case the "processing" time would have been the I/O time. Note that by including the matrices in the program as static data, a compiler could choose to completely evaluate the Gauss-Jordan elimination at compile-time.
For MLton, compiling with "-profile time" incurs some cost; some optimizations are not as effective in the presence of profiling and there will be a runtime cost to sampling the program during execution. It would be better to simply use "time" to measure the run time, as you do to measure the compile time. I don't know if "PolyML.timing true" incurs a significant overhead. Another option would be to use the SML "Timer" and/or "Time" structures to do the timing; that might have the experimental advantage of using the same code to measure the time for both MLton and Poly/ML.
-Matthew