Thanks for sorting that out - that has certainly solved the problem!
I have rerun the tests using the latest CVS snapshot and the results are below.
Phil
(* read times in seconds on Intel Core 2 Duo CPU T7500 @ 2.20GHz, 2GB *) (* Poly/ML 4.1.3 Poly/ML 5.2 SML/NJ 110.52 Poly/ML latest
text-01MB 0.7 - 0.8 0.3 - 1.5 0.03 - 0.05 0.035 - 0.075 text-02MB 2.2 - 2.3 1.9 - 2.8 0.06 - 0.09 0.067 - 0.096 text-03MB 4.3 - 4.5 2.0 - 4.3 0.08 - 0.14 0.107 - 0.147 text-04MB 7.2 - 7.5 4.2 - 6.6 0.11 - 0.16 0.135 - 0.170 text-05MB 11.0 - 11.1 4.0 - 5.8 0.14 - 0.18 0.191 - 0.233
text-40MB out of store 101.4 1.1 - 1.2 1.54 - 1.57
*)
David Matthews wrote:
Philip Clayton wrote:
I have found a performance issue when using TextIO.StreamIO.input1 to read a functional stream. Looking at gc/non-gc times and using PolyML.profiling, it appears that garbage collection accounts for most of the time. There is some code below to demonstrate with stats that include comparison with SML/NJ.
The profiling shows that readFromReader in basis/BasicStreamIO.sml is responsible for creating values that are being garbage collected. Looking at this code, I can see various things that would contribute to this garbage collection but nothing that is obviously problematic. Is it simply the case that overheads in the implementation mean that it is not suitable for a large number of small reads?
I've finished redoing the functional IO code and I think I've eliminated most of the inefficiencies. The new version is now in CVS. Could you rerun your tests and let me know if it all works satisfactorily?
The new version is actually faster than using TextIO.input1 because TextIO.input1 has to lock a mutex to be thread-safe whereas TextIO.StreamIO.input1, being functional, only needs to lock a mutex when reading a block.
David
The information contained in this E-Mail and any subsequent correspondence is private and is intended solely for the intended recipient(s). The information in this communication may be confidential and/or legally privileged. Nothing in this e-mail is intended to conclude a contract on behalf of QinetiQ or make QinetiQ subject to any other legally binding commitments, unless the e-mail contains an express statement to the contrary or incorporates a formal Purchase Order.
For those other than the recipient any disclosure, copying, distribution, or any action taken or omitted to be taken in reliance on such information is prohibited and may be unlawful.
Emails and other electronic communication with QinetiQ may be monitored and recorded for business purposes including security, audit and archival purposes. Any response to this email indicates consent to this.
Telephone calls to QinetiQ may be monitored or recorded for quality control, security and other business purposes.
QinetiQ Limited Registered in England & Wales: Company Number:3796233 Registered office: 85 Buckingham Gate, London SW1E 6PD, United Kingdom Trading address: Cody Technology Park, Cody Building, Ively Road, Farnborough, Hampshire, GU14 0LX, United Kingdom http://www.qinetiq.com/home/notices/legal.html