Poly/ML September 2008

polyml@lists.polyml.org

4 participants
2 discussions

Garbage collection issue with functional input streams
by pclayton＠taz.qinetiq.com 16 Sep '08

16 Sep '08

I have found a performance issue when using TextIO.StreamIO.input1 to read a functional stream. Looking at gc/non-gc times and using PolyML.profiling, it appears that garbage collection accounts for most of the time. There is some code below to demonstrate with stats that include comparison with SML/NJ. The profiling shows that readFromReader in basis/BasicStreamIO.sml is responsible for creating values that are being garbage collected. Looking at this code, I can see various things that would contribute to this garbage collection but nothing that is obviously problematic. Is it simply the case that overheads in the implementation mean that it is not suitable for a large number of small reads? Thanks, Phil (* `makeTextFile (filename, length)` generates a text file *) (* called `filename` of length `length` where every character *) (* is the digit '0'. *) local val l = 2048; val s = implode (List.tabulate (l, fn _ => #"0")); in fun makeTextFile (filename, length) = let open TextIO; val ostream = getOutstream (openOut filename); fun write n = if n < l then StreamIO.output (ostream, String.substring (s, 0, n)) else (StreamIO.output (ostream, s); write (n - l)); in write length; StreamIO.closeOut ostream end; end; (* `readTextFile filename` reads the file called `filename` *) (* one character at a time using a functional input stream *) (* and returns the time taken to read it. *) fun readTextFile filename = let open TextIO; val src = getInstream (openIn filename); fun read src = case StreamIO.input1 src of SOME (_, src') => read src' | NONE => (); val startTime = Timer.startCPUTimer (); in read src; (fn {sys, usr} => Time.+ (sys, usr)) (Timer.checkCPUTimer startTime) before StreamIO.closeIn src end; (* Create files and then read *) makeTextFile ("text-01MB", 1024 * 1024 * 1); makeTextFile ("text-02MB", 1024 * 1024 * 2); makeTextFile ("text-03MB", 1024 * 1024 * 3); makeTextFile ("text-04MB", 1024 * 1024 * 4); makeTextFile ("text-05MB", 1024 * 1024 * 5); readTextFile "text-01MB"; readTextFile "text-02MB"; readTextFile "text-03MB"; readTextFile "text-04MB"; readTextFile "text-05MB"; (* read times in seconds on Intel Core 2 Duo CPU T7500 @ 2.20GHz, 2GB *) (* Poly/ML 4.1.3 Poly/ML 5.2 SML/NJ 110.52 text-01MB 0.7 - 0.8 0.3 - 1.5 0.03 - 0.05 text-02MB 2.2 - 2.3 1.9 - 2.8 0.06 - 0.09 text-03MB 4.3 - 4.5 2.0 - 4.3 0.08 - 0.14 text-04MB 7.2 - 7.5 4.2 - 6.6 0.11 - 0.16 text-05MB 11.0 - 11.1 4.0 - 5.8 0.14 - 0.18 *) (* for large files, Poly/ML 4.1.3 can run out of store: *) makeTextFile ("text-40MB", 1024 * 1024 * 40); readTextFile "text-40MB"; (* Poly/ML 4.1.3 Poly/ML 5.2 SML/NJ 110.52 text-40MB out of store 101.4 1.1 - 1.2 4.1.3 gives the following message: Run out of store - interrupting console processes Exception- Interrupt raised *) The information contained in this E-Mail and any subsequent correspondence is private and is intended solely for the intended recipient(s). The information in this communication may be confidential and/or legally privileged. Nothing in this e-mail is intended to conclude a contract on behalf of QinetiQ or make QinetiQ subject to any other legally binding commitments, unless the e-mail contains an express statement to the contrary or incorporates a formal Purchase Order. For those other than the recipient any disclosure, copying, distribution, or any action taken or omitted to be taken in reliance on such information is prohibited and may be unlawful. Emails and other electronic communication with QinetiQ may be monitored and recorded for business purposes including security, audit and archival purposes. Any response to this email indicates consent to this. Telephone calls to QinetiQ may be monitored or recorded for quality control, security and other business purposes. QinetiQ Limited Registered in England & Wales: Company Number:3796233 Registered office: 85 Buckingham Gate, London SW1E 6PD, United Kingdom Trading address: Cody Technology Park, Cody Building, Ively Road, Farnborough, Hampshire, GU14 0LX, United Kingdom http://www.QinetiQ.com/home/legal.html

3 4

Re: Poly/ML 5.2 and signals on older Linux systems
by David.Matthews＠prolingua.co.uk 09 Sep '08

09 Sep '08

Rob Arthan wrote: > David, > However, while I agree that my "fix" is a hack because it's working around > what looks like a bug in the old glibc, I am a little concerned that your > signal handling design is not POSIX-conformant. The POSIX specification of > sigaction includes the following statement: > > "The result of the use of sigaction() and a sigwait() function concurrently > within a process on the same signal is unspecified." > > I take this to mean that if you change the disposition of a signal using > sigaction() while you are waiting for it with sigwait() then the result is > unspecified. So the Cygwin behaviour would be conformant. The documentation > on signal() doesn't say anything about sigwait(), but it does say: > > "Use of this function is unspecified in a multi-threaded process." > > I think the POSIX-compliant way of implementing the ML signal function would > be to service ML requests to change the disposition of a signal by waking up > the signal detection thread and having it call sigaction. Thanks for pointing this out. I've also had another look at the documentation for sigwait and discovered that the Cygwin behaviour might be acceptable, not because of the interaction with sigaction but because the behaviour of sigwait when a signal is not blocked is undefined. As a result of all this I've revisited the whole mechanism for signal handling. The difficulty is that the pthread mutex and condition variable functions are not safe when used in a signal handler but it turns out that POSIX sem_post is safe. I've reworked the code to use a signal handler rather than sigwait and this appears to work on all the platforms I've tried, with the inevitable tweaks for the peculiarities of semaphores on Mac OS X and Cygwin. The new version is in CVS so you can try it out if you wish. I still have to get to the bottom of the other bug. Regards, David Copied to the mailing list for information.

2 2

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

Poly/ML September 2008