Poly/ML October 2016

polyml@lists.polyml.org

7 participants
7 discussions

linking polyML modules to C
by dtopham＠gmail.com 09 Nov '18

09 Nov '18

After more experimenting with GUI in SML, I would like to now try calling an SML module from a C program. That way, the GUI could be built using traditional imperative techniques and the program logic could be written in SML. I see how to call a C function from SML, but and not so clear on how to call an SML function from a C program. I want to build code using polyc and then link with a C main program, but could anyone show me a simple example of calling that function and getting its return value? for example, I have defined: fun sum i n f = if i > n then 0.0 else f i + sum (i+1) n f ; fun int2real x = real x ; in linkToC.sml and #include <iostream> #include <fstream> std::ofstream ofs; int main() { std::cout << sum(1,10,int2real) << std::endl; ofs.open("table.dat"); for(int i = 0; i < 20; i++) ofs << i << '\t' << sum(1,i,int2real) << std::endl; } in linkToC.cpp Looking in the polyc script, I thought adding the -c option might allow me to build the object code which I could then link to the sml functions, but it complains about not having a main. Could someone point me in the right direction? Thanks, Dave On Mon, Dec 15, 2014 at 11:44 AM, David Topham <dtopham at gmail.com> wrote: > David, Thank you so much! I really appreciate your help with this. Your > sample below builds fine with polyc on my system (PolyML 5.5.2). I am using > a minimal Linux distribution named TinyCore ( > http://distro.ibiblio.org/tinycorelinux/) which comes with OpenMotif > 2.3.3 It runs in a virtual machine within another computer only taking up > a few hundred MBs! > > I am going to use it to help my students explore Discrete Math next > semester (this textbook uses SML for that purpose: > http://cs.wheaton.edu/~tvandrun/dmfp/). > > I also found a reasonably current project using PolyML and Motif here: > http://www.lemma-one.com/ProofPower/index/ > > ...however it was built on PolyML 4 and has not yet been ported to the > newer version (e.g. depends on PolyML.commit which no longer seems to be > supported). > > Happy Holidays to everyone in the PolyML community! > > -David Topham > > >> Re: GUI Interface (David Matthews) >> >> I've experimented with building a stand-alone executable with Motif and >> succeeded in getting it to work for me using polyc. (It required >> building Poly/ML with --with-x). I took the example from >> http://www.polyml.org/docs/Motif.html and wrapped it up in a function. >> I did find a problem, though. It looks as though the function has to >> suspend itself with something like Posix.Process.pause otherwise nothing >> happens. I seem to recall that the Motif stuff is handled on a separate >> thread to allow the REPL to continue to accept commands. With a >> stand-alone application there isn't a REPL so without the "pause" it >> terminates immediately. >> >> David >> >> open XWindows ; >> open Motif ; >> >> fun main() = >> let >> val shell = >> XtAppInitialise "" "xed" "Editor" [] [XmNwidth 400, XmNheight 400] ; >> >> val main = XmCreateMainWindow shell "main" [] ; >> >> val bar = XmCreateMenuBar main "bar" [] ; >> >> val fileMenu = XmCreateCascadeButton bar "file" [XmNlabelString "File"] ; >> val editMenu = XmCreateCascadeButton bar "edit" [XmNlabelString "Edit"] ; >> val viewMenu = XmCreateCascadeButton bar "view" [XmNlabelString "View"] ; >> val helpMenu = XmCreateCascadeButton bar "help" [XmNlabelString "Help"] ; >> >> val command = XmCreateText main "command" [XmNeditMode >> XmSINGLE_LINE_EDIT] ; >> >> val hscroll = XmCreateScrollBar main "hscroll" [XmNorientation >> XmHORIZONTAL] ; >> val vscroll = XmCreateScrollBar main "vscroll" [XmNorientation >> XmVERTICAL] ; >> >> val work = XmCreateDrawingArea main "work" [] ; >> in >> >> XtManageChildren [fileMenu, editMenu, viewMenu, helpMenu] ; >> XtManageChildren [bar, command, hscroll, vscroll, work] ; >> >> XmMainWindowSetAreas main bar command hscroll vscroll work ; >> >> XtManageChild main ; >> XtRealizeWidget shell; >> Posix.Process.pause() >> end; >> >> >> ------------------------------ >> >> _______________________________________________ >> polyml mailing list >> polyml at inf.ed.ac.uk >> http://lists.inf.ed.ac.uk/mailman/listinfo/polyml >> >> End of polyml Digest, Vol 110, Issue 12 >> *************************************** >> >> -- >> The University of Edinburgh is a charitable body, registered in >> Scotland, with registration number SC005336. >> >>

2 3

ConditionVar
by kostirya＠gmail.com 24 Oct '16

24 Oct '16

Hello. Tell me, please, how to use the Thread and ConditionVar. The following code unpredictably hangs up... open Thread open Thread Mutex ConditionVar val cv = conditionVar() and m = mutex(); fun foo() = ( print "OK\n"; signal cv ); fork(foo, [EnableBroadcastInterrupt true, InterruptState InterruptSynch]); wait(cv, m); print "The End\n"; Nick. P.S. I use fixes-5.6 git version PolyML.

2 1

Update to code-generator and run-time system interface
by David.Matthews＠prolingua.co.uk 23 Oct '16

23 Oct '16

I have now pushed a major update to git master which is the result of work going back to the beginning of the year. It covers a number of areas but largely the code-generator and the run-time system interface. The basic design of much of the low-level parts of Poly/ML dated back to the early days. While it has changed a lot in detail the overall structure has remained the same. The idea was to bring the whole thing up to date. The run-time system interface has been redesigned and instead of a vector of entries to the run-time system the ML code now uses a form of foreign-function interface to call individual functions. The advantage is that it is much easier to add new functions. In addition when building an executable it is possible for the linker to select only the parts of libpolyml that are actually required for the application, at least if the static library version is used. It should be possible to adapt the foreign-function interface it uses to speed up calls made using the Foreign structure, although that's not done at the moment. The code-generator has been rewritten particularly the register assignment. The previous version had been the result of a series of adaptations to new architectures over the years. The new version focusses solely on the X86. I'm still working on this. Doing this has needed a temporary, slow, code-generator which is why it has taken until now to push this to git master. The representation of strings has been simplified. Previously, single character strings were represented by the character itself as a tagged value. This was largely a relic of SML 90 which didn't have a separate char type. That has been removed and all strings are now represented as a vector of bytes. This speeds up string operations since the code no longer has to consider the special case. SSE2 instructions are now used for floating point on the X86/64. Floating point support in the new code-generator is rudimentary at the moment and not to the same extent as the previous code-generator. The handling of return addresses from functions has been simplified. A consequence of this is that when pieces of code are compiled they are stored in a separate area of memory rather than in the general heap and not garbage-collected. It is no longer possible to get an exception trace so PolyML.exception_trace has no effect. The debugger handles this much better anyway. Although the focus has been on the X86 the portable byte-code interpreter has been improved and is significantly faster. The system has had some basic testing but there are bound to be bugs in something as complex as this. I'm also continuing to work on improvements. When testing this it is essential to run "make compiler" at least once and generally twice to build the new compiler and then build the compiler itself with the new compiler. David

4 32

Update to code-generator and run-time system interface
by David.Matthews＠prolingua.co.uk 19 Oct '16

19 Oct '16

Phil, Thanks for your comments. References from the heap to the start of a code cell are not too much of a problem. The difficulty is that some code cannot be garbage collected because it may still be running. Consider: fun f () = (OS.Process.sleep(Time.fromSeconds 60); print "Done\n"); Thread.Thread.fork(f, []); fun f () = print "A new function"; We would like to be able to garbage-collect the code associated with the first definition of "f" after the second definition is made. But it's not possible to do that until the thread has finished. The only reference to the code is through the return address from the sleep function so if we're going to try to garbage collect code we need the GC to be able to find and recognise the return address as a valid reference. The new mechanism for handling pieces of code deals with most of the issues apart from the question of garbage collection. I was really trying to get a feeling for how serious this was. From the comments on the mailing list it looks as though this is something that is wanted so I'll try and see if I can find a solution. David On 19/10/2016 09:09, Phil Clayton wrote: > Hi David, > > Almost certainly I don't understand the full picture here. Still... I > wondered if each code cell could have a unique 'handle' in the heap > (that just contains a pointer to the code cell). All code would be > called via its handle to ensure the handle is referenced wherever the > code is referenced. When the handle is garbage collected, the code can > be removed (the next time the code area is made writable). The > top-level, like other code, would have references to the handles. > > Whatever approach is taken, there seems to be an inherent conflict > between using functions as first class values and separation of code and > data for security purposes. > > Phil > > On 18/10/2016 15:13, David Matthews wrote: >> On 18/10/2016 13:43, Makarius wrote: >>> On 17/10/16 23:58, David Matthews wrote: >>>> >>>> Although the lack of garbage collection of code would mean that >>>> repeatedly defining the same function would be a memory leak I would be >>>> surprised if it was a serious problem. Is it likely that one would >>>> repeatedly redefine the same function within a particular session? >>> >>> This happens all the time in IDE interaction: things are compiled, >>> edited, re-compiled; thus old this become inaccessible. >>> >>> You introduced that principle yourself many years ago, by providing the >>> very nice PolyML.Compiler interface. >>> >>> That is one of the big assets of Poly/ML and consequently of >>> Isabelle/ML. >> >> Yes, but the actual memory needed for the function code is not going to >> be large compared with the total heap size. We have heaps in the orders >> of gigabytes but the whole of Isabelle is just tens of megabytes. >> >> Perhaps I should explain why I made this change and what could be done >> to mitigate the effects. There were two reasons. The first was to >> simplify the code and avoid the contortions that were necessary and the >> second was for security and long-term stability. >> >> For the garbage collector to be able to compact the heap it has to be >> able to find and modify all the addresses of heap cells. To do that >> values are distinguished by a tag bit. If the bottom bit is set the >> value is an integer and is ignored by the GC. Other values are >> addresses. An address points always to the start of a cell so will be >> word aligned, either XXXX00 (32-bit) or XXX000 (64-bit) in the bottom >> bits. Before the start of a cell is a word containing the length of the >> cell and some bits that indicate whether the cell is a tuple/vector >> containing values (i.e. either tagged integers or addresses) or is byte >> data, typically a string. >> >> This is extended to cope with cells containing machine code. This is >> just another type of cell. A code cell is not quite byte data because >> it can contain addresses if there are values that are compile-time >> constants. >> >> Provided we're dealing with the entry point addresses of code cells this >> all works fine. The complication comes when the code is actually >> executed. If one function calls another, or even itself recursively, it >> uses the X86 CALL instruction. It is important to use this instruction >> and not try to do the function call any other way because among other >> things the prefetching hardware recognises CALL/RET pairs. The CALL >> instruction pushes the return address, the address of the next >> instruction, to the stack. >> >> This, though, causes problems for the GC. Return addresses are >> inherently addresses into the middle of cells. They are also on an >> arbitrary alignment since X86 instructions are not aligned in any way. >> For the GC to be able to compact the heap it has to be able to find and >> update the return addresses. If Poly/ML used "stack frames" it might be >> possible to find return addresses using the frame pointer register but >> using frame pointers requires an extra register and increases the cost >> of every function call. Instead the code-generator added no-op >> instructions before each CALL such that the return address after the >> instruction was on a word+2 byte alignment i.e. XXX10 in the bottom >> bits. This is neither an integer nor a word address so the GC can >> recognise these as return addresses. There is still the problem of >> finding the actual start of the code cell and to do this there is a zero >> word at the end of each code cell. That requires that the >> code-generator never generates a full word of zeros anywhere else in the >> code, or at least not on a word boundary, so there are a few constraints >> on the code to ensure that is the case. >> >> Removing the code from the normal heap and using a separate, non-garbage >> collected area means that these contortions are no longer needed. The >> length word is retained since it is needed when copying the code to an >> object file or saved state and when loading from a saved state. >> >> The other reason for making the change is that having the code cells in >> the normal heap requires the heap to be given read, write and execute >> permissions. This is a problem for security and I was concerned that a >> future operating system update might ban the use of both write and >> execute permissions on the same area of memory. I think this is already >> an issue with SELinux. Using a separate "code" area avoids this. >> Although the code area needs to be writable to add new code cells to it >> there are tricks that can be used to get round this. >> >> It might be possible to use a non-compacting GC on the code area i.e. to >> mark code cells that are no longer reachable and then reuse the space. >> That can lead to fragmentation but would reduce the memory leak. In >> almost all cases a code cell that is reachable will have at least one >> address in the heap that points at the start. It is, though, possible >> to construct pathological cases where the only reference is through a >> return address. We're not going to change the return address since >> we're not compacting so it might be possible to get round that by >> assuming that any value on the stack that looks like it might be a >> return address is a sufficient reason to keep the code cell, even if it >> is actually an integer. >> >> Sorry that's been rather long but I wanted to put on record the thinking >> behind the change and maybe get some other ideas. >> >> David >> _______________________________________________ >> polyml mailing list >> polyml at inf.ed.ac.uk >> http://lists.inf.ed.ac.uk/mailman/listinfo/polyml >>

1 0

Can't unify int (*In Basis*) with LargeInt.int ...
by David.Matthews＠prolingua.co.uk 15 Oct '16

15 Oct '16

I think Mlton uses the same precision for C int and ML int so there won't be the issue of an overflow exception when passing values from C to ML. That's easier to do with a full program compiler than the incremental compilation of Poly/ML. David On 10/10/2016 15:40, Kostirya wrote: > I think it should "to turn these conversions back to use FixedInt.int and > raise an exception". > It's important to preserve full compatibility with MLton, SML/NY, SML# ... > > MLton: > val plus = _import "plus": (int * int) -> int; > val r = 0 + plus(1, 2) > val _ = print ( (Int.toString(r) ^ "\n" ) ) > > New Poly/ML: > val plus = buildCall2 ((getSymbol mylib "plus"), (cInt, cInt), cInt) > val r = 0 + plus(1, 2) > val _ = print ( (LargeInt.toString(r) ^ "\n" ) ) > > The difference in SML code has already appeared in: > > MLton: val _ = print ( (Int.toString(r) ^ "\n" ) ) > PolyML: val _ = print ( (LargeInt.toString(r) ^ "\n" ) ) > > > 2016-10-10 16:02 GMT+03:00 David Matthews <David.Matthews at prolingua.co.uk>: > >> I noticed this and thought I'd committed a fix but it seems not. The >> issue is that the C "int" type may have, depending on the platform, one >> more bit of precision than FixedInt.int in Poly/ML. >> >> Now that the default int in Poly/ML is FixedInt.int that raises a question >> about how to deal with the various conversions (e.g. cInt) that could >> return a value that would overflow. Currently, c(U)Int8 and 16 have type >> "int conversion" and the rest, c(U)Int32/64 and c(U)Int have type >> "LargeInt.int conversion". While this ensures that there will never be an >> overflow when converting C values to ML it does mean there's a mismatch >> between C "int" and ML "int". I'm wondering whether to turn these >> conversions back to use FixedInt.int and raise an exception when a value >> will not fit and perhaps introduce new conversions (e.g. cIntLarge) to >> handle the full range of values. >> >> Does anyone have any comments? >> David >> >> >> On 10/10/2016 13:18, Kostirya wrote: >> >>> Hello. >>> I updated Poly/ML from git (commit >>> aa7bea141f227ea5d7feaa849763ad53fb4147d3) and found that >>> mlsource/extra/CInterface/Examples/NewForeignTest.sml is broken: >>> >>> env LD_LIBRARY_PATH=. poly --script NewForeignTest.sml >>>> >>> (4, 4) >>> (5, 5) >>> (12345, ...) >>> NewForeignTest.sml:133: error: Type error in function application. >>> Function: buildClosure1 : >>> (int -> unit) * int conversion * unit conversion -> >>> (int -> unit) closure >>> Argument: (fn i => print (Int.toString i), cInt, cVoid) : >>> (int -> unit) * LargeInt.int conversion * unit conversion >>> Reason: >>> Can't unify int (*In Basis*) with LargeInt.int (*In Basis*) >>> (Different type constructors) >>> Found near buildClosure1 (fn i => print (... ...), cInt, cVoid) >>> >>> Is LargeInt.int a default int type for FFI now? >>> >>> Nick. >>> >>> >>> >>> _______________________________________________ >>> polyml mailing list >>> polyml at inf.ed.ac.uk >>> http://lists.inf.ed.ac.uk/mailman/listinfo/polyml >>> >>> _______________________________________________ >> polyml mailing list >> polyml at inf.ed.ac.uk >> http://lists.inf.ed.ac.uk/mailman/listinfo/polyml >> >

2 2

sets of sets
by dtopham＠ohlone.edu 11 Oct '16

11 Oct '16

I doubt if I am the first person to ask this question, but I have not seen an "answer" in any of the ML books I have read yet... Is there a way to represent a "set of sets" so that a function can process a query such as " is [2] an element of [ 3 , [2], 4, 5]?" I know we can easily do a query such as " is 2 an element of [ 3 , 2, 4, 5]?" I know the problem is that lists cannot have different types, but this is a common, normal query in mathematics so someone must have figured out a way to do it perhaps? -Dave

2 1

Can't unify int (*In Basis*) with LargeInt.int ...
by kostirya＠gmail.com 10 Oct '16

10 Oct '16

Hello. I updated Poly/ML from git (commit aa7bea141f227ea5d7feaa849763ad53fb4147d3) and found that mlsource/extra/CInterface/Examples/NewForeignTest.sml is broken: > env LD_LIBRARY_PATH=. poly --script NewForeignTest.sml (4, 4) (5, 5) (12345, ...) NewForeignTest.sml:133: error: Type error in function application. Function: buildClosure1 : (int -> unit) * int conversion * unit conversion -> (int -> unit) closure Argument: (fn i => print (Int.toString i), cInt, cVoid) : (int -> unit) * LargeInt.int conversion * unit conversion Reason: Can't unify int (*In Basis*) with LargeInt.int (*In Basis*) (Different type constructors) Found near buildClosure1 (fn i => print (... ...), cInt, cVoid) Is LargeInt.int a default int type for FFI now? Nick.

2 1

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

Poly/ML October 2016