I have finally tried out the new FFI with Giraffe Library with partial success. For some examples, calls to C and callbacks from C are working but other examples result a seg. fault. From the debug output, I noticed that the seg. faults occur when a callback occurs during a callback. I've attached a small example that demonstrates the issue (call_c_test_16.tar.gz). (Although this example uses dynamic loading, the same happens if dynamic linking is used.) The backtrace from gdb provided no useful information so I didn't investigate further.
Also, I have some minor observations about the interface. I note that the signature FOREIGN specifies: val touchClosure: 'a -> unit I wondered whether that should be val touchClosure: 'a closure -> unit (RunCall.touch is visible in the Poly/ML top-level which has the type of the former, so there is no loss of capability with the latter, which would catch cases where touchClosure is applied to the wrong value.)
In both the old and new Foreign modules, the type `'a Foreign.closure` is abstract. Giraffe Library uses `Foreign.LowLevel.cFunctionWithAbi` define its own function for creating a closure but there is no way to create a `'a Foreign.closure` value from a `Memory.voidStar` value. This is easily worked around by copying the type declaration and definition of `Foreign.cFunction` but I wondered if there could be a way to avoid this copying.
In the past, I found it useful to have val nullClosure : 'a closure This is easily declared with one's own closure type but if using Foreign.closure, it may be useful to have in Foreign. Giraffe Library no longer needs this since I changed the callback mechanism to avoid the need to free closures, not realizing this would become available a month or two later! https://github.com/giraffelibrary/giraffe/commit/2dc239946c77bdf8cb8b55223f9...
Regards, Phil
On 19/10/20 12:12, David Matthews wrote:
Hi, I've just pushed a collection of changes to master that have been in the pipeline for quite a long time.? Some of these are internal changes to the run-time system and some are extensions, such as the addition of IPv6 networking with INet6Sock and Net6HostDB structures.
The major change, though, is with the foreign function interface.? On X86 platforms libffi is no longer used and the version of it included in the libpolyml directory has been removed.? Libffi is still used in the interpreted version but only if the library is installed on the system.
Instead the foreign function interface is handled essentially as part of the compiler.? The high-level interface in the Foreign structure remains unchanged but the buildCallN functions now actually compile interface functions.? This results in foreign function calls being substantially faster than with libffi; at least 10 times faster for trivial calls on the X86/64.? The cost is, of course, some extra work when buildCallN is called, meaning that it is essential that these functions are only used at the top level.
The reason for the speed-up is that the interface has to place the arguments in the correct registers for the ABI and the rules for placing arguments can be quite complicated, particular on the X86/64 on Unix. Libffi computes the placement on every call whereas the compiler can do this once and build code that moves the arguments into the right registers and returns the result.
For backwards compatibility buildClosureN functions have been retained but these are wrappers around new buildCallback functions.? The buildCallback functions differ in two respects.? Closures created with buildCallback are garbage-collected which means that if they are used to register callbacks with a C library it may be necessary to keep a reference in ML.? There is a touchClosure function that should be called when the callback is no longer needed.? For compatibility closures created with buildClosure are retained in a global list to avoid garbage collection.
The other difference is that the buildCallback functions have a slightly different type from buildClosure and that reflects the underlying implementation.? For example, val buildCallback1: 'a conversion * 'b conversion -> ??? ('a -> 'b) -> ??????? ('a -> 'b) closure The first application builds the interface code that handle the conversion between the ML and C ABIs.? The second application applies this to an ML function to build a closure value that can be passed to C. ?This second application builds a small additional piece of code that simply loads the address of the ML function into a register and jumps to the interface code.? What this means is that while the first application should always be done at the top level it is possible to embed an application of this to a particular ML function inside ML code.? There is still an overhead compared with creating a closure in ML and it is better if possible to do the application at the top level but the cost is significantly less than if the whole buildCallback function were called within a function.
As always please give this a try and let me know if there are problems. Regards, David _______________________________________________ polyml mailing list polyml at inf.ed.ac.uk http://lists.inf.ed.ac.uk/mailman/listinfo/polyml