[PPL-devel] GMP memory allocation problem in SWI-Prolog 5.6.38 and following versions

[Roberto Bagnara]

Jan Wielemaker wrote:
> Roberto,
> 
> On Tuesday 02 October 2007 10:25, Roberto Bagnara wrote:
>> Jan Wielemaker wrote:
>>> Rebinding the allocation functions is done to allow SWI-Prolog to detect
>>> and act upon GMP memory overflows. Unfortunately GMP is very weak in
>>> allowing applications to act gracefully on overflows. I see no reason
>>> for problems as long as you initialize Prolog before touching any GMP
>>> number.
>> This is not always possible, unfortunately.
> 
> Most of the time it is, certainly with open source :-)

True, but having to change code just for that...
Think about global variables.  If you have something like

mpz_class n;

in C++ code I don't know a reliable way to initialize Prolog before the
constructor is run.

>>> If that should be the case anyway, we must add something to the
>>> foreign interface that allows for initializing Prolog without rebinding
>>> the GMP allocation.
>> This is certainly useful.  But what happens if, in some computation in
>> the C++ code that has nothing to do with SWI-Prolog,  pl_abort() is
>> called (I am looking at pl-gmp.c)?
> 
> Actually it never calls pl_abort(), unless something is really wrong. It
> does call longjmp() back to the Prolog context :-) Note that it only
> does so if it cannot allocate.

Yes. But I think we disagree with our definition of "really wrong."
The PPL is exception-safe, meaning that the application can be programmed
in a way that, whenever allocation fails (something that does happen
when working with [big] collections of [big] polyhedra) all the memory
used by the failed (sub)computation is automatically discarded, a simplification
takes place, and computation restart.  This is only possible if the
overall application is the only entity allowed to change GMP's allocation
function.

> The only thing you are allowed to do from
> the GMP allocation functions is stop the process.

This is not my understanding: where did you get this restriction from?

> Prolog knows a few
> cases where it can safely do better (discussed on the GMP mailinglist),
> by throwing a Prolog resource_error exception.

I am not contending this.  What I am saying is that in a complex application,
Prolog can be one of several components all using GMP.  Thus, the decision
about which allocation functions are adequate can only be taken at the
application level, not at the component level.

> True. Same goes for signals and a few more of these things. The thing
> is, the way Prolog uses GMP, it *can* deal with GMP caused memory
> overflows. This can be exploited as long as Prolog is the only component
> in the application using GMP or at least it is possible to initialise
> Prolog GMP handling before GMP numbers are used elsewhere.  This deals
> with most applications.

OK: we agree on that.

> Anyway for the others, I added 
> 
> 	PL_action(PL_GMP_SET_ALLOC_FUNCTIONS, TRUE)
> 
> if you want Prolog's GMP allocation initialised *now* (without
> initializing the rest of SWI-Prolog).  or
> 
> 	PL_action(PL_GMP_SET_ALLOC_FUNCTIONS, FALSE)
> 
> If you want to stop Prolog from setting the allocation functions.  Must be
> called before PL_initialise(), of course.

I see how to use that in the foreign-calls-Prolog case.
How can I achieve that in the Prolog-calls-foreign case (which is the one
I am mostly interested in)?

> I also added some comments to the section of the manual dealing with GMP
> and Prolog.  On Windows the situation is entirely different :-(

Well, I am not so worried about Windows :-)

> Please install from CVS.  Do not forget a "make clean"!

Right.  I hope people packaging SWI-Prolog for the various distributions
do not select a version in the range 5.6.38-5.6.43.
Cheers,

     Roberto

-- 
Prof. Roberto Bagnara
Computer Science Group
Department of Mathematics, University of Parma, Italy
http://www.cs.unipr.it/~bagnara/
mailto:bagnara at cs.unipr.it