[PPL-devel] ppl-0.10.2 check thorough test fail

[Roberto Bagnara]

On 03/29/11 02:39, David Fang wrote:
> With ppl-0.10.2, several Mac OS X fink users (including myself) have observed the following single test failure on 10.5 and 10.6, using Apple's gcc-4.2, arch i686-apple-darwin{9,10}, with thorough tests enabled:
>
> /usr/bin/grep -E "^Optimum value: " ../../../demos/ppl_lpsol/expected_mpz
>> expected_optima && /usr/bin/grep -E "^Optimum value: " obtained obtained_optima && diff -u expected_optima obtained_optima
> --- expected_optima 2011-03-22 21:05:58.000000000 -0400
> +++ obtained_optima 2011-03-22 21:05:58.000000000 -0400
> @@ -1,9 +1,10 @@
> -Optimum value: -3
> -Optimum value: 2
> +Optimum value: -2
> +Optimum value: 1
> Optimum value: 24.33333333
> Optimum value: 43
> Optimum value: 3089
> Optimum value: 5201
> +Optimum value: -2
> Optimum value: 0
> Optimum value: 225494.9632
> Optimum value: -464.7531429
> @@ -15,8 +16,8 @@
> Optimum value: -73.36896911
> Optimum value: 149.5887662
> Optimum value: 964.30053
> -Optimum value: -3.75
> -Optimum value: 2.5
> +Optimum value: -2
> +Optimum value: 1
> Optimum value: -1749.90013
> Optimum value: 0
> Optimum value: 834.6823529
> @@ -44,9 +45,10 @@
> Optimum value: 0
> Optimum value: -70
> Optimum value: 0
> +Optimum value: -2
> Optimum value: 0
> Optimum value: 3438.2921
> -Optimum value: 2.5
> +Optimum value: 1
> Optimum value: 46.42857143
> Optimum value: 0
> Optimum value: 0
> make[4]: *** [check-local] Error 1
>
> Most fink users that reported this have built ppl against gmp-4.3.2.
> I tested a build against gmp-5.0.1 and got the exact same result on
> i686-apple-darwin10.
>
> However, the same set of tests (check thorough) pass *cleanly* on
> powerpc-apple-darwin8 apple-gcc-4.0.1 (ppl-0.10.2 using gmp-4.3.2),
> which took over 4 days to run.

Hi David,

yes, `make check' is heavy.  However, our experience suggests no tests
in the regression test suite is really redundant.

> Has anyone else observed this failure on this version/configuration?

Not that I know, but it would be useful to know the compiler version
used.  In the (rather distant) past I remember some users had problems
with some Apple-modified versions of GCC.

> How severe is this test failure?

I would consider it severe, as it might indicate that the PPL has been
miscompiled.  This is my first suspect.  Of course, we cannot exclude
the cause is an architecture/compiler/platform-dependent bug in
PPL 0.10.2 that went undetected up to now: perhaps `git bisect' could
help locating it.

> (Unrelated note: ppl-0.11's tests run cleanly on i686-darwin10.)

PPL 0.11.2 is the latest version of the PPL and the one we warmly
recommend to anyone.  If for some reason you cannot switch to PPL
0.11.2, please try compiling PPL 0.10.2 with a different compiler and
see if the problem persists.  If it does, we may investigate other
possibilities.
All the best,

    Roberto

-- 
Prof. Roberto Bagnara
Applied Formal Methods Laboratory
Department of Mathematics, University of Parma, Italy
http://www.cs.unipr.it/~bagnara/
mailto:bagnara at cs.unipr.it