[PPL-devel] Re: Doing something just a bit more complex than `make check'

[Roberto Bagnara]

Ralf Wildenhues wrote:
> Hello there,
> 
> * Roberto Bagnara wrote on Wed, Oct 19, 2005 at 09:13:16PM CEST:
> 
>>Instead, what I would like to have is to only say
>>
>>test1_SOURCES = test1.cc
>>test2_SOURCES = test2.cc
>>...
>>
>>and then achieve the effect of (sorry for the pseudo-code)
>>
>>   for flags in $FLAGS_CHOICES
>>   do
>>      make check with CXXFLAGS="$flags"
>>      force recompilation at the next iteration (e.g., by erasing 
>>      executables)
>>   done
>>
>>How can I best obtain this effect without giving up (too many of)
>>the advantages offered by Automake?
> 
> 
> I believe something along these lines may be achieved by
> 
> | check_SCRIPTS = runtests
> | check_PROGRAMS = test1 test2
> | test1_SOURCES = test1.cc
> | test2_SOURCES = test2.cc
> | TESTS = $(check_SCRIPTS)
> | my-check-clean:
> |       rm -f $(check_PROGRAMS) *.o *.obj
> 
> in Makefile.am, and your pseudo code in `./runtests':
>     for flags in $FLAGS_CHOICES
>     do
>        make check CXXFLAGS="$flags" TESTS='test1 test2'
>        make my-check-clean
>        # to force recompilation at the next iteration
>     done
> 
> Surely the object removal is a bit of a hack, and should be adapted
> unless your test suite lives in its own directory/Makefile.am.
> I haven't tested this, by the way; please complain if it doesn't work.  :)

Dear Ralf,

thank you very much for your help (and sorry for the delay, but I wanted
to make sure we did all our homework before getting back to you).
Here is the script we are using:

   http://www.cs.unipr.it/cgi-bin/cvsweb.cgi/ppl/tests/BD_Shape/run_tests?rev=1.11

The dirty_marker trick allows us to avoid useless recompilations yet
addressing the case where one run of tests is interrupted.
We have also managed not to duplicate the list of tests and
to deal with XFAILs in a satisfactory way.  Any further advice you may
have is very welcome.

> If you want to go much further, you either end up creating more complex
> shell scripts, or using one of the more advanced test suite creation
> tools: Autoconf's Autotest, DejaGNU, ...

Autotest looks attractive.  We may consider switching to it as soon
as it stabilizes.
All the best, and thanks again,

     Roberto

-- 
Prof. Roberto Bagnara
Computer Science Group
Department of Mathematics, University of Parma, Italy
http://www.cs.unipr.it/~bagnara/
mailto:bagnara at cs.unipr.it