use more declarative style in test descriptions

This commit is contained in:
George Danchev 2011-06-22 14:44:07 +00:00
parent 59a8eed523
commit 256db7fda1
1 changed files with 23 additions and 69 deletions

View File

@ -1,74 +1,28 @@
From danchev@spnet.net Tue Jun 21 18:33:30 2011
Date: Tue, 21 Jun 2011 19:22:08 +0300
From: George Danchev <danchev@spnet.net>
To: <libburn-hackers@pykix.org>
Subject: [Libburn-hackers] cheat tests proposal
NAME: Compilation
FILE: libburn/releng/releng_build_os
WHO: george, thomas
Hi,
NAME: Static check
FILE: http://people.debian.org/~danchev/xorriso/sclogs/
WHO: george, thomas
Here is a cheat list of performance tests I have in my mind for testing the
whole thing, for instance before release. This list does not depreciate the
practical use case pattern of a potential user or developer might happen to
have. The are put together to second-guess he potential testers, and anyone
can perform them at will (even partially) on a best effort fashion. I won't go
that deep to assign them artificial weights or priority.
TEST: Build logs (for various hardware architectures and kernels: linux, kfreebsd, hurd)
FILE: http://buildd.debian.org
WHO: george, thomas
1) Various compilation tests, like:
export CPPFLAGS=-DLibburn_use_sg_dummY; ./configure
repeat for the adapters of the all target OSes, there should be other
interesting macros, static check logs can easily reveal them:-)
This can be scripted of course.
NAME: Dialog mode, large trees
this requires some specific knowledge of how the tool works (see comments in the
source), to interpret the results and compare them with the previous runs.
The aim is to catch speed performace regressions, prove correctness of results,
etc when working on various input data trees, like very large BD-sized ones.
FILE: http://anonscm.debian.org/gitweb/?p=users/danchev/medistimator.git;a=summary
FILE: http://people.debian.org/~danchev/medistimator/log/
WHO: george, thomas
2) Inspect the static check logs:
(revision numbers are well indicated)
http://people.debian.org/~danchev/xorriso/sclogs/
3) Inspect previous versions build daemon logs (various architectures, three
kernels: linux, kfreebsd, hurd) for interesting compiler warnings or errors:
http://buildd.debian.org
I'm sure I've seen FreeBSD public build logs before, but my searching fails so
far!
4) Testing xorriso in dialog mode:
perhaps this 'proof of concept' toy could be helpful:
http://anonscm.debian.org/gitweb/?p=users/danchev/medistimator.git;a=summary
this requires some specific knowledge of how the tool works (there are
hopefully very helpful comments in the source), to interpret the results and
compare them with the previous runs; my logs could be found at:
http://people.debian.org/~danchev/medistimator/log/
the aim is to catch speed performace hits, corectness of results, crashes, etc
when working on various input data trees, like very large BD-sized ones.
5) Profiling tests with gprof. We did that once :-)
Each of these tests has already caught a flaw or another, so they are stolen
from the past experience we already had. The list of course if open for adding
more interesting checks, like real burning ones for instance. I'm sure I
forgot at least one, since I had 6 in mind... Alois....
6) jigdo production.
I'll try to simplify a run like that one:
xorriso -as mkisofs -r
-checksum_algorithm_iso md5,sha1,sha256,sha512
-V 'Debian testing amd64 1'
-o /org/cdbuilder.debian.org/dst/deb-cd/out/sidamd64/debian-testing-amd64-CD-1.iso
-jigdo-jigdo /org/cdbuilder.debian.org/dst/deb-cd/out/sidamd64/debian-testing-amd64-CD-1.jigdo
-jigdo-template /org/cdbuilder.debian.org/dst/deb-cd/out/sidamd64/debian-testing-amd64-CD-1.template
-jigdo-map Debian=/org/cdbuilder.debian.org/src/ftp/debian/
-jigdo-exclude boot1 -md5-list /org/cdbuilder.debian.org/src/deb-cd/tmp/sidamd64/wheezy/md5-check
-jigdo-min-file-size 1024
-jigdo-exclude 'README*' -jigdo-exclude /doc/
-jigdo-exclude /md5sum.txt -jigdo-exclude /.disk/
-jigdo-exclude /pics/
-jigdo-exclude 'Release*'
-jigdo-exclude 'Packages*'
-jigdo-exclude 'Sources*'
-J -isohybrid-mbr syslinux/usr/lib/syslinux/isohdpfx.bin
-partition_offset 16
-J -joliet-long
-cache-inodes -b isolinux/isolinux.bin -c isolinux/boot.cat
-no-emul-boot -boot-load-size 4 -boot-info-table
boot1 CD1
TEST: jigdo
FILE: libisoburn/releng/releng_build_jigdo
WHO: george
TEST: gprof
FILE: -pg
WHO: george, thomas