From danchev@spnet.net Tue Jun 21 18:33:30 2011
Date: Tue, 21 Jun 2011 19:22:08 +0300
From: George Danchev <danchev@spnet.net>
To:  <libburn-hackers@pykix.org>
Subject: [Libburn-hackers] cheat tests proposal

Hi,

Here is a cheat list of performance tests I have in my mind for testing the 
whole thing, for instance before release. This list does not depreciate the 
practical use case pattern of a potential user or developer might happen to 
have. The are put together to second-guess he potential testers, and anyone 
can perform them at will (even partially) on a best effort fashion. I won't go 
that deep to assign them artificial weights or priority.

1) Various compilation tests, like:
export CPPFLAGS=-DLibburn_use_sg_dummY; ./configure
repeat for the adapters of the all target OSes, there should be other 
interesting macros, static check logs can easily reveal them:-)
This can be scripted of course.

2) Inspect the static check logs:
(revision numbers are well indicated)
http://people.debian.org/~danchev/xorriso/sclogs/

3) Inspect previous versions build daemon logs (various architectures, three 
kernels: linux, kfreebsd, hurd) for interesting compiler warnings or errors:
http://buildd.debian.org
I'm sure I've seen FreeBSD public build logs before, but my searching fails so 
far!

4) Testing xorriso in dialog mode:
perhaps this 'proof of concept' toy could be helpful:
http://anonscm.debian.org/gitweb/?p=users/danchev/medistimator.git;a=summary
this requires some specific knowledge of how the tool works (there are 
hopefully very helpful comments in the source), to interpret the results and 
compare them with the previous runs; my logs could be found at:
http://people.debian.org/~danchev/medistimator/log/
the aim is to catch speed performace hits, corectness of results, crashes, etc 
when working on various input data trees, like very large BD-sized ones.

5) Profiling tests with gprof. We did that once :-)

Each of these tests has already caught a flaw or another, so they are stolen 
from the past experience we already had. The list of course if open for adding 
more interesting checks, like real burning ones for instance. I'm sure I 
forgot at least one, since I had 6 in mind... Alois....

6) jigdo production.

I'll try to simplify a run like that one:

xorriso -as mkisofs -r
 -checksum_algorithm_iso md5,sha1,sha256,sha512
 -V 'Debian testing amd64 1'
 -o /org/cdbuilder.debian.org/dst/deb-cd/out/sidamd64/debian-testing-amd64-CD-1.iso
 -jigdo-jigdo /org/cdbuilder.debian.org/dst/deb-cd/out/sidamd64/debian-testing-amd64-CD-1.jigdo
 -jigdo-template /org/cdbuilder.debian.org/dst/deb-cd/out/sidamd64/debian-testing-amd64-CD-1.template
 -jigdo-map Debian=/org/cdbuilder.debian.org/src/ftp/debian/
 -jigdo-exclude boot1 -md5-list /org/cdbuilder.debian.org/src/deb-cd/tmp/sidamd64/wheezy/md5-check
 -jigdo-min-file-size 1024
 -jigdo-exclude 'README*' -jigdo-exclude /doc/
 -jigdo-exclude /md5sum.txt -jigdo-exclude /.disk/
 -jigdo-exclude /pics/
 -jigdo-exclude 'Release*'
 -jigdo-exclude 'Packages*'
 -jigdo-exclude 'Sources*'
 -J -isohybrid-mbr syslinux/usr/lib/syslinux/isohdpfx.bin
 -partition_offset 16
 -J -joliet-long
 -cache-inodes -b isolinux/isolinux.bin -c isolinux/boot.cat
 -no-emul-boot -boot-load-size 4 -boot-info-table
 boot1 CD1