------------------------------------------------------------------------------ http:libburnia-project.org ------------------------------------------------------------------------------ libisoburn/releng. By George Danchev and Thomas Schmitt Test suite for xorriso and libburnia libraries. Copyright (C) 2011 George Danchev, Thomas Schmitt Provided under GPL version 2 or later. ------------------------------------------------------------------------------ The impatient tester will build libisoburn according to ./README and then do cd ./releng ./run_all_auto -x ../xorriso/xorriso More patient testers will first read the following description. The test suite +++ introduce the test suite, Directory ./releng of libisoburn contains a collection of test scripts and auxiliary data. They exercise some typical use cases of building libisoburn applications and running the ISO 9660 filesystem manipulation and CD/DVD/BD burn program xorriso. +++ inform about resource needs, It is assumed that libburn and libisofs are installed, so that libisoburn can be configured and built. It is not mandatory that libisoburn is already installed. The tests may use an installed xorriso program as well as a freshly built one. +++ auto and manual tests There are two groups of test scripts: auto_* gets started and watched by script run_all_auto. These tests have a moderate resource consumption and do not cause mechanical movements of drive trays. manual_* get started by the user if desired. Manual tests may create larger sets of temporary files, may download test data from the internet, may need system privileges beyond the reach of a sandbox user, and operate the mechanics of a CD drive. +++ tell how to run the tests, Running automated tests The test scripts expect to get run while the working directory is ./releng of a libisoburn source tree. E.g.: libisoburn-1.1.4/releng They create all their temporary files underneath ./releng/releng_generated_data Some of these files are persistent between tests. Nevertheless it is safe to empty ./releng/releng_generated_data after tests are done. The directory itself must be kept. To run the unobtrusive automatic tests, build libisoburn and xorriso, go to directory ./releng, and execute ./run_all_auto -x ../xorriso/xorriso or if you want to use an installed xorriso program: ./run_all_auto -x $(type -p xorriso) +++ General options -x , -k, -f, -c, -h, -- +++ + All scripts support -h There are several options which work with run_all_auto and any single test. -x absolute or relative path to xorriso binary to be run. -k keep self-generated data. -c cleanup temporary data kept from previous run and exit. -f simulate failure. -h print this help text -- end of general options, begin of test specific options. After option "--", there may be given options which are specific to particular manually executable test scripts. Manually executable tests +++ list of manual tests Currently there are the following tests which should have the attention of the user or require sysadmin considerations before they are run: ./manual_isojigdo -x ../xorriso/xorriso Exercises the production of a bootable Debian GNU/Linux image and its Jigdo files. This test downloads a Debian daily image for i386 of about 70 MB, extracts its content and composes a new image. Thus it needs about 250 MB of disk space in releng/releng_generated_data . This test will only work with GNU xorriso or if libjte was installed already when libisofs was built. libjte is part of package jigit, version >= 1.18, available at: http://www.einval.com/~steve/software/JTE/ ./manual_devices -x ../xorriso/xorriso [-- [--dev device_file_to_use]] Exercises listing of all accessible optical drives and the examination of a one of these drives. The user needs the permission to operate the CD drives. This might involve the need for superuser authority. The media tray of the examined drive will get loaded if it is not already. If no option --dev is given, then the user gets asked which of the listed drives to examine more closely. ./manual_burn -x ../xorriso/xorriso [-- [--dev device_file_to_use] [--what ...directory...] [--any_media]] Burns the content of the directory given with --what onto re-usable media: CD-RW, DVD-RW, DVD-RAM, DVD+RW, BD-RE. Other media types get refused, unless option --any_media is given. Data, which are possibly present on the media, get overwritten. The result gets check read and compared with the state of the input directory. MD5 mismatch causes a test failure. Differences to the directory state are reported but still regarded as success. >>> ./run_all_manual >>> + Any auto_* script can be run on its own, regardless. >>> List auto tests +++ give hints what to do with FAIL results. The text output of the automatic tests is recorded in file releng_generated_data/log.run_all_auto Script ./run_all_auto will detect failure of perticular tests and report lines from the log file which contain problem indicating key words: NEVER|ABORT|FATAL|FAILURE|MISHAP|SORRY|WARNING|HINT|FAIL|ERROR|WRONG If the program messages in log.run_all_auto do not explain the failure, please contact mailing list libburn-hackers@pykix.org . ---------------------------------------------------------------------------- Creating a new test script +++ * DELIVERING A NEW TEST SCRIPT +++ + Start a new test script over the top of template_new_releng +++ + Each completed script starts with auto_ or manual_ prefix If you want to provide an own test, manual or auto, then first invent a name for it test_name="releng/manual_"...some.name... or test_name="releng/auto_"...some.name... Then copy file releng/template_new to $test_name . Edit $test_name and process any line that begins by "# === TEMPLATE:". Do what the line prescribes and then remove it from the script. You are not done as long as such a line remains. +++ + Each script' self-generated data are to be stored in +++ ./releng_generated_data/scriptname/ directory Your test must not start if no file ./inc/releng_getopts.inc exists, i.e. if the current working direcoty is not ./releng. If your test creates own files on disk, then it must do this underneath directory ./releng_generated_data/$test_name (resp. $GEN_DATA_DIR, see below). +++ + Throw FAIL string to stdout if any failure is detected +++ + Return exit code 0 on success, non-zero on failure +++ + Use different exit codes for any failure (range 0-31) In case of failure, issue a line to stdout that begins by the word "FAIL", followed by " : " and the name of the test (e.g. $SELF, see below). Make sure that the test script finally returns a non-zero exit value. This value should be between 1 and 28. Each type of failure should have its own exit value. Predefined are: 31 = Unknown option or unusable argument with known option 30 = Unexpected state of own directory for self generated files 29 = Not in ./releng directory or missing essential parts of ./releng When exiting prematurely, make sure to call function cleanup. General options, variables, and functions +++ inc/releng_getopts.inc variables, functions, initializations The code piece inc/releng_getopts.inc should get executed inline at the start of a test script. It initializes the following variables and sets some of them according to the general options of the test suite: SELF basename $0 GEN_DATA_DIR releng_generated_data/${SELF} RELENG_XORRISO Path to xorriso binary. "" or "0" means no xorriso. Default "0". Adjustable by option -x. SIMULATE_FAILURE 0=normal operation, 1=test script shall simulate a failure. Default 0. Setable to 1 by option -f. CLEANUP 0=do not cleanup temporary data, 1=normal operation Default 1. Setable to 0 by option -k. SPECIFIC_HELP 0=normal operation, 1=print help text of script and exit 0 Default 0. Setable to 1 by option -h The code piece inc/releng_getopts.inc defines the follwoing functions for use by the single tests: print_help Prints the help text for general options. check_for_xorriso [-x] Will exit with value 31 if no path to a xorriso binary was defined by option -x of ./run_all_auto or a single test. Option -x of check_for_xorriso additionally tests whether the given path leads to an executable program. cleanup Removes the directory tree GEN_DATA_DIR after making some safety checks. boldify Try to set the terminal mode for future output to a more noticable style of writing. unboldify Reset terminal mode to normal style of writing. Specific options +++ + Keep the newly added script options in sync with other scripts Options which are specific to the test should begin with a double dash. They may have further arguments. Implement them in the prepared interpreter loop which begins after line next_is=ignore Specific options shall only be interpreted by tests which get run manually. If you plan to introduce a specific option, look at the description of existing tests whether one of them would match your needs. In that case, please re-use the name of that existing option. ============================================================================= TODO: * Delete debian-testing-i386-businesscard.iso with ./run_all_auto -c * Have a script ./run_all_manual * derive a GNU xorriso test suite from libisoburn test suite * auto_ tests should know whether they work underneath run_all_auto or whether they run standalone