Documented the zisofs format by H. Peter Anvin.

doc/zisofs_format.txt

@@ -0,0 +1,134 @@
+
+                      Description of the zisofs Format
+
+                 as of zisofs-tools-1.0.8 by H. Peter Anvin
+                 and cdrtools-2.01.01a39 by Joerg Schilling
+ 
+     For libburnia-project.org by Thomas Schmitt <scdbackup@gmx.net>
+                              Apr 7 2009
+
+
+The zisofs format was invented by H. Peter Anvin. It compresses data file
+content, marks it by a header and provides a pointer array for coarse random
+access. Within a RRIP enhanced ISO 9660 image the format is additionally marked
+by a System Use entry with signature "ZF".
+
+The uncompressed size of a single zisofs compressed file is restricted
+to 4 GiB - 1. Larger files shall not be compressed.
+
+
+                                File Header
+
+The file header has this layout (quoted from zisofs-tools-1.0.8/mkzftree.c):
+    Byte offset   iso9660 type    Contents
+      0           (8 bytes)       Magic number (37 E4 53 96 C9 DB D6 07)
+      8           7.3.1           Uncompressed file size
+     12           7.1.1           header_size >> 2 (currently 4)
+     13           7.1.1           log2(block_size)
+     14           (2 bytes)       Reserved, must be zero
+So its size is 16.
+7.3.1 means little endian 4-byte words. 7.1.1. means unsigned single bytes.
+
+                               Block Pointers
+
+There are ceil(input_size / block_size) input resp. output blocks.
+Each input block is of fixed size whereas the output blocks have varying
+size (down to 0). For each output block there is an offset pointer giving
+its byte address in the overall file content. The next block pointer in the
+array tells the start of the next block which begins immediately after the
+end of its predecessor. A final pointer gives the first invalid byte address
+and thus marks the end of the last block.
+
+The block pointers are stored as an array of 4-byte values which are in ISO
+9660:7.3.1 format directly after the file header, i.e. beginning at byte 16.
+
+                                 Data Part
+
+The data part begins immediately after the pointer array. In principle it
+consists of the variable length output blocks as delivered by zlib function
+compress2() when fed with the fixed size input blocks.
+A special case of output block is defined:
+- Zero-length blocks represent a block full of 0-bytes.
+  They do not get processed by compress2() but have to be implemented as
+  bypass of compress2().
+
+
+                         ZF System Use Entry Format
+
+ZF may only be applied to files with a single extent and less than 4 GiB of
+uncompressed size.
+
+The ZF entry follows the general layout of SUSP resp. RRIP.
+Its fields are:
+
+  [1] "BP 1 to BP 2 - Signature Word" shall be (5A)(46) ("ZF").
+
+  [2] "BP 3 - Length" shall specify as an 8-bit number the length in bytes of
+      the ZF entry recorded according to ISO 9660:7.1.1.
+      This length is 16 decimal.
+
+  [3] "BP 4 - System Use Entry Version" shall be 1 as in ISO 9660:7.1.1.
+
+  [4] "BP 5 to BP 6 - Algorithm"  shall be (70)(7A) ("pz") to indicate 
+      "paged zlib".
+
+  [5] "BP 7 - Header Size" shall specify as an 8-bit number the number of
+      4-byte words in the header part of the file data recorded according
+      to ISO 9660:7.1.1.
+      (This is a copy of header byte 12, resp. header BP 13).
+
+  [6] "BP 8 - Block Size" shall specify as an 8-bit number the binary
+      logarithm of the compression block size recorded according to
+      ISO 9660:7.1.1.
+      (This is a copy of header byte 13, resp. header BP 14.)
+      Implementations shall be able to handle values 15, 16 and 17 i.e.
+      block sizes 32 kB, 64 kB, and 128 kB.
+
+  [7] "BP 9 to BP 16 - Uncompressed Size" shall tell the number of uncompressed
+      bytes represented by the given extent. This field shall be recorded
+      according to ISO 9660:7.3.3.
+      (This number is the same as in header bytes 8 to 11, resp header BP 9
+       to BP 12.)
+
+  | 'Z' | 'F' | LENGTH | 1 | 'p' | 'z' | HEADER SIZE | BLOCK SIZE | 
+                                                            UNCOMPRESSED SIZE |
+
+ISO 9660:7.3.3 means 4-byte word in both byte orders, first little endian, then
+big endian. I.e. the decimal number 513 is represented in 8 bytes as:
+  { 0x02, 0x01, 0x00, 0x00,   0x00, 0x00, 0x01, 0x02 }
+
+Example (uncompressed file size = 1,234,567 bytes):
+  { 'Z',  "F',   16,    1,   'p',  'z',   16,   15,
+   0x87, 0xD6, 0x12, 0x00,  0x00, 0x12, 0xD6, 0x87 }
+
+-------------------------------------------------------------------------------
+Revoked specification aspects:
+
+The comments in zisofs-tools-1.0.8 indicate a special case of output block:
+  "a block the length of which is equal to the block size is unencoded."
+This was never implemented and it is questionable whether it can be added to
+the existing implementations in an unambigous way.
+Therefore that rule is not part of this description and shall not be
+implemented.
+
+-------------------------------------------------------------------------------
+References:
+
+zisofs-tools
+  http://freshmeat.net/projects/zisofs-tools/
+
+cdrtools with mkisofs
+  ftp://ftp.berlios.de/pub/cdrecord/alpha
+
+ECMA-119 aka ISO 9660
+  http://www.ecma-international.org/publications/files/ECMA-ST/Ecma-119.pdf
+
+SUSP 1.12
+  ftp://ftp.ymi.com/pub/rockridge/susp112.ps
+
+RRIP 1.12
+  ftp://ftp.ymi.com/pub/rockridge/rrip112.ps
+
+zlib:
+  /usr/include/zlib.h
+