Installing a zisofs uncompression filter on file from ISO image which bear

a ZF entry. Storing ZF entries during image write if that filter is found
directly on a fsrc stream from the loaded image and content does not get
copied.
This commit is contained in:
Thomas Schmitt 2009-04-11 14:15:34 +02:00
parent 77591e4546
commit 6886777ea0
7 changed files with 187 additions and 25 deletions

View File

@ -1,11 +1,13 @@
Description of the zisofs Format Description of the zisofs Format
as of zisofs-tools-1.0.8 by H. Peter Anvin as of zisofs-tools-1.0.8 by H. Peter Anvin
and cdrtools-2.01.01a39 by Joerg Schilling and cdrtools-2.01.01a39 by Joerg Schilling
For libburnia-project.org by Thomas Schmitt <scdbackup@gmx.net> For libburnia-project.org by Thomas Schmitt <scdbackup@gmx.net>
Apr 7 2009 - distribute freely , please report any errors or ambiguities -
Apr 11 2009
The zisofs format was invented by H. Peter Anvin. It compresses data file The zisofs format was invented by H. Peter Anvin. It compresses data file
@ -30,7 +32,7 @@ So its size is 16.
7.3.1 means little endian 4-byte words. 7.1.1. means unsigned single bytes. 7.3.1 means little endian 4-byte words. 7.1.1. means unsigned single bytes.
Readers shall be able to handle log2(block_size) values 15, 16 and 17 Readers shall be able to handle log2(block_size) values 15, 16 and 17
i.e. block sizes 32 kB, 64 kB, and 128 kB. i.e. block sizes 32 kB, 64 kB, and 128 kB. Writers must not use other sizes.
Block Pointers Block Pointers
@ -80,17 +82,16 @@ Its fields are:
[4] "BP 5 to BP 6 - Algorithm" shall be (70)(7A) ("pz") to indicate [4] "BP 5 to BP 6 - Algorithm" shall be (70)(7A) ("pz") to indicate
"paged zlib". "paged zlib".
[5] "BP 7 - Header Size" shall specify as an 8-bit number the number of [5] "BP 7 - Header Size Div 4" shall specify as an 8-bit number the number of
4-byte words in the header part of the file data recorded according 4-byte words in the header part of the file data recorded according
to ISO 9660:7.1.1. to ISO 9660:7.1.1.
(This is a copy of header byte 12, resp. header BP 13). (This is a copy of header byte 12, resp. header BP 13).
[6] "BP 8 - Block Size" shall specify as an 8-bit number the binary [6] "BP 8 - Log2 of Block Size" shall specify as an 8-bit number the binary
logarithm of the compression block size recorded according to logarithm of the compression block size recorded according to
ISO 9660:7.1.1. ISO 9660:7.1.1.
(This is a copy of header byte 13, resp. header BP 14.) (This is a copy of header byte 13, resp. header BP 14.
Implementations shall be able to handle values 15, 16 and 17 i.e. The value has to be 15, 16 or 17 i.e. 32 kiB, 64 kiB, or 128 kiB.)
block sizes 32 kiB, 64 kiB, and 128 kiB.
[7] "BP 9 to BP 16 - Uncompressed Size" shall tell the number of uncompressed [7] "BP 9 to BP 16 - Uncompressed Size" shall tell the number of uncompressed
bytes represented by the given extent. This field shall be recorded bytes represented by the given extent. This field shall be recorded
@ -98,8 +99,8 @@ Its fields are:
(This number is the same as in header bytes 8 to 11, resp header BP 9 (This number is the same as in header bytes 8 to 11, resp header BP 9
to BP 12.) to BP 12.)
| 'Z' | 'F' | LENGTH | 1 | 'p' | 'z' | HEADER SIZE | BLOCK SIZE | | 'Z' | 'F' | LENGTH | 1 | 'p' | 'z' | HEADER SIZE DIV 4 | LOG2 BLOCK SIZE
UNCOMPRESSED SIZE | | UNCOMPRESSED SIZE |
ISO 9660:7.3.3 means 4-byte word in both byte orders, first little endian, then ISO 9660:7.3.3 means 4-byte word in both byte orders, first little endian, then
big endian. big endian.
@ -107,13 +108,16 @@ Example (block size 32 kiB, uncompressed file size = 1,234,567 bytes):
{ 'Z', "F', 16, 1, 'p', 'z', 4, 15, { 'Z', "F', 16, 1, 'p', 'z', 4, 15,
0x87, 0xD6, 0x12, 0x00, 0x00, 0x12, 0xD6, 0x87 } 0x87, 0xD6, 0x12, 0x00, 0x00, 0x12, 0xD6, 0x87 }
------------------------------------------------------------------------------- -------------------------------------------------------------------------------
Revoked specification aspects: Revoked specification aspects:
The comments in zisofs-tools-1.0.8 indicate a special case of output block: A comment in zisofs-tools-1.0.8 indicates a special case of output block:
"a block the length of which is equal to the block size is unencoded." "a block the length of which is equal to the block size is unencoded."
This is not implemented in zisofs-tools and in the Linux kernel. This is not implemented in zisofs-tools and in the Linux kernel. Existing
Therefore that rule is not part of this description and must not be zisofs enhanced ISO images might contain encoded blocks which could be
mistaken for unencoded blocks.
Therefore this rule is not part of this description and must not be
implemented. implemented.
------------------------------------------------------------------------------- -------------------------------------------------------------------------------

View File

@ -886,8 +886,9 @@ int ziso_create_context(FilterContext **filter, int flag)
* @param flag bit0= if_block_reduction rather than if_reduction * @param flag bit0= if_block_reduction rather than if_reduction
* bit1= Install a decompression filter * bit1= Install a decompression filter
* bit2= only inquire availability of zisofs filtering * bit2= only inquire availability of zisofs filtering
* bit3= do not inquire size
*/ */
int iso_file_add_zisofs_filter(IsoFile *file, int flag) int ziso_add_filter(IsoFile *file, int flag)
{ {
#ifdef Libisofs_with_zliB #ifdef Libisofs_with_zliB
@ -919,6 +920,9 @@ int iso_file_add_zisofs_filter(IsoFile *file, int flag)
if (ret < 0) { if (ret < 0) {
return ret; return ret;
} }
if (flag & 8) /* size will be filled in by caller */
return ISO_SUCCESS;
/* Run a full filter process getsize so that the size is cached */ /* Run a full filter process getsize so that the size is cached */
stream = iso_file_get_stream(file); stream = iso_file_get_stream(file);
filtered_size = iso_stream_get_size(stream); filtered_size = iso_stream_get_size(stream);
@ -946,6 +950,41 @@ int iso_file_add_zisofs_filter(IsoFile *file, int flag)
} }
int iso_file_add_zisofs_filter(IsoFile *file, int flag)
{
return ziso_add_filter(file, flag & ~8);
}
int ziso_add_osiz_filter(IsoFile *file, uint8_t header_size_div4,
uint8_t block_size_log2, uint32_t uncompressed_size,
int flag)
{
#ifdef Libisofs_with_zliB
int ret;
ZisofsUncomprStreamData *unstd;
ret = ziso_add_filter(file, 2 | 8);
if (ret < 0)
return ret;
unstd = iso_file_get_stream(file)->data;
unstd->header_size_div4 = header_size_div4;
unstd->block_size_log2 = block_size_log2;
unstd->std.size = uncompressed_size;
return ISO_SUCCESS;
#else
return ISO_ZLIB_NOT_ENABLED;
#endif /* ! Libisofs_with_zliB */
}
/* Determine stream type : 1=ziso , -1=osiz , 0=other /* Determine stream type : 1=ziso , -1=osiz , 0=other
and eventual ZF field parameters and eventual ZF field parameters
*/ */

View File

@ -287,6 +287,12 @@ struct image_fs_data
unsigned int opened : 2; /**< 0 not opened, 1 opened file, 2 opened dir */ unsigned int opened : 2; /**< 0 not opened, 1 opened file, 2 opened dir */
#ifdef Libisofs_with_zliB
uint8_t header_size_div4;
uint8_t block_size_log2;
uint32_t uncompressed_size;
#endif
/* info for content reading */ /* info for content reading */
struct struct
{ {
@ -1109,6 +1115,8 @@ int iso_file_source_new_ifs(IsoImageFilesystem *fs, IsoFileSource *parent,
size_t cs_value_length = 0; size_t cs_value_length = 0;
char msg[160]; char msg[160];
uint8_t zisofs_alg[2], zisofs_hs4 = 0, zisofs_bsl2 = 0;
uint32_t zisofs_usize = 0;
if (fs == NULL || fs->data == NULL || record == NULL || src == NULL) { if (fs == NULL || fs->data == NULL || record == NULL || src == NULL) {
return ISO_NULL_POINTER; return ISO_NULL_POINTER;
@ -1375,6 +1383,21 @@ int iso_file_source_new_ifs(IsoImageFilesystem *fs, IsoFileSource *parent,
continue; continue;
} }
#ifdef Libisofs_with_zliB
} else if (SUSP_SIG(sue, 'Z', 'F')) {
ret = read_zisofs_ZF(sue, zisofs_alg, &zisofs_hs4,
&zisofs_bsl2, &zisofs_usize, 0);
if (ret < 0 || zisofs_alg[0] != 'p' || zisofs_alg[1] != 'z') {
/* notify and continue */
ret = iso_msg_submit(fsdata->msgid, ISO_WRONG_RR_WARN, ret,
"Invalid ZF entry");
zisofs_hs4 = 0;
continue;
}
#endif /* Libisofs_with_zliB */
/* This message is inflationary */ /* This message is inflationary */
/* /*
@ -1653,7 +1676,11 @@ int iso_file_source_new_ifs(IsoImageFilesystem *fs, IsoFileSource *parent,
ifsdata = (*src)->data; ifsdata = (*src)->data;
ifsrc = (*src); ifsrc = (*src);
free(ifsdata->name); /* we will assign a new one */ free(ifsdata->name); /* we will assign a new one */
ifsdata->name = NULL;
atts.st_size += (off_t)ifsdata->info.st_size; atts.st_size += (off_t)ifsdata->info.st_size;
if (ifsdata->aa_string != NULL)
free(ifsdata->aa_string);
ifsdata->aa_string = NULL;
} }
/* fill data */ /* fill data */
@ -1667,6 +1694,16 @@ int iso_file_source_new_ifs(IsoImageFilesystem *fs, IsoFileSource *parent,
ifsdata->name = name; ifsdata->name = name;
ifsdata->aa_string = aa_string; ifsdata->aa_string = aa_string;
#ifdef Libisofs_with_zliB
if (zisofs_hs4 > 0) {
ifsdata->header_size_div4 = zisofs_hs4;
ifsdata->block_size_log2 = zisofs_bsl2;
ifsdata->uncompressed_size = zisofs_usize;
} else {
ifsdata->header_size_div4 = 0;
}
#endif
/* save extents */ /* save extents */
ifsdata->sections = realloc(ifsdata->sections, ifsdata->sections = realloc(ifsdata->sections,
(1 + ifsdata->nsections) * sizeof(struct iso_file_section)); (1 + ifsdata->nsections) * sizeof(struct iso_file_section));
@ -2466,6 +2503,14 @@ int image_builder_create_node(IsoNodeBuilder *builder, IsoImage *image,
char *name; char *name;
ImageFileSourceData *data; ImageFileSourceData *data;
#ifdef Libisofs_with_zliB
/* Intimate friendship with this function in filters/zisofs.c */
int ziso_add_osiz_filter(IsoFile *file, uint8_t header_size_div4,
uint8_t block_size_log2,
uint32_t uncompressed_size, int flag);
#endif /* Libisofs_with_zliB */
if (builder == NULL || src == NULL || node == NULL || src->data == NULL) { if (builder == NULL || src == NULL || node == NULL || src->data == NULL) {
return ISO_NULL_POINTER; return ISO_NULL_POINTER;
} }
@ -2544,6 +2589,22 @@ int image_builder_create_node(IsoNodeBuilder *builder, IsoImage *image,
file->stream = stream; file->stream = stream;
file->node.type = LIBISO_FILE; file->node.type = LIBISO_FILE;
#ifdef Libisofs_with_zliB
if (data->header_size_div4 > 0) {
ret = ziso_add_osiz_filter(file, data->header_size_div4,
data->block_size_log2,
data->uncompressed_size, 0);
if (ret < 0) {
free(name);
iso_stream_unref(stream);
return ret;
}
}
#endif /* Libisofs_with_zliB */
new = (IsoNode*) file; new = (IsoNode*) file;
new->refcount = 0; new->refcount = 0;

View File

@ -4913,6 +4913,7 @@ int iso_stream_get_external_filter(IsoStream *stream,
* bit1= Install a decompression filter rather than one for compression. * bit1= Install a decompression filter rather than one for compression.
* bit2= Only inquire availability of zisofs filtering. file may be NULL. * bit2= Only inquire availability of zisofs filtering. file may be NULL.
* If available return 2, else return error. * If available return 2, else return error.
* bit3= is reserved for internal use and will be forced to 0
* @return * @return
* 1 on success, 2 if filter available but installation revoked * 1 on success, 2 if filter available but installation revoked
* <0 on error, e.g. ISO_ZLIB_NOT_ENABLED * <0 on error, e.g. ISO_ZLIB_NOT_ENABLED

View File

@ -806,7 +806,7 @@ int add_zf_field(Ecma119Image *t, Ecma119Node *n, struct susp_info *info,
int ret, will_copy = 1, stream_type = 0, do_zf = 0; int ret, will_copy = 1, stream_type = 0, do_zf = 0;
int header_size_div4 = 0, block_size_log2 = 0; int header_size_div4 = 0, block_size_log2 = 0;
uint32_t uncompressed_size = 0; uint32_t uncompressed_size = 0;
IsoStream *stream = NULL, *input_stream; IsoStream *stream = NULL, *input_stream, *next_stream;
IsoFile *file; IsoFile *file;
/* Intimate friendship with this function in filters/zisofs.c */ /* Intimate friendship with this function in filters/zisofs.c */
@ -822,12 +822,15 @@ int add_zf_field(Ecma119Image *t, Ecma119Node *n, struct susp_info *info,
if (t->appendable && file->from_old_session) if (t->appendable && file->from_old_session)
will_copy = 0; will_copy = 0;
stream = iso_file_get_stream(file); next_stream = stream = iso_file_get_stream(file);
while (!will_copy) { /* Obtain most original stream (image stream) */ while (!will_copy) { /* Obtain second-most original stream
input_stream = iso_stream_get_input_stream(stream, 0); (the eventual osiz filter on the image stream) */
input_stream = iso_stream_get_input_stream(next_stream, 0);
if (input_stream == NULL) if (input_stream == NULL)
break; break;
stream = input_stream;
stream = next_stream;
next_stream = input_stream;
} }
/* Determine stream type : 1=ziso , -1=osiz , 0=other */ /* Determine stream type : 1=ziso , -1=osiz , 0=other */
@ -1048,7 +1051,7 @@ int susp_calc_nm_sl_al(Ecma119Image *t, Ecma119Node *n, size_t space,
if (t->aaip) { if (t->aaip) {
ret = iso_node_get_xinfo(n->node, aaip_xinfo_func, &xipt); ret = iso_node_get_xinfo(n->node, aaip_xinfo_func, &xipt);
if (ret == 1) { if (ret == 1) {
num_aapt = aaip_count_bytes((unsigned char *) xipt, 0); num_aapt = aaip_count_bytes((unsigned char *) xipt, 0);
} }
} }
/* let the expert decide where to add num_aapt */ /* let the expert decide where to add num_aapt */

View File

@ -10,18 +10,29 @@
/** /**
* This header defines the functions and structures needed to add RockRidge * This header defines the functions and structures needed to add RockRidge
* extensions to an ISO image. * extensions to an ISO image. It also handles AAIP and zisofs extensions.
* *
* References: * References:
* *
* - SUSP (IEEE 1281). * - SUSP (IEEE 1281).
* System Use Sharing Protocol, draft standard version 1.12. * System Use Sharing Protocol, draft standard version 1.12.
* See ftp://ftp.ymi.com/pub/rockridge/susp112.ps
* *
* - RRIP (IEEE 1282) * - RRIP (IEEE 1282)
* Rock Ridge Interchange Protocol, Draft Standard version 1.12. * Rock Ridge Interchange Protocol, Draft Standard version 1.12.
* See ftp://ftp.ymi.com/pub/rockridge/rrip112.ps
* *
* - ECMA-119 (ISO-9660) * - ECMA-119 (ISO-9660)
* Volume and File Structure of CDROM for Information Interchange. * Volume and File Structure of CDROM for Information Interchange. See
* http://www.ecma-international.org/publications/files/ECMA-ST/Ecma-119.pdf
*
* - AAIP
* Arbitrary Attribute Interchange Protocol. See doc/susp_aaip_2_0.txt
*
* - zisofs
* Blockwise compression of data file content with transparent read support
* in the Linux kernel. See doc/zisofs_format.txt
*
*/ */
#ifndef LIBISO_ROCKRIDGE_H #ifndef LIBISO_ROCKRIDGE_H
@ -133,6 +144,12 @@ struct aaip_AL {
}; };
/** zisofs entry (see doc/zisofs_format.txt) */
struct zisofs_ZF {
uint8_t parameters[1]; /* begins with BP 5 */
};
/** /**
* Struct for a SUSP System User Entry (SUSP, 4.1) * Struct for a SUSP System User Entry (SUSP, 4.1)
*/ */
@ -153,6 +170,7 @@ struct susp_sys_user_entry
struct rr_SL SL; struct rr_SL SL;
struct aaip_AA AA; struct aaip_AA AA;
struct aaip_AL AL; struct aaip_AL AL;
struct zisofs_ZF ZF;
} data; /* 5 to 4+len_sue */ } data; /* 5 to 4+len_sue */
}; };
@ -315,5 +333,14 @@ int read_aaip_AL(struct susp_sys_user_entry *sue,
unsigned char **aa_string, size_t *aa_size, size_t *aa_len, unsigned char **aa_string, size_t *aa_size, size_t *aa_len,
size_t *prev_field, int *is_done, int flag); size_t *prev_field, int *is_done, int flag);
/**
* Reads the zisofs parameters from a ZF field (see doc/zisofs_format.txt).
*
* @return
* 1 on success, < 0 on error
*/
int read_zisofs_ZF(struct susp_sys_user_entry *zf, uint8_t algorithm[2],
uint8_t *header_size_div4, uint8_t *block_size_log2,
uint32_t *uncompressed_size, int flag);
#endif /* LIBISO_ROCKRIDGE_H */ #endif /* LIBISO_ROCKRIDGE_H */

View File

@ -579,4 +579,31 @@ int read_aaip_AL(struct susp_sys_user_entry *sue,
return ISO_SUCCESS; return ISO_SUCCESS;
} }
/**
* Reads the zisofs parameters from a ZF field (see doc/zisofs_format.txt).
*
* @return
* 1 on success, < 0 on error
*/
int read_zisofs_ZF(struct susp_sys_user_entry *zf, uint8_t algorithm[2],
uint8_t *header_size_div4, uint8_t *block_size_log2,
uint32_t *uncompressed_size, int flag)
{
if (zf == NULL) {
return ISO_NULL_POINTER;
}
if (zf->sig[0] != 'Z' || zf->sig[1] != 'F') {
return ISO_WRONG_ARG_VALUE;
}
if (zf->len_sue[0] != 16) {
return ISO_WRONG_RR;
}
algorithm[0] = zf->data.ZF.parameters[0];
algorithm[1] = zf->data.ZF.parameters[1];
*header_size_div4 = zf->data.ZF.parameters[2];
*block_size_log2 = zf->data.ZF.parameters[3];
*uncompressed_size = iso_read_bb(&(zf->data.ZF.parameters[4]), 4, NULL);
return ISO_SUCCESS;
}