Introduced checksum tag for relocated 64 kB superblock on overwriteable media.

doc/checksums.txt

@@ -27,8 +27,8 @@ iso_md5_start(), iso_md5_compute(), iso_md5_clone(), iso_md5_end()
 
                           Representation in the Image
 
-The checksums are stored as stream recognizable checksum tags and as a compact
+There may be several stream recognizable checksum tags and a compact array
-array at the end of the session. The latter allows to quickly load many
+of MD5 items at the end of the session. The latter allows to quickly load many
 file checksums from media with slow random access.
 
 
@@ -69,26 +69,58 @@ array. (Then follows eventual padding.)
 The tags are a single lines of printable text, padded by 0 bytes. They have
 the following format:
 
- Tag_id pos=# range_start=# range_size=# [next=#] md5=# self=#\n
+ Tag_id pos=# range_start=# range_size=# [session_start|next=#] md5=# self=#\n
 
-Tag_id distinguishes the three tag types
+Tag_id distinguishes the following tag types
+  "libisofs_rlsb32_checksum_tag_v1"     Relocated 64 kB superblock tag
   "libisofs_sb_checksum_tag_v1"         Superblock tag
   "libisofs_tree_checksum_tag_v1"       Directory tree tag
   "libisofs_checksum_tag_v1"            Session tag
 
+A relocated superblock may appear at LBA 0 of an image which was produced for
+being stored in a disk file or on overwriteable media (e.g. DVD+R, BD-RE).
+Typically there is a first session recorded with a superblock at LBA 32 and
+the next session may follow shortly after its session tag. (There may be a gap
+of padding, often 150 blocks, and aligning to the next address that is
+divisible by 32.) Normally no session starts after the address given by
+parameter session_start=.
 
-Example (session starts at at Logical Block Address 32):
+Session oriented media like CD-R[W], DVD+R, BD-R will have no relocated
+superblock but rather bear a table-of-content on media level (to be inquired
+by MMC commands(.
 
+
+Example:
+A relocated superblock which points to the last session. Then the first session
+which starts at Logical Block Address 32. The following sessions have the same
+structure as the first one.
+
+LBA 0:
    <... ECMA-119 System Area and Volume Descriptors ...>
-libisofs_sb_checksum_tag_v1 pos=50 range_start=32 range_size=18 md5=17471035f1360a69eedbd1d0c67a6aa2 self=52d602210883eeababfc9cd287e28682
+LBA 18:
-   <... ECMA-119 Directory Entries ...>
+   libisofs_rlsb32_checksum_tag_v1 pos=18 range_start=0 range_size=18 session_start=311936 md5=6fd252d5b1db52b3c5193447081820e4 self=526f7a3c7fefce09754275c6b924b6d9
-libisofs_tree_checksum_tag_v1 pos=334 range_start=32 range_size=302 md5=41acd50285339be5318decce39834a45 self=fe100c338c8f9a494a5432b5bfe6bf3c
+   <... padding up to LBA 32 ...>
+LBA 32:
+   <... First Session: ECMA-119 System Area and Volume Descriptors ...>
+   libisofs_sb_checksum_tag_v1 pos=50 range_start=32 range_size=18 md5=17471035f1360a69eedbd1d0c67a6aa2 self=52d602210883eeababfc9cd287e28682
+   <... ECMA-119 Directory Entries (the tree of file names) ...>
+LBA 334:
+   libisofs_tree_checksum_tag_v1 pos=334 range_start=32 range_size=302 md5=41acd50285339be5318decce39834a45 self=fe100c338c8f9a494a5432b5bfe6bf3c
    <... Data file payload and checksum array ...>
-libisofs_checksum_tag_v1 pos=81554 range_start=32 range_size=81522 md5=8adb404bdf7f5c0a078873bb129ee5b9 self=57c2c2192822b658240d62cbc88270cb
+LBA 81554:
+   libisofs_checksum_tag_v1 pos=81554 range_start=32 range_size=81522 md5=8adb404bdf7f5c0a078873bb129ee5b9 self=57c2c2192822b658240d62cbc88270cb
+
+   <... more sessions ...>
+
+LBA 311936:
+   <... Last Session: ECMA-119 System Area and Volume Descriptors ...>
+LBA 311954:
+   libisofs_sb_checksum_tag_v1 pos=311954 range_start=311936 range_size=18 next=312286 md5=7f1586e02ac962432dc859a4ae166027 self=2c5fce263cd0ca6984699060f6253e62
+   <... Last Session: tree, tree checksum tag, data payload, session tag ...>
 
 
-There are five tag parameters. The first three are decimal numbers, the others
+There are several tag parameters. Addresses are given as decimal numbers, MD5
-are strings of 32 hex digits:
+checksums as strings of 32 hex digits.
 
   pos=
   gives the block address where the tag supposes itself to be stored.
@@ -117,6 +149,11 @@ are strings of 32 hex digits:
   uniquely placed directly after the ECMA-119 Volume Descriptor Set Terminator
   where no such cleartext is supposed to reside by accident.
 
+  Only with relocated 64 kB superblock tag:
+  session_start=
+  The start block address (System Area) of the session to which the relocated
+  superblock points.  
+
   md5=
   The checksum payload of the tag as lower case hex digits.
 
@@ -135,26 +172,39 @@ are strings of 32 hex digits:
                      Checking Before Image Tree Loading
 
 In order to check for a trustworthy loadable image tree, read the first 32
-blocks from to the session start and look in block 16 to 32 for the superblock
+blocks from to the session start and look in block 16 to 32 for a superblock
 checksum tag by
   iso_util_decode_md5_tag(block, &tag_type, &pos,
-                          &range_start, &range_size, &next_tag, md5, 2);
+                          &range_start, &range_size, &next_tag, md5, 0);
-If it appears and has plausible parameters, then check whether its MD5 matches
-the MD5 of the data blocks which were read before.
-(Keep the original MD5 context of the data blocks and clone one for obtaining
-the MD5 bytes.)
-Compute the block into the MD5 checksum after your are done with interpreting
-it.
 
-If those MD5s match, then compute the checksum block into the kept MD5 context
+If a tag of type 2 or 4 appears and has plausible parameters, then check
+whether its MD5 matches the MD5 of the data blocks which were read before.
+
+With tag type 2:
+
+Keep the original MD5 context of the data blocks and clone
+one for obtaining the MD5 bytes.
+If the MD5s match, then compute the checksum block into the kept MD5 context
 and go on with reading and computing for the tree checksum tag. This will be
 found at block address next_tag, verified and parsed by:
   iso_util_decode_md5_tag(block, &tag_type, &pos,
                           &range_start, &range_size, &next_tag, md5, 3);
+
 Again, if the parameters match the reading state, the MD5 must match the
 MD5 computed from the data blocks which were before.
 If so, then the tree is ok and safe to be loaded by iso_image_import().
 
+With tag type 4:
+
+End the MD5 context and start a new context for the session which you will
+read next.
+
+You may look for the first session by starting to read at LBA 32, or you may
+look for the last session by starting to read at the address given by parameter
+session_start=. The former is suitable for a check of the whole image, the
+latter is the shortest way to ensure that the tree of the last session is
+not corrupted.
+
 
                            Checking a Whole Session
 
@@ -175,6 +225,12 @@ read before the tag block.
 
                    Checking Single Files in a Loaded Image
 
+An image may consist of many sessions wherein many data blocks may not belong
+to files in the directory tree of the most recent session. Checking this
+tree and all its data files can ensure that all actually valid data in the
+image are trustworthy. This will leave out the trees of the older sessions
+and the obsolete data blocks of overwritten or deleted files.
+
 Once the image has been loaded, you can obtain MD5 sums from IsoNode objects
 which fulfill
   iso_node_get_type(node) == LIBISO_FILE
@@ -202,3 +258,8 @@ for reading file content from the loaded image.
 iso_image_get_session_md5() gives start LBA and session payload size as of
 "isofs.ca" and the session checksum as of the checksum array.
 
+For reading you may use the IsoDataSource object which you submitted
+to iso_image_import() when reading the image. If this source is associated
+to a libburn drive, then libburn function burn_read_data() can read directly
+from it.
+

libisofs/ecma119.c

@@ -1097,6 +1097,7 @@ int ecma119_image_new(IsoImage *src, IsoWriteOpts *opts, Ecma119Image **img)
     target->checksum_idx_counter = 0;
     target->checksum_ctx = NULL;
     target->checksum_counter = 0;
+    target->checksum_rlsb_tag_pos = 0;
     target->checksum_sb_tag_pos = 0;
     target->checksum_tree_tag_pos = 0;
     target->checksum_tag_pos = 0;
@@ -1104,6 +1105,7 @@ int ecma119_image_new(IsoImage *src, IsoWriteOpts *opts, Ecma119Image **img)
     target->checksum_array_pos = 0;
     target->checksum_range_start = 0;
     target->checksum_range_size = 0;
+    target->opts_overwrite = 0;
 
 #endif
 
@@ -1265,7 +1267,6 @@ int ecma119_image_new(IsoImage *src, IsoWriteOpts *opts, Ecma119Image **img)
 
     /* check if we need to provide a copy of volume descriptors */
     if (opts->overwrite) {
-
         /*
          * Get a copy of the volume descriptors to be written in a DVD+RW
          * disc
@@ -1318,6 +1319,28 @@ int ecma119_image_new(IsoImage *src, IsoWriteOpts *opts, Ecma119Image **img)
         vol->vol_desc_type[0] = 255;
         memcpy(vol->std_identifier, "CD001", 5);
         vol->vol_desc_version[0] = 1;
+
+#ifdef Libisofs_with_checksumS
+
+        /* Write relocated superblock checksum tag */
+        if (target->md5_session_checksum) {
+            target->checksum_rlsb_tag_pos = voldesc_size / BLOCK_SIZE + 16 + 1;
+            if (target->checksum_rlsb_tag_pos < 32) {
+                ret = iso_md5_start(&(target->checksum_ctx));
+                if (ret < 0)
+                    return ret;
+                target->opts_overwrite = (char *) opts->overwrite;
+                iso_md5_compute(target->checksum_ctx, target->opts_overwrite,
+                                target->checksum_rlsb_tag_pos * 2048);
+                ret = iso_md5_write_tag(target, 4);
+                target->opts_overwrite = NULL; /* opts might not persist */
+                if (ret < 0)
+                    goto target_cleanup;
+            }
+        }
+
+#endif /* Libisofs_with_checksumS */
+
     }
 
     /*
@@ -1333,6 +1356,8 @@ int ecma119_image_new(IsoImage *src, IsoWriteOpts *opts, Ecma119Image **img)
 #ifdef Libisofs_with_checksumS
     if (target->md5_session_checksum) {
         /* After any fake writes are done: Initialize image checksum context */
+        if (target->checksum_ctx != NULL)
+            iso_md5_end(&(target->checksum_ctx), target->image_md5);
         ret = iso_md5_start(&(target->checksum_ctx));
         if (ret < 0)
             return ret;

libisofs/ecma119.h

@@ -454,6 +454,7 @@ struct ecma119_image
     unsigned int checksum_idx_counter;
     void *checksum_ctx;
     off_t checksum_counter;
+    uint32_t checksum_rlsb_tag_pos;
     uint32_t checksum_sb_tag_pos;
     uint32_t checksum_tree_tag_pos;
     uint32_t checksum_tag_pos;
@@ -463,6 +464,15 @@ struct ecma119_image
     uint32_t checksum_range_start;
     uint32_t checksum_range_size;
 
+    char *opts_overwrite; /* Points to IsoWriteOpts->overwrite.
+                             Use only underneath ecma119_image_new()
+                             and if not NULL*/
+
+    /* ??? Is there a reason why we copy lots of items from IsoWriteOpts
+           rather than taking ownership of the IsoWriteOpts object which
+           is submitted with ecma119_image_new() ?
+     */
+
 #endif /* Libisofs_with_checksumS */
 
     /* Buffer for communication between burn_source and writer thread */

libisofs/libisofs.h

@@ -5057,6 +5057,7 @@ int iso_file_get_md5(IsoImage *image, IsoFile *file, char md5[16], int flag);
  *      1= session tag
  *      2= superblock tag
  *      3= tree tag
+ *      4= relocated 64 kB superblock tag (at LBA 0 of overwriteable media)
  * @param pos
  *      Returns the LBA where the tag supposes itself to be stored.
  *      If this does not match the data block LBA then the tag might be
@@ -5071,7 +5072,13 @@ int iso_file_get_md5(IsoImage *image, IsoFile *file, char md5[16], int flag);
  *      covered by parameter md5.
  * @param next_tag
  *      Returns the predicted block address of the next tag.
- *      Valid only with return values 2 and 3 and if not 0.
+ *      next_tag is valid only if not 0 and only with return values 2, 3, 4.
+ *      With tag types 2 and 3, reading shall go on sequentially and the MD5
+ *      computation shall continue up to that address.
+ *      With tag type 4, reading shall resume either at LBA 32 for the first
+ *      session or at the given address for the session which is to be loaded
+ *      by default. In both cases the MD5 computation shall be re-started from
+ *      scratch.
  * @param md5
  *      Returns 16 byte of MD5 checksum.
  * @param flag
@@ -5081,12 +5088,13 @@ int iso_file_get_md5(IsoImage *image, IsoFile *file, char md5[16], int flag);
  *                 1= session tag
  *                 2= superblock tag
  *                 3= tree tag
+ *                 4= relocated superblock tag
  * @return
  *      0= not a checksum tag, return parameters are invalid
- *      1= checksum tag found, return parameters arevalid
+ *      1= checksum tag found, return parameters are valid
  *     <0= error 
- *         return parameters are valid with error ISO_MD5_AREA_CORRUPTED
+ *         (return parameters are valid with error ISO_MD5_AREA_CORRUPTED
- *         but not trustworthy because the tag seems corrupted. 
+ *          but not trustworthy because the tag seems corrupted)
  *
  * @since 0.6.22
  */

libisofs/md5.c

@@ -659,6 +659,8 @@ int checksum_writer_create(Ecma119Image *target)
  *               1= session tag (End checksumming.)
  *               2= superblock tag (System Area and Volume Descriptors)
  *               3= tree tag (ECMA-119 and Rock Ridge tree)
+ *               4= relocated superblock tag (at LBA 0 of overwriteable media)
+ *                  Write to target->opts_overwrite rather than to iso_write().
  */
 int iso_md5_write_tag(Ecma119Image *t, int flag)
 {
@@ -668,27 +670,30 @@ int iso_md5_write_tag(Ecma119Image *t, int flag)
     int res, mode, l, i, wres;
     void *ctx = NULL;
     char md5[16], tag_block[2048];
-    uint32_t size= 0, pos, next_pos = 0;
+    uint32_t size = 0, pos = 0, start;
-    static char *tag_ids[4]= {"",
+    static char *tag_ids[] = {"",
                               "libisofs_checksum_tag_v1",
                               "libisofs_sb_checksum_tag_v1",
-                              "libisofs_tree_checksum_tag_v1"};
+                              "libisofs_tree_checksum_tag_v1",
+                              "libisofs_rlsb32_checksum_tag_v1"};
 
+    start = t->checksum_range_start;
     memset(tag_block, 0, 2048);
     mode = flag & 255;
     if (mode == 1) {
         res = iso_md5_end(&(t->checksum_ctx), md5);
         size = t->checksum_range_size;
         pos = t->checksum_tag_pos;
-    } else if (mode == 2 || mode == 3) {
+    } else if (mode >= 2 && mode <= 4) {
         if (mode == 2) {
             pos = t->checksum_sb_tag_pos;
-            next_pos = t->checksum_tree_tag_pos;
+        } else if (mode == 3) {
-        } else {
             pos = t->checksum_tree_tag_pos;
-            next_pos = t->checksum_tag_pos;
+        } else if (mode == 4) {
+            pos = t->checksum_rlsb_tag_pos;
+            start = pos - (pos % 32);
         }
-        size = pos - t->checksum_range_start;
+        size = pos - start;
         res = iso_md5_clone(t->checksum_ctx, &ctx);
         if (res < 0)
             return res;
@@ -699,13 +704,15 @@ int iso_md5_write_tag(Ecma119Image *t, int flag)
     if (res > 0) {
         sprintf(tag_block,
                 "%s pos=%u range_start=%u range_size=%u",
-                tag_ids[mode], pos,
+                tag_ids[mode], pos, start, size);
-                t->checksum_range_start, size);
+
         l = strlen(tag_block);
         if (mode == 2) {
             sprintf(tag_block + l, " next=%u", t->checksum_tree_tag_pos);
         } else if (mode == 3) {
             sprintf(tag_block + l, " next=%u", t->checksum_tag_pos);
+        } else if (mode == 4) {
+            sprintf(tag_block + l, " session_start=%u", t->ms_block);
         }
         strcat(tag_block + l, " md5=");
         l = strlen(tag_block);
@@ -726,11 +733,16 @@ int iso_md5_write_tag(Ecma119Image *t, int flag)
         }
         tag_block[l + 32] = '\n';
     }
+    if (mode == 4) {
+        if (t->opts_overwrite != NULL)
+            memcpy(t->opts_overwrite + pos * 2048, tag_block, 2048);
+    } else {
         wres = iso_write(t, tag_block, 2048);
         if (wres < 0) {
             res = wres;
             goto ex;
         }
+    }
     res = ISO_SUCCESS;
 ex:;
     if (ctx != NULL)

libisofs/util.c

@@ -1567,12 +1567,13 @@ int iso_util_decode_md5_tag(char data[2048], int *tag_type, uint32_t *pos,
                             uint32_t *range_start, uint32_t *range_size,
                             uint32_t *next_tag, char md5[16], int flag)
 {
-    static char *tag_magic[4] = {"",
+    static char *tag_magic[] = {"",
                                 "libisofs_checksum_tag_v1",
                                 "libisofs_sb_checksum_tag_v1",
-                                 "libisofs_tree_checksum_tag_v1"};
+                                "libisofs_tree_checksum_tag_v1",
-    static int magic_len[4]= {0, 24, 27, 29};
+                                "libisofs_rlsb32_checksum_tag_v1"};
-    int ret, bin_count, i, mode, magic_first = 1, magic_last = 3, found = 0;
+    static int magic_len[]= {0, 24, 27, 29, 31};
+    int ret, bin_count, i, mode, magic_first = 1, magic_last = 4;
     char *cpt, self_md5[16], tag_md5[16];
     void *ctx = NULL;
 
@@ -1614,6 +1615,13 @@ int iso_util_decode_md5_tag(char data[2048], int *tag_type, uint32_t *pos,
         ret = iso_util_dec_to_uint32(cpt + 5, next_tag, 0);
         if (ret <= 0)
             return 0;
+    } else if (*tag_type == 4) {
+        cpt = strstr(cpt, "session_start=");
+        if (cpt == NULL)
+            return(0);
+        ret = iso_util_dec_to_uint32(cpt + 14, next_tag, 0);
+        if (ret <= 0)
+            return 0;
     }
     cpt = strstr(cpt, "md5=");
     if (cpt == NULL)