Re: mkisofs mastering sparse files - end result file full of zeros

[Thomas Schmitt]

Hi,

i wrote:
> > They really should not simply call xorrisofs "mkisofs" ...

Graham Leggett wrote:
> Been very careful to contact the correct place :)

I'm glad you found it.
(That's one of the reasons why one cannot silence the initial version
message of xorriso.)


> Once mastered to the filesystem, burned, and then read back,

Which program runs with which arguments exactly did you perform ?

(I still have to emphasize that the way how libisofs reads files should
not be affected by the way how the local filesystem stores the files.
libisofs reads each single byte and writes it into the ISO filesystem,
regardless whether really stored or virtually generated as 0-byte from
file gaps.)


> Analysing the file shows the whole file is just zeros, beginning to end.
> [...]
> Is there a way to test this by creating a purely sparse file, and then
> concatenating it with another not-sparse file, and then check if the sha256
> sigs are the same after the burn process? The checksums are different for
> me.

I now tried with data at start, middle, and end of the file:

  $ rm 1gb_sparse
  $ echo hello_start > 1gb_sparse
  $ echo hello_middle | dd bs=1 seek=512M of=1gb_sparse
  13+0 records in
  13+0 records out
  13 bytes (13 B) copied, 0.000176127 s, 73.8 kB/s
  $ echo hello_end | dd bs=1 seek=1G of=1gb_sparse
  10+0 records in
  10+0 records out
  10 bytes (10 B) copied, 0.000188871 s, 52.9 kB/s
  $ du 1gb_sparse
  12      1gb_sparse
  $ od -t x1 1gb_sparse
  0000000 68 65 6c 6c 6f 5f 73 74 61 72 74 0a 00 00 00 00
  0000020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  *
  4000000000 68 65 6c 6c 6f 5f 6d 69 64 64 6c 65 0a 00 00 00
  4000000020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  *
  10000000000 68 65 6c 6c 6f 5f 65 6e 64 0a
  10000000012
  $ md5sum 1gb_sparse
  4521a86a215a5647551ea0c9ce266551  1gb_sparse
  $ sha256sum 1gb_sparse
  903eb7b19989ebb70d042aa4628ef91f8fa147715620d95845e60c0e291774f2  1gb_sparse
  $

So the file is sparse with non-zero data at start, middle, and end.
(One could use a shell loop to grow the file from 0-size in smaller
steps of e.g. 1M in order to get more non-zero content.)

I make the ISO image with option --for_backup which causes MD5 sums to
be recorded in the metadata of the ISO filesystem:

  $ xorriso-1.5.4 -as mkisofs -o test.iso --for_backup 1gb_sparse
  ...
  ISO image produced: 524474 sectors
  ...

and use xorriso means to verify that it still is as xorriso wrote it:

  $ xorriso -for_backup -indev test.iso -check_md5_r sorry / -- -find / -exec 
get_md5 --
  ...
  File contents and their MD5 checksums match.
  4521a86a215a5647551ea0c9ce266551  /1gb_sparse
  $

Command -check_md5_r confirms that the recorded MD5 in the metadata of the
ISO filesystem matches the MD5 computed from the file content in the ISO.
Command -find -exec get_md5 shows the recorded MD5.

I mount the filesystem and let local tools compute checksums:

  $ sudo mount test.iso /mnt/iso
  mount: /dev/loop0 is write-protected, mounting read-only
  $ md5sum /mnt/iso/1gb_sparse
  4521a86a215a5647551ea0c9ce266551  /mnt/iso/1gb_sparse
  $ sha256sum /mnt/iso/1gb_sparse
  903eb7b19989ebb70d042aa4628ef91f8fa147715620d95845e60c0e291774f2  
/mnt/iso/1gb_sparse
  $

Comparing the checksums by copy+paste and shell means:
  $ test 4521a86a215a5647551ea0c9ce266551 = 4521a86a215a5647551ea0c9ce266551 && 
echo YES
  YES
  $ test 903eb7b19989ebb70d042aa4628ef91f8fa147715620d95845e60c0e291774f2 = 
903eb7b19989ebb70d042aa4628ef91f8fa147715620d95845e60c0e291774f2 && echo YES
  YES
  $

--------------------------------------------------------------------------

My unfounded guesses would be that you give xorrisofs a different file
than you believe to do, or that the way of extracting the file from the
burnt medium somehow zeroes it.

The xorriso check run would work with /dev/sr0, too:

  xorriso -for_backup -indev /dev/sr0 -check_md5_r sorry / -- -find / -exec 
get_md5 --

(But the ISO would have to be recorded with --for_backup in order to
bear MD5 sums for -check_md5_r and -find -exec get_md5.)


Have a nice day :)

Thomas