[Libcdio-devel] [RFC] Alternative proposals: pragmatic-multiextent ts-multiextent

[Thomas Schmitt]

Hi,

i have (for now) completed the branches

  ts-multiextent

  pragmatic-multiextent

After due testing, one of these two should be merged with master.

3 of the 8 bug fix commits in pragmatic-multiextent apply to our
multiextent efforts only. But 5 of them fix inherited problems from
master.

Both proposals provide the old API. So applications which shall not
be augmented to deal with multi-extent files, need at worst to be
re-compiled to work with the new version of libiso9660.
Their behavior with multi-extent files will stay as buggy as before.

Application which want to surpass the 4 GiB - 1 limit of single-extent
files, need to use some new API components.


Pete, may i ask you to test pragmatic-multiextent in your programs
and with your ISO collection ?


----------------------------------------------------------------------

ts-multiextent introduces new iso9660_statv2_t with a bunch of methods.
It can deal with all multi-extent situations which comply to ISO 9660
aka ECMA-119 specs.
The limitation to 10,000 extents per file is just a protection against
memory waste with ill ISO images. (ISO_BLOCKSIZE + 1 = 2049 extents of
4 GiB - ISO_BLOCKSIZE are enough to fill the maximum size of an ISO.)

ts-multiextent is ABI compatible to master and will not cause API/ABI
breakage if the wish for enhancement arises in future.
Not to forget that it improves the inner architecture of libiso9660 by
consolidating memory management aspects in dedicated functions.

This intrusiveness of my work might have introduced regressions.
(I think it is now better tested than master, though. valgrind clean.)


Taking full advantage of iso9660_statv2_t imposes quite some work on
application programmers. See the paragraphs under "The old API around
iso9660_stat_t is still provided without change" in
  http://lists.gnu.org/archive/html/libcdio-devel/2018-07/msg00005.html

But by reply "false" to API call  iso9660_statv2_has_extent_gaps() ,
an application can recognize and reject files which do not comply to
the expectation that they can be read sequentially starting at
  extent[0].lsn
up to the byte count that is told by
  iso9660_statv2_get_total_size()
All other aspects of a file can be read from an old iso9660_stat_t
which can be obtained by call
  iso9660_statv2_get_v1()

This is not much more effort than adapting an application to the proposal
in branch pragmatic-multiextent. It will not bring more exotic multi-extent
capability than with pragmatic-multiextent, of course.

----------------------------------------------------------------------

pragmatic-multiextent is a stripped-down version of pbatard-multiextent2
which can create iso9660_stat_t objects only from files of which the
extents form a sequentially readable byte string.

If this expectation is not fulfilled, then pragmatic-multiextent issues
a warning message and does not create an iso9660_stat_t object for the
affected file. I.e. the file will not appear in readdir()-ish results and
individual inquiries will yield NULL rather than an iso9660_stat_t object.

mkisofs and libisofs currently only produce the expected simple extent
layout. Up to now, no other kind of multi-extent files have been found
in the wild. (I had to fake a test ISO with unaligned extent size.
An ISO where extents hop forth and back has still to be counterfeit.)

pragmatic-multiextent breaks the current ABI for once.
But because it has no limitation for the number of extents, it does not
provide the main foreseeable reason to expand pbatard-multiextent2 and
thus break ABI again.

Regressions are less likely than in ts-multiextent.
On the other hand i found three bugs in their common ancestor
pbatard-multiextent2, which i might have replaced by own ones.
So it's not really a low risk change.


Applications only have to switch from using
  iso9660_stat_t.size
to using new member
  iso9660_stat_t.total_size
and to make sure that their loop controlling variables can cope with
integer numbers up to 43 bit (32 bit block address, 11 bit byte address in
block).

(Several programs in pragmatic-multiextent are not yet using the new API
 component. I only changed those for now, where pbatard-multiextent2 had
 introduced the use of its API changes.)

----------------------------------------------------------------------

Have a nice day :)

Thomas