[Duplicity-talk] Re: Help me replace tar!

[Joerg Schilling]

>From address@hidden  Sat Oct 18 01:57:05 2003

>On Fri, 17 Oct 2003 15:21:16 +0200 (CEST)
>Joerg Schilling <address@hidden> wrote:
>> THE way to go for an archiver is to use the extensible POSIX.1-2001
>>> format.  Star is the first tar implementation for the POSIX.1-2001
>>> tar format aka. PAX.  During the last two years, star did add a lot
>>> of features based on POSIX.1-2001 extensions. This allows star to
>>> implement things that other tar implementations cannot do.

>Ok, I'm starting to understand a bit better now.  So star complies
>with the SUSv3 description of pax?  (There does not seem to be any
>entry for "tar" itself.)  Then what is the status of the pax utility
>that I seem to have on my system, but never use?  And why is your
>project named "star" instead of "spax"?  (Aesthetic considerations
>aside of course :-))

The last standard that mentiones tar is SUSv2.

SUSv3 is now used as POSIX standard - the IEEE/POSIX commitee stopped 
working themselves on POSIX.

The reason why there is no tar in POSIX is the same why there is no cpio.
Early POSIX drafts in 1986 did not mention cpio. Then AT&T massively intervented
and requested the CPIO format to make it into the standard.
POSIX.1-1988 does not include commands at all - it only deals with interfaces 
as POSIX is a source standard!

PAX is a result of the idoim "When two are fighting, the third is happy."
They did what usually happend in standard committees: If they don't agree,
they use something everybody is equally unhappy with.

There is still no real interest and first pax implementations appeared around
1990 but star did start in 1982.

If you did take a look at star, you would know hat there already _is_ "spax".
However, it is only to also have command line interface compliance.
The spax command line only allows a minimal set of extensions to the standard 
and star still is and will remain _the_ way to go if you like the complete 
set of features.

>> Are you sure? I don't know any description of the archive format
>> used by GNU tar. All I could do when implementing GNU tar
>> compatibility inside star was to read the GNU tar source and check
>> the behavior :-( .... This is why I know that GNU tar is full of
>> bugs.

>Well, I'm referring to pages such as:

>http://www.gnu.org/manual/tar/html_node/tar_123.html#SEC118

Well it does not really help you as it does not mention the GNU extensions
in a usable way and for the rest, it is better to read the standard itself.

A big problem wit GNU tools is that they don'e follow the POSIX standard that
requires a man page. Instead they offer a nonstandard documentation that is 
only viewable with a nonstandard command line interface. For this reason, it is
much unlikely that you find the documentation although it might be there just
because you cannot search for it using standard methods.


>> The POSIX standard is at the Open Group. Go to www.opengroup.org and
>> register for free html view of the SUSv3 standard. This has been
>> taken as the POSIX.1-2001 by ISO.

>This paragraph explains why I wasn't able to find a copy before:

>    You are permitted to read the HTML and PDF versions of Open Group
>    publications using your HTML browser/Acrobat software and to
>    download them for your own personal use provided you have given
>    your name and email address for each publication
>    requested. However, you are NOT permitted to amend, copy, reprint,
>    offer for sale, or otherwise re-use material from these documents
>    without explicit permission from The Open Group.

>What kind of standard doesn't allow people to distribute it??  This

What is your problem? From what do you believe should the standard commitee be 
paid?

Also if the standard could be redistributed freely then Linux would thake the
texts and use them for the missing documentation. Do you really believe that
the companies that contribute to the standard like to make it Linux easier to
compete against their OS? 

>seems worse than no standard at all, because at least with no
>standard, you can write up a description of a project's intended
>behavior.  With this kind of license that may become copyright
>violation.

You are completely wrong: it only prohibits that people take the text that
is based on hard work and take it to compete against the people who did work on 
it. But this does not limit the usability of the standard itself!
If you like to have documentation for your software, write it yourself, use 
your own words and describe the _real_ behavior of _your_ program - not
what POSIX requires it to do.

>> -    GNU tar is far from the POSIX archive standard, so if you move
>>      to star, your machine may be used fine, but other people who
>>      only have GNU tar may get into problems because GNU tar may
>>      have problems to read even POSIX.1-1988 archives.

>I see, GNU tar may be holding everything up.  Have you talked to them?
>>From my brief interaction they don't seem to have too much time.
>Maybe you could just get your project accepted by GNU as the new
>version of GNU tar?  If that doesn't appeal to you, maybe you could
>contribute just enough code to gnu tar so it doesn't barf on basic
>star archives?

I started to make bug reports in 1993 - most of the bugs I reported in 1993 are 
still unfixed :-(

I even made suggestions how the hard deviations from the standard could be fixed
in a way that allows a smooth transition to no avail.

I stopped making bug reports in 1994 when I discovered that making the reports 
is just an effort spend in vain. 

The big problem with GNU tar is that is has been written by many  people who 
don't have the needed background knowledge and that it never has been 
rewritten to allow future maintainability.

A second important fact is that RMS believes that GNU tar is a key application
and for this reason he is constantly obligating the GNU tar maintainers
to do what he likes although he is missing the needed background knowledge.
The way RMS deals with maintainers already caused many GNU maintainers to 
give up.

RMS doesn't like me to work on official GNU projects because he knows that I
would reject his absurd wishes. I would never use the broken indentation style 
RMS
likes to see, I would never give up standard compliant man page in favor of 
a nonstandard info page and I would never switch to the outdated and broken GNU
portability system.

RMS is not interested in standards, he only likes to dominate the world with 
GNU.

If it would be simple to fix the GNU tar problems, then I even may have done it.
The problem is that because GNU tar is a big chunk of spaghetti code, it is 
really hard to find the code that causes bugs. GNU tar never has been tested 
cleanly. There is an interesting rule of thumb:

Every time I thought about a feature that was in GNU tar but not in star, I 
first thought about implementation problems related to this feature. It never
took more than 10 minutes to find a bug in GNU tar this way :-(


Jörg

-- 
 EMail:address@hidden (home) Jörg Schilling D-13353 Berlin
       address@hidden           (uni)  If you don't have iso-8859-1
       address@hidden   (work) chars I am J"org Schilling
 URL:  http://www.fokus.fraunhofer.de/usr/schilling 
ftp://ftp.berlios.de/pub/schily