Re: [PATCH] OS X Resource Fork and FinderType support

[John Valdes]

On Mon, Aug 18, 2003 at 01:21:03PM -0400, David Botsch wrote:

> So, files on the Macintosh can have up to 3 parts, if you will (though
> the 3rd part is more a feature of the file system and not the file
> itself).
> 
> 1. Data fork - this is the data -- what we consider the file under most
> other file systems
> 2. Resource fork - as Martin said, this contains resources used by the
> program. Dialog boxes, icons, and any of a number of other pieces.
> 3. Metadata - file meta data contains two things
> 3a. File Creator - identifies the application that created the file. This
> allows you to double click on the file and have it opened in the
> application that created it.
> 3b. File Type Code - a 4 letter code identifying the type of the file

There are also other bits in the Metadata (traditionally called the
"Finder Info" or sometimes "File Info"), but the file creator & type
codes are probably the most important.

Just to give some historical context for why Mac files have multiple
forks, I believe this was done for efficiency from back in the day
(mid '80s) when Macs had 128KB of RAM and ran off floppy disks.  The
resource fork is effectively a file whose contents has a well defined
structure.  The structure presumedly makes it easy and/or quick to
find the data structures (aka resources) stored within the file (it
also makes it easy to replace/patch the resources without having to
say, recompile source in the case of application files).  The data
fork can then be used to hold unstructured data, or data that can
otherwise be treated as a stream of bytes.  Then by combining the two
forks into a single "file", file management is made easier (eg,
there's just one icon to drag around when copying files).  BTW,
PalmOS, which initially ran on hardware nearly identical to the
original Mac, also uses this idea of structured "resource" files and
unstructured "data" files (though in Palm's case they aren't combined
as forks into a single file).

More history: (yes, this will be made relevant to Cfengine; keep
reading ;) ) most application files had empty data forks while most
data files (eg, word processor documents) had empty resource forks,
though not always (eg, some applications sorted license/registration
codes in their data fork, while some documents stored window size and
position information or selection (highlight) information in the
resource fork).  Later, when Apple switched from the m68k processor to
the PowerPC processor, application files stored the ppc code in the
data fork (in COFF format or some similar format) while the m68k code
remained in the resource fork (in CODE resources); this way a single
application file (a so-called "fat" binary) could be used on either a
m68k mac or a ppc mac.  PPC-only applications lacked the CODE
resources, but they still used the resource fork for storing the
resource data structures (menus, dialogs, etc).  This was true for all
Mac OS up through and including Mac OS 9.  Mac OS X dropped the
multifork file format (its files only use the data fork), but since it
supports pre-Mac OS X applications (for compatibility), it still needs
to deal with multi-forked files.  Hence, in order to use Cfengine to
completely manage Mac OS X, it too will need to deal with multi-fork
files.

Yet more history: the multiple forks of Mac files are implemented via
the Mac OS filesystems, HFS and HFS+.  This means that storing Mac
files on non-Mac OS filesystems has traditionally been problematic.
The usual way to deal with this is to encode the mac file in
some format that combines the forks and finder info into a single file
or to store the forks/finder info into separate files on the non-Mac
OS filesystem.  The most common formats that are used are:

1) "Binhex 4.0" - a text encoding (a'la uuencode or base-64) that
   combines the data fork, resource fork & finderinfo into a single
   file.  Files in binhex format traditionally have a ".hqx" filename
   extension.

2) MacBinary - a binary encoding that effectively concatenates the 3
   pieces of a mac file together.  Typically uses ".bin" as the
   filename extension

3) AppleSingle - another binary encoding that stores all 3 mac file
   pieces together into one file.  Encoding defined by Apple

4) AppleDouble - uses two files to store mac files.  The data fork is
   placed in its own file while the resource fork and finder info are
   stored in a second file.  The file holding the data fork uses the
   same name as the original Mac file, while the file holding the
   resource fork/finder info uses the same name as the original
   prefixed by some character (or two).  Most commonly, this was a
   percent, so a file called "Foo" would have its data fork stored in
   a file called "Foo" and its resource fork in a file called "%Foo".
   In Mac OS X, Apple decided to use a dot-underscore instead (so
   "Foo" and "._Foo").

There are other variants, but these are the main ones.  The latter two
are described briefly in RFC 1740 (http://www.faqs.org/rfcs/rfc1741.html)

OK, enough history.  Back to Cfengine.  So the main issue with
Cfengine and multiforked files is getting Cfengine to read & write the
multiple forks regardless of the underlying filesystem.  This will be
an issue for both the server (cfservd) and client (cfagent) sides, as
cfservd could be running under Mac OS X too (in which case it'd be
nice if it could serve up multiforked files without them having to be
encoded or otherwise split up).  BTW, this issue may not be unique to
Mac filesystems, as I believe NTFS also supports multiforked files (in
Microsoft's case, I believe they call forks "file streams" instead);
don't know for what they're used though (probably for Macintosh File
Services under Windwos...).  Nevertheless, "multifork" capability may
be useful for Cfengine under Windows too.

I was going to comment on your patches, but this has gotten a bit long
already, so I'll follow up in a separate message.

John

-------------------------------------------------------------------------
John Valdes                        Department of Astronomy & Astrophysics
valdes@uchicago.edu                                 University of Chicago