[Gcl-devel] gcl on intel mac

[Camm Maguire]

Greetings!  Pleased to announce that I've got this working, though the
final incarnation is still not clear in my mind.

1) gcc-4.0, the standard on this box, passes all function calls
through a jump table.  gcc-4.2 removes the section entirely and does
calls directly.  This is of course one instruction faster.  The
former, however, allows one not to relocate the .text section at all,
as there are no external references.  (The other local section
referenced is a "pointers" section, which gcc-4.2 also emits.  This
definitely appears slower than say i386 Linux, as one has to do an
extra copy and address indirection for each referenced variable. But
there appears to be no choice here.)  

The reason why skipping the relocation might be important is because
the bfd library does not do this correctly, and gcl will have to
override some of its functions.  Which means expanding, not
contracting, our local patch, and this begins to appear fragile.  So
question one is whether gcc-4.2 needs supporting on the mac.  On a
totally open source platform this question is ridiculous as one must
keep abreast with the latest, but I've been told that Apple has
decided not to make latest gcc available for licensing reasons (latest
is 4.4, soon to be 4.5).  So what is the proper, hopefully stable
target for this port?

2) The relocation records for either compiler are experimentally
observed to be either SECTDIFF_32, PAIR_32, DISP32, or 32.  I don't
know if I can count on this, but this is all I can test.

3) I've been told that mach-o has a 64bit format that differs in some
respects.  Are these machines out there?  Are they different?  Are the
ppc machines still to be supported?  The ppc code (existing courtesy
of Aurelien) does work on Matt's machine with a few minor bit rot
changes.  But it does require a separate relocation file against a
much earlier version of the bfd library.  I have not merged this with
the intel code under one lib yet.  

This brings up the general question of the gcl 2.6.8 release.  I had
wanted to fix the mac port before releasing this.  Currently, all
Debian platforms have the latest which build all apps
(maxima,acl2,axiom,hol88) without error on all 13 Debian platforms.
All synchronous app versions are currently in Debian testing for the
next release.  Debian as you all know feeds Ubuntu.  I'm hesitant
about upgrading the local bfd library in the 2.6.8 source tree to the
latest upstream source at this point, but perhaps for no good reason.
Debian uses the externally supplied latest bfd without issue where it
uses bfd at all.  gclcvs uses the local tree for mips and alpha
patches, but this is not in heavy use yet.  I think Tim might on
occasion use the local bfd library in the gcl source, but I really
don't know who else uses this.  In any case, this might argue either
to backport the intel mac work to the earlier bfd lib we have
alongside Aurelien's stuff, or to write the reloc stuff without bfd
for the intel mac.

Along these lines, the local gmp sources in the gcl tree do not
compile on the intel mac either.  I had to compile upstream latest
externally, and patch that too (minor) to boot.  I've been told there
is some mechanism for the user to just install gmp on the mac, but I
don't know how hard this is, and whether therefore we need to provide
the local convenience copy.

4) There appears to be no explicit section flag (set by bfd at least)
which indicates that all relocation records refer to locally appended
"IMPORT" sections (jump_table, pointers).  So this is just an
assumption at this point, but one that appears to be robust against a
wide variety of code.

5) It would appear that we can either a) try to integrate patches into
bfd upstream, ... failed in the past, b) minimize a separately
maintained patch preferably in one wrapper function to
bfd_get_relocated_section_contents, or c) skip bfd and implement the
relocation directly for the four relocation types mentioned above,
and abort if others are found later.  This decision depends of course
on the bugs, so here they are:

a) No relocation records generated for jumptable and pointers
sections, so bfd_get_relocated_section_contents returns garbage for
these sections

b) bfd_mach_o_canonicalize_one_reloc miscalculates the target
symbol here:

      unsigned int num = BFD_MACH_O_GET_R_SYMBOLNUM (symnum);
      res->addend = 0;
      res->address = addr;
      if (symnum & BFD_MACH_O_R_EXTERN)
        sym = syms + num;
      else
        {
          bfd_mach_o_section *sect;
          bfd_mach_o_section *ssp=NULL;
          int j;

          assert (num != 0);
          assert (num <= mdata->nsects);

          /* sect=mdata->sections[num-1]; <-BUG */
          for (j=0;j<mdata->nsects;j++) /* Correct */
            if ((sect=mdata->sections[j]) && addr>=sect->addr && addr < 
sect->addr+sect->size)
              break;
          sym = sect->bfdsection->symbol_ptr_ptr;

          ...

c) The relocation types SECTDIFF_32 and PAIR_32, via their howto
definitions, erronously add in the output_section vma.  Aurelien ran
into the same I believe, leading to the solution of subtracting the
vma from the relocation "addend" in a howto 'special function'.  The
other platforms need this vma to be set as they use it correctly, so
we would have to either conditionally not set it on this platform, or
write a howto special function wrapper to set the addend, or directly
patch bfd_mach_o_canonicalize_one_reloc, which needs doing anyway
given the above.

d) Likewise, the PAIR_32 addend is set to the address offset
of the instruction, but this is already in place in the data, leading
to it being added twice.  So similarly, the addend needs this quantity
subtracted here.

e) pc_relative relocs (DISP32) subtract the instruction address in
bfd_get_relocated_section_contents, but the data is already relative
so this is redundant.  The addend needs correcting as above in this
case too. 

The current patch 1) provides two helper functions to set the pointer
and jumptable sections 2) overrides bfd_mach_o_canonicalize_one_reloc
with the one bug fix in b), and 3) wraps
bfd_get_relocated_section_contents to skip .text, and call the
functions in 1) when appropriate.  Supporting gcc-4.2 will require
relocating .text too, and patching bfd_mach_o_canonicalize_one_reloc
with the changes for c, d, and e as well.  This is all doable, I'm
mostly concerned with maintenance going forward.  Hence I'd love
suggestions and feedback.

Take care,
-- 
Camm Maguire                                        address@hidden
==========================================================================
"The earth is but one country, and mankind its citizens."  --  Baha'u'llah