[Gcl-devel] Re: gcl on intel mac

[Camm Maguire]

Greetings!

Tim Daly <address@hidden> writes:

> Camm,
>
> I have to say that I'm unable to generate opinions on any of
> the points you made. The last linker-related code I wrote was
> on an IBM Series/1 in 1978 so I'm a little behind on the details.
>
> All I'm concerned about is that I have some version of GCL
> that can run Axiom on all of the platforms. I'm happy to supply
> or elide any options needed to run on intel macs (e.g. don't
> use local-bfd).
>

Does this mean that the builder of axiom on mac should be counted on
to install external gmp, certain gcc versions, and/or bfd?  Or is this
in general beyond the target audience, and gcl's local copies should e
relied upon?

> In your position I'd make whatever choice minimizes your future
> maintenance work.
>

Would of course love to externalize gmp and bfd.  Alas this cannot
quite be done yet, but I'd like evolution to point in this direction.

> I tried to upgrade my Xcode distribution on the mac but Apple
> seems to make that much more difficult than it needs to be.
>

Perhaps this answers my question above.  In brief, can we at least
rely on an externally installed gmp?

Take care,

> Camm Maguire wrote:
>> Greetings!  Pleased to announce that I've got this working, though the
>> final incarnation is still not clear in my mind.
>>
>> 1) gcc-4.0, the standard on this box, passes all function calls
>> through a jump table.  gcc-4.2 removes the section entirely and does
>> calls directly.  This is of course one instruction faster.  The
>> former, however, allows one not to relocate the .text section at all,
>> as there are no external references.  (The other local section
>> referenced is a "pointers" section, which gcc-4.2 also emits.  This
>> definitely appears slower than say i386 Linux, as one has to do an
>> extra copy and address indirection for each referenced variable. But
>> there appears to be no choice here.)  
>>
>> The reason why skipping the relocation might be important is because
>> the bfd library does not do this correctly, and gcl will have to
>> override some of its functions.  Which means expanding, not
>> contracting, our local patch, and this begins to appear fragile.  So
>> question one is whether gcc-4.2 needs supporting on the mac.  On a
>> totally open source platform this question is ridiculous as one must
>> keep abreast with the latest, but I've been told that Apple has
>> decided not to make latest gcc available for licensing reasons (latest
>> is 4.4, soon to be 4.5).  So what is the proper, hopefully stable
>> target for this port?
>>
>> 2) The relocation records for either compiler are experimentally
>> observed to be either SECTDIFF_32, PAIR_32, DISP32, or 32.  I don't
>> know if I can count on this, but this is all I can test.
>>
>> 3) I've been told that mach-o has a 64bit format that differs in some
>> respects.  Are these machines out there?  Are they different?  Are the
>> ppc machines still to be supported?  The ppc code (existing courtesy
>> of Aurelien) does work on Matt's machine with a few minor bit rot
>> changes.  But it does require a separate relocation file against a
>> much earlier version of the bfd library.  I have not merged this with
>> the intel code under one lib yet.  
>>
>> This brings up the general question of the gcl 2.6.8 release.  I had
>> wanted to fix the mac port before releasing this.  Currently, all
>> Debian platforms have the latest which build all apps
>> (maxima,acl2,axiom,hol88) without error on all 13 Debian platforms.
>> All synchronous app versions are currently in Debian testing for the
>> next release.  Debian as you all know feeds Ubuntu.  I'm hesitant
>> about upgrading the local bfd library in the 2.6.8 source tree to the
>> latest upstream source at this point, but perhaps for no good reason.
>> Debian uses the externally supplied latest bfd without issue where it
>> uses bfd at all.  gclcvs uses the local tree for mips and alpha
>> patches, but this is not in heavy use yet.  I think Tim might on
>> occasion use the local bfd library in the gcl source, but I really
>> don't know who else uses this.  In any case, this might argue either
>> to backport the intel mac work to the earlier bfd lib we have
>> alongside Aurelien's stuff, or to write the reloc stuff without bfd
>> for the intel mac.
>>
>> Along these lines, the local gmp sources in the gcl tree do not
>> compile on the intel mac either.  I had to compile upstream latest
>> externally, and patch that too (minor) to boot.  I've been told there
>> is some mechanism for the user to just install gmp on the mac, but I
>> don't know how hard this is, and whether therefore we need to provide
>> the local convenience copy.
>>
>> 4) There appears to be no explicit section flag (set by bfd at least)
>> which indicates that all relocation records refer to locally appended
>> "IMPORT" sections (jump_table, pointers).  So this is just an
>> assumption at this point, but one that appears to be robust against a
>> wide variety of code.
>>
>> 5) It would appear that we can either a) try to integrate patches into
>> bfd upstream, ... failed in the past, b) minimize a separately
>> maintained patch preferably in one wrapper function to
>> bfd_get_relocated_section_contents, or c) skip bfd and implement the
>> relocation directly for the four relocation types mentioned above,
>> and abort if others are found later.  This decision depends of course
>> on the bugs, so here they are:
>>
>> a) No relocation records generated for jumptable and pointers
>> sections, so bfd_get_relocated_section_contents returns garbage for
>> these sections
>>
>> b) bfd_mach_o_canonicalize_one_reloc miscalculates the target
>> symbol here:
>>
>>       unsigned int num = BFD_MACH_O_GET_R_SYMBOLNUM (symnum);
>>       res->addend = 0;
>>       res->address = addr;
>>       if (symnum & BFD_MACH_O_R_EXTERN)
>>         sym = syms + num;
>>       else
>>         {
>>           bfd_mach_o_section *sect;
>>        bfd_mach_o_section *ssp=NULL;
>>        int j;
>>
>>           assert (num != 0);
>>           assert (num <= mdata->nsects);
>>
>>        /* sect=mdata->sections[num-1]; <-BUG */
>>        for (j=0;j<mdata->nsects;j++) /* Correct */
>>          if ((sect=mdata->sections[j]) && addr>=sect->addr && addr < 
>> sect->addr+sect->size)
>>            break;
>>           sym = sect->bfdsection->symbol_ptr_ptr;
>>
>>           ...
>>
>> c) The relocation types SECTDIFF_32 and PAIR_32, via their howto
>> definitions, erronously add in the output_section vma.  Aurelien ran
>> into the same I believe, leading to the solution of subtracting the
>> vma from the relocation "addend" in a howto 'special function'.  The
>> other platforms need this vma to be set as they use it correctly, so
>> we would have to either conditionally not set it on this platform, or
>> write a howto special function wrapper to set the addend, or directly
>> patch bfd_mach_o_canonicalize_one_reloc, which needs doing anyway
>> given the above.
>>
>> d) Likewise, the PAIR_32 addend is set to the address offset
>> of the instruction, but this is already in place in the data, leading
>> to it being added twice.  So similarly, the addend needs this quantity
>> subtracted here.
>>
>> e) pc_relative relocs (DISP32) subtract the instruction address in
>> bfd_get_relocated_section_contents, but the data is already relative
>> so this is redundant.  The addend needs correcting as above in this
>> case too. 
>>
>> The current patch 1) provides two helper functions to set the pointer
>> and jumptable sections 2) overrides bfd_mach_o_canonicalize_one_reloc
>> with the one bug fix in b), and 3) wraps
>> bfd_get_relocated_section_contents to skip .text, and call the
>> functions in 1) when appropriate.  Supporting gcc-4.2 will require
>> relocating .text too, and patching bfd_mach_o_canonicalize_one_reloc
>> with the changes for c, d, and e as well.  This is all doable, I'm
>> mostly concerned with maintenance going forward.  Hence I'd love
>> suggestions and feedback.
>>
>> Take care,
>>   
>
>
>
>

-- 
Camm Maguire                                        address@hidden
==========================================================================
"The earth is but one country, and mankind its citizens."  --  Baha'u'llah