Ad: Re: join

[Maarten . Vanneste]

Thank you very much again for your quick replies on this matter. I had
thought that it would work on numerically ordered files, which may thus not
necessarily be the case, and indeed, there was a change from something 9000
to over 12000 more or less where the program failed...
In the meanwhile I am upgrading the cygwin installation here.

Kind regards,
Maarten

=================
Dr. Maarten Vanneste
International Centre for Geohazards - Norwegian Geotechical Institute
Oslo, Norway
www.geohazards.no
www.ngi.no


                                                                           
             Paul Eggert                                                   
             <address@hidden                                             
             DU>                                                       Til 
                                       address@hidden             
             13/03/2007 23:06                                           cc 
                                       address@hidden               
                                                                      Emne 
                                       Re: join                            
                                                                           
                                                                           
                                                                           
                                                                           
                                                                           
                                                                           




address@hidden writes:

> This suggests that
> the join command has a maximum number of lines it can read, is this
correct?

It depends on your machine and OS.  For example, if you have many
lines with the same key 'join' has to keep them all in memory, and if
they don't fit in memory it will fail.  It has to count the lines,
too, and the counter is of limited size.

> From my test, it turns out to be somewhere around 6600 lines after which
> the program terminates.

Most likely the problem is somewhere else, then.  On 32-bit machines
the limit is around 4 billion bytes' worth of lines (or less if your
computer has less memory).

I'd guess your input files were not sorted.





_____________________________
Neither the confidentiality nor the integrity of this message can be
guaranteed following transmission on the Internet.  The addressee should
consider this risk and take full responsibility for use of this message.
This e-mail has been swept by Norman antivirus software.