Re: Regarding Octave's gcd()

[Przemek Klosowski]

On 03/19/2010 07:37 PM, Mansour Moufid wrote:
> Allow me to elaborate.
>
> Currently, Octave's gcd() doesn't correctly handle large integers. For example:
>
> octave:1>  gcd(69160909843710856058,12341318070000405756)
> ans =  2048
>
> The correct answer is 2606. (In fact, gcd() of any two large integers
> always returns a power of 2...) Note that there is no warning either.

That's because the above line computes the gcd() of two double precision 
floating point numbers which have about 16  significant digits. Your 
20-digit integer values are rounded and lose last four significant 
digits. They also are too big to fit in Octave's largest native integer 
type uint64,  not to speak of regular integer 32-bit values.

The only way to do this kind of stuff would be to use octave symbolic
toolbox and its variable precision arithmetic vpa(). I don't know if 
gcd() is capable of processing symbolic values returned by the vpa() 
function, and I can't check because it turns out that my Octave 3.2.3 
octave-forge-20090607 on Fedora 12 has buggy symbolic toolbox:

vpa("2")*vpa("2")
error: T& Array<T>::checkelem (2, -1, -1): range error