[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Octave-bug-tracker] [bug #54405] octave_idx_type index integer overflow
From: |
Dan Sebald |
Subject: |
[Octave-bug-tracker] [bug #54405] octave_idx_type index integer overflow math check doesn't work correctly |
Date: |
Sun, 29 Jul 2018 15:53:43 -0400 (EDT) |
User-agent: |
Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:55.0) Gecko/20100101 Firefox/55.0 |
Follow-up Comment #1, bug #54405 (project octave):
In the Array.h file is stated:
//! ### size_type
//!
//! Array::size_type is `octave_idx_type` which is a typedef for `int`
//! or `long int`, depending whether Octave was configured for 64-bit
//! indexing.
so I guess that octave_idx_type is a signed integer then. That means what I
had printed out with std::cerr is, in fact, how the NR and NC numbers are
treated by the compiler.
I changed the overflow check in fread() to the following (note the extra
return statement added):
// Check for overflow.
if (nr != 0 && abs (nc) > abs (std::numeric_limits<octave_idx_type>::max
() / nr))
{
std::cerr << "ERROR OCCURED\n";
error ("fread: dimension too large for Octave's index type");
return retval;
}
and the error is caught as desired. But still, the message is now
ERROR OCCURED
error: value on right hand side of assignment is undefined
I guess the error() message doesn't work the way I'm imagining. Or perhaps
the large octave_idx_type values are causing strange problems elsewhere.
Nonetheless, the signed index value seems to make checking overflow a little
more complex. I wonder if it would be better to change octave_idx_type to
unsigned and use the
if (nr != 0 && nc > (std::numeric_limits<octave_idx_type>::max () / nr))
construct without the complicating signed math such as abs() in the location
wherever that "out of memory or dimension too large for Octave's index type"
determination is made.
_______________________________________________________
Reply to this item at:
<http://savannah.gnu.org/bugs/?54405>
_______________________________________________
Message sent via Savannah
https://savannah.gnu.org/
- [Octave-bug-tracker] [bug #54405] octave_idx_type index integer overflow math check doesn't work correctly, Dan Sebald, 2018/07/29
- [Octave-bug-tracker] [bug #54405] octave_idx_type index integer overflow math check doesn't work correctly,
Dan Sebald <=
- [Octave-bug-tracker] [bug #54405] octave_idx_type index integer overflow math check doesn't work correctly, Dan Sebald, 2018/07/29
- [Octave-bug-tracker] [bug #54405] octave_idx_type index integer overflow math check doesn't work correctly, Dan Sebald, 2018/07/29
- [Octave-bug-tracker] [bug #54405] octave_idx_type index integer overflow math check doesn't work correctly, Rik, 2018/07/30
- [Octave-bug-tracker] [bug #54405] octave_idx_type index integer overflow math check doesn't work correctly, Dan Sebald, 2018/07/30
- [Octave-bug-tracker] [bug #54405] octave_idx_type index integer overflow math check doesn't work correctly, Dan Sebald, 2018/07/30
- [Octave-bug-tracker] [bug #54405] octave_idx_type index integer overflow math check doesn't work correctly, Dan Sebald, 2018/07/31
- [Octave-bug-tracker] [bug #54405] octave_idx_type index integer overflow math check doesn't work correctly, John W. Eaton, 2018/07/31
- [Octave-bug-tracker] [bug #54405] octave_idx_type index integer overflow math check doesn't work correctly, Rik, 2018/07/31
- [Octave-bug-tracker] [bug #54405] octave_idx_type index integer overflow math check doesn't work correctly, Rik, 2018/07/31
- [Octave-bug-tracker] [bug #54405] octave_idx_type index integer overflow math check doesn't work correctly, Dmitri A. Sergatskov, 2018/07/31