[Octave-patch-tracker] [patch #10314] matlab compatible std, var, mean, median to core from statistics package

[Rik]

Follow-up Comment #6, patch #10314 (project octave):

I don't think there's a problem with high-dimensional data as long as you are
using a 64-bit octave_idx_type which just about everyone is these days.  If
you want to use 32-bit types on a Raspberry Pi you can do so, but that's not
really a platform for high performance numerical computing.

The reason is that the number of elements on the squashed dimension is


sz = size (data);
nel = prod (sz(squashed_dimensions));


In order to overflow the octave_idx_type which is int64_t the number of
elements would have to exceed 2^63 - 1.  If this was a double array then each
element would take 8 bytes and this would be 2^66 bytes.  I don't know any
machine that has that type of memory.  Hence, there would have been an
out-of-memory error when the attempt was made to create the array well before
any squashing could happen.


    _______________________________________________________

Reply to this item at:

  <https://savannah.gnu.org/patch/?10314>

_______________________________________________
Message sent via Savannah
https://savannah.gnu.org/