[Octave-bug-tracker] [bug #65544] cross.m: vector dimension should produce error, can produce erroneous result

[Nicholas Jankowski]

URL:
  <https://savannah.gnu.org/bugs/?65544>

                 Summary: cross.m: vector dimension should produce error, can
produce erroneous result
                   Group: GNU Octave
               Submitter: nrjank
               Submitted: Sun 31 Mar 2024 07:32:26 PM EDT
                Category: Octave Function
                Severity: 3 - Normal
                Priority: 5 - Normal
              Item Group: Incorrect Result
                  Status: None
             Assigned to: None
         Originator Name: 
        Originator Email: 
             Open/Closed: Open
                 Release: 9.1.0
         Discussion Lock: Any
        Operating System: Any
           Fixed Release: None
         Planned Release: 9.2.0 (current stable)


    _______________________________________________________

Follow-up Comments:


-------------------------------------------------------
Date: Sun 31 Mar 2024 07:32:26 PM EDT By: Nicholas Jankowski <nrjank>
following on the discussion on bug #65527 with compiled functions not catching
non-scalar dims, realized that cross does the same but instead of just using
dim(1) and ignoring the rest, it can either produce not-overly-meaningful
errors or silently produce incorrect output:


a = magic(3);
b = eye(3);

cross(a,b)
ans =

   0  -9   7
   4   0  -6
  -3   1   0

cross(a,b,1)
ans =

   0  -9   7
   4   0  -6
  -3   1   0

cross(a,b,2)
ans =

   0   6  -1
  -7   0   3
   9  -4   0

cross(a,b,[1 2])
ans =

   3
  -6
   3

cross(a,b,[2 1])
ans =

   3  -6   3

cross(a,b,[3 1])
error: x(_,_,2): out of bound 1 (dimensions are 3x3)
error: called from
    cross at line 96 column 8


there is no vecdim compatibility concern here, even matlab requires it to be
an integer valued positive scalar.

also looks like there's a FIXME to add a lot more input validation.  working
up a patch now that will add an error if dim doesn't meet the conditions
above, and add BISTS against stable







    _______________________________________________________

Reply to this item at:

  <https://savannah.gnu.org/bugs/?65544>

_______________________________________________
Message sent via Savannah
https://savannah.gnu.org/