'expr' now detects and reports integer overflow

[Paul Eggert]

I installed the following in coreutils HEAD so that 'expr' detects and
reports integer overflow.  Formerly, 'expr' silently ignored integer
overflow, typically wrapping around, which is not a good idea in
general and in a few cases (x86 division, I suspect) meant that 'expr'
might dump core.

It's not easy writing test cases for this because the value of
INTMAX_MIN varies from host to host.

This implementation assumes that *, +, -, silently wrap around on
overflow, which is pretty generally true these days on coreutils
platforms.

2006-06-06  Paul Eggert  <address@hidden>

        * NEWS: The 'expr' command now detects and reports integer overflow.
        (It would be better to use extended precision instead, but that
        would be more work.)
        * src/expr.c (integer_overflow): New function.
        (eval4, eval3): Check for integer overflow.

--- NEWS        3 Jun 2006 09:04:05 -0000       1.382
+++ NEWS        7 Jun 2006 05:50:34 -0000
@@ -44,7 +44,8 @@ GNU coreutils NEWS                      
   (the anchor is ignored), or about regular expressions like A** (the
   second "*" is ignored).  expr now exits with status 2 (not 3) for
   errors it detects in the expression's values; exit status 3 is now
-  used only for internal errors like arithmetic overflow.
+  used only for internal errors (such as integer overflow, which expr
+  now checks for).
 
   ln now uses different (and we hope clearer) diagnostics when it fails.
   ln -v now acts more like FreeBSD, so it generates output only when
--- src/expr.c  12 Apr 2006 07:37:11 -0000      1.111
+++ src/expr.c  7 Jun 2006 05:58:57 -0000       1.113
@@ -175,6 +175,13 @@ syntax_error (void)
   error (EXPR_INVALID, 0, _("syntax error"));
 }
 
+/* Report an integer overflow for operation OP and exit.  */
+static void
+integer_overflow (char op)
+{
+  error (EXPR_FAILURE, ERANGE, "%c", op);
+}
+
 int
 main (int argc, char **argv)
 {
@@ -631,12 +638,26 @@ eval4 (bool evaluate)
          if (!toarith (l) || !toarith (r))
            error (EXPR_INVALID, 0, _("non-numeric argument"));
          if (fxn == multiply)
-           val = l->u.i * r->u.i;
+           {
+             val = l->u.i * r->u.i;
+             if (! (l->u.i == 0 || val / l->u.i == r->u.i))
+               integer_overflow ('*');
+           }
          else
            {
              if (r->u.i == 0)
                error (EXPR_INVALID, 0, _("division by zero"));
-             val = fxn == divide ? l->u.i / r->u.i : l->u.i % r->u.i;
+             if (l->u.i < - INTMAX_MAX && r->u.i == -1)
+               {
+                 /* Some x86-style hosts raise an exception for
+                    INT_MIN / -1 and INT_MIN % -1, so handle these
+                    problematic cases specially.  */
+                 if (fxn == divide)
+                   integer_overflow ('/');
+                 val = 0;
+               }
+             else
+               val = fxn == divide ? l->u.i / r->u.i : l->u.i % r->u.i;
            }
        }
       freev (l);
@@ -672,7 +693,18 @@ eval3 (bool evaluate)
        {
          if (!toarith (l) || !toarith (r))
            error (EXPR_INVALID, 0, _("non-numeric argument"));
-         val = fxn == plus ? l->u.i + r->u.i : l->u.i - r->u.i;
+         if (fxn == plus)
+           {
+             val = l->u.i + r->u.i;
+             if ((val < l->u.i) != (r->u.i < 0))
+               integer_overflow ('+');
+           }
+         else
+           {
+             val = l->u.i - r->u.i;
+             if ((l->u.i < val) != (r->u.i < 0))
+               integer_overflow ('-');
+           }
        }
       freev (l);
       freev (r);