Weird behaviour

exercise No. 253

Q:

We consider a method computing a given int's square value: 

static public int getSquare (int value) {
  return value * value;
}

A larger piece of software using this method produces erroneous results. Hunting for errors the developing team adds a println() statement: 

static public int getSquare (int value) {
  final int result = value * value;
  IO.println("Square: " + result);
  return result;
}

The team is baffled when negative results show up:

Square: -1757895751

  1. Explain the underlying reason.

  2. You are allowed to modify the method's return type. Provide a solution.

A:

  1. The problem is caused by arithmetic overflow errors. The largest possible int is 2 31 - 1 or 2147483647. Its square root is 46340.95... .Thus up to 46340 all squares are being calculated correctly e.g.:

    Code Result 
    IO.println(46340 * 46340);
    		 
    2147395600

    The product 46341 * 46341 of type int for example exceeds 2 31 - 1 for the very first time. Due to an int's four-byte two complement representation we have:

    Code Result 
    IO.println(46341 * 46341);
    		 
    -2147479015

    Starting from 46341 all computed squares are simply wrong due to arithmetic overflow regardless of their result's sign.

  2. Solving the issue requires a data type accommodating squares of arbitrary int values. The smallest int value (having maximum amount) is - 2 31 . Its square is thus 2 62 representing the method's largest possible result. Choosing return type long allows for a maximum value of 2 63 - 1 being ways larger than required. Replacing the int return type by long and correcting the square multiplication solves the issue: 

    static public long getSquare (int value) {
  final long result = (long) value * value; // Cast: long * int expression
  return result;
}

    Due to a cast's higher precedence the above expression is equivalent to ((long) value) * value.
Prev  Up  Next
Return type  Home  SD1 examination winter 2019