Accuracy Information

Numeric Accuracy

Any displayed number is a rounded representation of an internally stored 13-digit value. This internal value, not the displayed number, is used during calculations.

The additional digits kept internally are referred to as "guard digits". Although you can usually disregard these digits, they can be important in interpreting unexpected results.

As an example of the effect of guard digits, it is possible for an expression equal to zero to produce a non-zero result ( for example 1 ÷ 3 × 3 - 1 ).


Press: 1 [ ÷ ] 3 [ × ] and the display shows: D_aQTxtk

Press: 3 [ - ] 1 [ = ] and the display shows: D_Q6bWtN

Differences in guard digits are especially important if you write a program that compares two values for equality.

If you suspect these differences are responsible for an unexpected result of a comparison, use the ROUND numeric function before making the comparison. This sets the internal value of the number to the value shown in the display.

Internal Values


The 13 digits of the mantissa are displayed when you use the [ 2nd ] [ 13d ] key sequence.

All the digits of the internal value are shown in the unformatted mode ( a selection of [ CONV ] { BAS } ). An unformatted number has three parts:

  • The left 13 digits are the base 10 mantissa with the decimal implied after the first digit.
  • The 14th digit conveys the sign of both the mantissa and the exponent. ( Because the signs are indicated by a digit, the [ +/- ] key has no effect in this mode).
  • The last two digits are the exponent of scientific notation.

The main use of the unformatted mode is the entry of hexadecimal values for CIO (call I/O) instead of numeric calculations.

Mantissa Sign: + Exponent Sign: + Sign Digit: 0
Mantissa Sign: – Exponent Sign: + Sign Digit: 4
Mantissa Sign: + Exponent Sign: – Sign Digit: 8
Mantissa Sign: – Exponent Sign: – Sign Digit: C

☚ Back