1.2. Precision

1.2.1. Rationale

>>> 0.1
0.1

>>> 0.2
0.2

>>> 0.3
0.3

>>> 0.1 + 0.2 == 0.3
False

>>> 0.1 + 0.2
0.30000000000000004

>>> round(0.1+0.2, 16) == 0.3
True

>>> round(0.1+0.2, 17) == 0.3
False

1.2.2. IEEE 754 standard

>>> 1.234 == 1234 * 10e-4
True

>>> 1234 * 10e-4
1.234
../_images/numpy-precision-float-anatomy.png

Figure 1.6. What is float as defined by IEEE 754 standard

../_images/numpy-precision-float-expression.png

Figure 1.7. Points chart

../_images/numpy-precision-float-mantissa-1.png

Figure 1.8. How computer store float? As defined by IEEE 754 standard

../_images/numpy-precision-float-mantissa-2.png

Figure 1.9. How to read/write float from/to memory?

../_images/numpy-precision-float-normalized.png

Figure 1.10. Normalized Line

1.2.3. Solutions

  • Round values to 4 decimal places (generally acceptable)

  • Store values as int, do operation and then divide. For example instead of 1.99 USD, store price as 199 US cents

  • Use Decimal type

  • Decimal type is much slower

Code 1.29. Problem
candy = 0.10      # price in dollars
cookie = 0.20     # price in dollars

result = candy + cookie
print(result)
# 0.30000000000000004
Code 1.30. Round values to 4 decimal places (generally acceptable)
candy = 0.10      # price in dollars
cookie = 0.20     # price in dollars

result = round(candy + cookie, 4)
print(result)
# 0.3
Code 1.31. Store values as int, do operation and then divide.
candy = 10        # price in cents
cookie = 20       # price in cents

result = (candy + cookie) / 100   # divide by 100 (number of cents in dollar)
print(result)
# 0.30
Code 1.32. Use Decimal type
from decimal import Decimal


candy = Decimal('0.10')     # price in dollars
cookie = Decimal('0.20')    # price in dollars

result = candy + cookie
print(result)
# 0.30