16.1. Test Doctest

  • Tests are always the most up-to-date code documentation

  • Tests cannot get out of sync from code

  • Checks if function output is exactly as expected

  • Useful for regex modifications

  • Can add text (i.e. explanations) between tests

Use Cases:

16.1.1. Docstring

  • Docstring is a first multiline comment in: File/Module, Class, Method/Function

  • Used for generating help() documentation

  • It is accessible in __doc__ property of an object

  • Used for doctest

  • PEP 257 -- Docstring Conventions: For multiline str always use three double quote (""") characters

  • More information in Function Doctest

Docstring used for documentation:

>>> def say_hello():
...     """This is the say_hello function"""
...     print('Hello')
>>>
>>>
>>> 
... help(say_hello)
Help on function say_hello in module __main__:

say_hello()
    This is the say_hello function
>>>
>>> print(say_hello.__doc__)
This is the say_hello function

Docstring used for documentation:

>>> def say_hello():
...     """
...     This is the say_hello function
...     And the description is longer then one line
...     """
...     print('Hello')
>>>
>>>
>>> help(say_hello)  
Help on function say_hello in module __main__:

say_hello()
    This is the say_hello function
    And the description is longer then one line
>>>
>>> print(say_hello.__doc__)

    This is the say_hello function
    And the description is longer then one line

16.1.2. Syntax

  • Docstring is a first multiline comment in: File/Module, Class, Method/Function

  • Used for generating help() documentation

  • It is accessible in __doc__ property of an object

  • Used for doctest

  • PEP 257 -- Docstring Conventions: For multiline str always use three double quote (""") characters

>>> def add(a, b):
...     """
...     >>> add(1, 2)
...     3
...     >>> add(-1, 1)
...     0
...     >>> add(0, 0)
...     0
...     """
...     return a + b

>>> 
... """
... >>> add(1, 2)
... 3
... >>> add(-1, 1)
... 0
... >>> add(0, 0)
... 0
... """
>>>
>>> def add(a, b):
...     return a + b

16.1.3. Running Tests

Running tests in Pycharm IDE (either option):

  • Right click on source code with doctests -> Run 'Doctest for ...'

  • View menu -> Run... -> Doctest in myfunction

  • Note, that doctests are not discovered in scratch files in PyCharm

Running Tests from Python Code:

>>> if __name__ == "__main__":
...     from doctest import testmod
...     testmod()

Running tests from command line (displays errors only):

$ python -m doctest myfile.py

Add -v to display more verbose output.

$ python -m doctest -v myfile.py

16.1.4. Test Int, Float

int values:

>>> 
... """
... >>> add(1, 2)
... 3
... >>> add(-1, 1)
... 0
... >>> add(0, 0)
... 0
... """
>>>
>>> def add(a, b):
...     return a + b

float values:

>>> 
... """
... >>> add(1.0, 2.0)
... 3.0
...
... >>> add(0.1, 0.2)
... 0.30000000000000004
...
... >>> add(0.1, 0.2)   
... 0.3000...
... """
>>>
>>> def add(a, b):
...     return a + b

This is due to the floating point arithmetic in IEEE 754 standard:

>>> 0.1 + 0.2
0.30000000000000004

>>> 0.1 + 0.2  
0.3000...

>>> round(0.1+0.2, 16)
0.3

>>> round(0.1+0.2, 17)
0.30000000000000004

More information in Math Precision

16.1.5. Test Bool

>>> 
... """
... Function checks if person is adult.
... Adult person is over 18 years old.
...
... >>> is_adult(18)
... True
...
... >>> is_adult(17.9)
... False
... """
>>>
>>> AGE_ADULT = 18
>>>
>>> def is_adult(age):
...     if age >= AGE_ADULT:
...         return True
...     else:
...         return False

16.1.6. Test Str

  • Python will change to single quotes in most cases

  • Python will change to double quotes to avoid escapes

  • print() function output, don't have quotes

Returning str. Python will change to single quotes in most cases:

>>> 
... """
... >>> echo('hello')
... 'hello'
...
... # Python will change to single quotes in most cases
... >>> echo("hello")
... 'hello'
...
... Following test will fail
... >>> echo('hello')
... "hello"
...
... Python will change to double quotes to avoid escapes
... >>> echo('It\\'s Twardowski\\'s Moon')
... "It's Twardowski's Moon"
... """
>>>
>>> def echo(data):
...     return data

There are no quotes in print() function output:

>>> 
... """
... >>> echo('hello')
... hello
... """
>>>
>>> def echo(data):
...     print(data)

Testing print(str) with newlines:

>>> 
... """
... >>> echo('hello')
... hello
... hello
... hello
... <BLANKLINE>
... """
>>>
>>> def echo(data):
...     print(f'{data}\n' * 3)

16.1.7. Test Ordered Sequence

>>> 
... """
... >>> echo([1,2,3])
... [1, 2, 3]
...
... >>> echo((1,2,3))
... (1, 2, 3)
... """
>>>
>>> def echo(data):
...     return data

>>> 
... """
... >>> echo([1,2,3])
... [1, 2, 3]
...
... >>> echo((1,2,3))
... [1, 2, 3]
... """
>>>
>>> def echo(data):
...     return [x for x in data]

>>> 
... """
... >>> echo([1,2,3])
... [274.15, 275.15, 276.15]
...
... >>> echo((1,2,3))
... (274.15, 275.15, 276.15)
... """
>>>
>>> def echo(data):
...     cls = type(data)
...     return cls(x+273.15 for x in data)

16.1.8. Test Unordered Sequence

Hash from numbers are constant:

>>> 
... """
... >>> echo({1})
... {1}
... >>> echo({1,2})
... {1, 2}
... """
>>>
>>> def echo(data):
...     return data

However hash from str elements changes at every run:

>>> 
... """
... >>> echo({'a', 'b'})
... {'b', 'a'}
... """
>>>
>>> def echo(data):
...     return data

Therefore you should test if element is in the result, rather than comparing output:

>>> 
... """
... >>> result = echo({'a', 'b'})
... >>> 'a' in result
... True
... >>> 'b' in result
... True
... """
>>>
>>> def echo(data):
...     return data

16.1.9. Test Mapping

>>> 
... """
... >>> result = echo({'a': 1, 'b': 2})
... >>> result
... {'a': 1, 'b': 2}
... >>> 'a' in result.keys()
... True
... >>> 1 in result.values()
... True
... >>> ('a', 1) in result.items()
... True
... >>> result['a']
... 1
... """
>>>
>>> def echo(data):
...     return data

16.1.10. Test Nested

>>> 
... """
... >>> DATA = [
... ...     ('sepal_length', 'sepal_width', 'petal_length', 'petal_width', 'species'),
... ...     (5.8, 2.7, 5.1, 1.9, 'virginica'),
... ...     (5.1, 3.5, 1.4, 0.2, 'setosa'),
... ...     (5.7, 2.8, 4.1, 1.3, 'versicolor'),
... ...     (6.3, 2.9, 5.6, 1.8, 'virginica'),
... ...     (6.4, 3.2, 4.5, 1.5, 'versicolor'),
... ...     (4.7, 3.2, 1.3, 0.2, 'setosa')]
...
... >>> echo(DATA)
... [('sepal_length', 'sepal_width', 'petal_length', 'petal_width', 'species'), (5.8, 2.7, 5.1, 1.9, 'virginica'), (5.1, 3.5, 1.4, 0.2, 'setosa'), (5.7, 2.8, 4.1, 1.3, 'versicolor'), (6.3, 2.9, 5.6, 1.8, 'virginica'), (6.4, 3.2, 4.5, 1.5, 'versicolor'), (4.7, 3.2, 1.3, 0.2, 'setosa')]
...
... >>> echo(DATA)  
... [('sepal_length', 'sepal_width', 'petal_length', 'petal_width', 'species'),
...  (5.8, 2.7, 5.1, 1.9, 'virginica'),
...  (5.1, 3.5, 1.4, 0.2, 'setosa'),
...  (5.7, 2.8, 4.1, 1.3, 'versicolor'),
...  (6.3, 2.9, 5.6, 1.8, 'virginica'),
...  (6.4, 3.2, 4.5, 1.5, 'versicolor'),
...  (4.7, 3.2, 1.3, 0.2, 'setosa')]
... """
>>>
>>> def echo(data):
...     return data

16.1.11. Test Exceptions

>>> 
... """
... >>> echo()
... Traceback (most recent call last):
... NotImplementedError
... """
>>>
>>> def echo():
...     raise NotImplementedError

>>> 
... """
... >>> echo()
... Traceback (most recent call last):
... NotImplementedError: This will work in future
... """
>>>
>>> def echo():
...     raise NotImplementedError('This will work in future')

16.1.12. Test Type

>>> 
... """
... >>> result = echo(1)
... >>> type(result)
... <class 'int'>
...
... >>> result = echo(1.1)
... >>> type(result)
... <class 'float'>
...
... >>> result = echo(True)
... >>> type(result)
... <class 'bool'>
...
... >>> result = echo([1, 2])
... >>> type(result)
... <class 'list'>
...
... >>> result = echo([1, 2])
... >>> any(type(x) is int
... ...     for x in result)
... True
... """
>>>
>>> def echo(data):
...     return data

The following doctest will fail:

>>> 
... """
... >>> add_numbers('one', 'two')
... Traceback (most recent call last):
... TypeError: not a number
...
... >>> add_numbers(True, 1)
... Traceback (most recent call last):
... ValueError: not a number
... """
>>>
>>> def add_numbers(a, b):
...     if not isinstance(a, (int, float)):
...         raise ValueError('c')
...     if not isinstance(b, (int, float)):
...         raise ValueError('not a number')
...     return a + b

Expected exception, got 2.0:

Expected:

Traceback (most recent call last): ValueError: not a number

Got:

2.0

This test will pass:

>>> 
... """
... >>> add_numbers('one', 'two')
... Traceback (most recent call last):
... TypeError: not a number
...
... >>> add_numbers(True, 1)
... Traceback (most recent call last):
... ValueError: not a number
... """
>>>
>>> def add_numbers(a, b):
...     if type(a) not in (int, float):
...         raise ValueError('not a number')
...     if type(b) not in (int, float):
...         raise ValueError('not a number')
...     return a + b

16.1.13. Test Python Expressions

Using python statements in doctest:

>>> def echo(text):
...     """
...     >>> name = 'Mark Watney'
...     >>> print(name)
...     Mark Watney
...     """
...     return text

>>> def when(date):
...     """
...     >>> from datetime import datetime, timezone
...     >>> moon = datetime(1969, 7, 21, 17, 54, tzinfo=timezone.utc)
...     >>> when(moon)
...     1969-07-21 17:54 UTC
...     """
...     print(f'{date:%Y-%m-%d %H:%M %Z}')

16.1.14. Flags

  • DONT_ACCEPT_TRUE_FOR_1

  • DONT_ACCEPT_BLANKLINE

  • NORMALIZE_WHITESPACE

  • ELLIPSIS

  • IGNORE_EXCEPTION_DETAIL

  • SKIP

  • COMPARISON_FLAGS

  • REPORT_UDIFF

  • REPORT_CDIFF

  • REPORT_NDIFF

  • REPORT_ONLY_FIRST_FAILURE

  • FAIL_FAST

  • REPORTING_FLAGS

16.1.15. Case Study

Docstring used for doctest:

>>> def apollo_dsky(noun, verb):
...     """
...     This is the Apollo Display Keyboard
...     It takes noun and verb
...
...     >>> apollo_dsky(6, 61)
...     Program selected. Noun: 06, verb: 61
...
...     >>> apollo_dsky(16, 68)
...     Program selected. Noun: 16, verb: 68
...     """
...     print(f'Program selected. Noun: {noun:02}, verb: {verb:02}')

Celsius to Kelvin conversion:

>>> def celsius_to_kelvin(data):
...     """
...     >>> celsius_to_kelvin([1,2,3])
...     [274.15, 275.15, 276.15]
...
...     >>> celsius_to_kelvin((1,2,3))
...     [274.15, 275.15, 276.15]
...     """
...     return [x+273.15 for x in data]

>>> def celsius_to_kelvin(data):
...     """
...     >>> celsius_to_kelvin([1,2,3])
...     [274.15, 275.15, 276.15]
...
...     >>> celsius_to_kelvin((1,2,3))
...     (274.15, 275.15, 276.15)
...     """
...     cls = type(data)
...     return cls(x+273.15 for x in data)

Adding two numbers:

>>> def add_numbers(a, b):
...     """
...     >>> add_numbers(1, 2)
...     3.0
...     >>> add_numbers(-1, 1)
...     0.0
...     >>> add_numbers(0.1, 0.2)  
...     0.3000...
...     >>> add_numbers(1.5, 2.5)
...     4.0
...     >>> add_numbers(1, 1.5)
...     2.5
...     >>> add_numbers([1, 2], 3)
...     Traceback (most recent call last):
...     ValueError: not a number
...
...     >>> add_numbers(0, [1, 2])
...     Traceback (most recent call last):
...     ValueError: not a number
...
...     >>> add_numbers('one', 'two')
...     Traceback (most recent call last):
...     ValueError: not a number
...
...     >>> add_numbers(True, 1)
...     Traceback (most recent call last):
...     ValueError: not a number
...     """
...     if type(a) not in (int, float):
...         raise ValueError('not a number')
...
...     if type(b) not in (int, float):
...         raise ValueError('not a number')
...
...     return float(a + b)

Celsius to Kelvin temperature conversion:

>>> def celsius_to_kelvin(celsius):
...     """
...     >>> celsius_to_kelvin(0)
...     273.15
...
...     >>> celsius_to_kelvin(1)
...     274.15
...
...     >>> celsius_to_kelvin(-1)
...     272.15
...
...     >>> celsius_to_kelvin(-273.15)
...     0.0
...
...     >>> celsius_to_kelvin(-273.16)
...     Traceback (most recent call last):
...     ValueError: Negative Kelvin
...
...     >>> celsius_to_kelvin(-300)
...     Traceback (most recent call last):
...     ValueError: Negative Kelvin
...
...     >>> celsius_to_kelvin(True)
...     Traceback (most recent call last):
...     TypeError: Argument must be: int, float or Sequence[int, float]
...
...     >>> celsius_to_kelvin([0, 1, 2, 3])
...     [273.15, 274.15, 275.15, 276.15]
...
...     >>> celsius_to_kelvin({0, 1, 2, 3})
...     {273.15, 274.15, 275.15, 276.15}
...
...     >>> celsius_to_kelvin([0, 1, 2, -300])
...     Traceback (most recent call last):
...     ValueError: Negative Kelvin
...
...     >>> celsius_to_kelvin([0, 1, [2, 3], 3])
...     [273.15, 274.15, [275.15, 276.15], 276.15]
...     """
...     datatype = type(celsius)
...
...     if type(celsius) in {list, tuple, set, frozenset}:
...         return datatype(celsius_to_kelvin(x) for x in celsius)
...
...     if datatype not in {int, float}:
...         raise TypeError('Argument must be: int, float or Sequence[int, float]')
...
...     kelvin = celsius + 273.15
...
...     if kelvin < 0.0:
...         raise ValueError('Negative Kelvin')
...
...     return float(kelvin)

16.1.16. Assignments

Code 16.1. Solution
"""
* Assignment: Test Doctest Distance
* Required: no
* Complexity: easy
* Lines of code: 21 lines
* Time: 13 min

English:
    1. Write doctests to a functions which convert distance given in kilometers to meters
    2. Valid arguments:
        a. `int`
        b. `float`
    3. Invalid argumentm, raise exception `TypeError`:
        a. `str`
        b. `list[int]`
        c. `list[float]`
        d. `bool`
        e. any other type
    4. Returned distance must be float
    5. Returned distance cannot be negative
    6. Run doctests - all must succeed

Polish:
    1. Napisz doctesty do funkcji, która przeliczy dystans podany w kilometrach na metry
    2. Poprawne argumenty:
        a. `int`
        b. `float`
    3. Niepoprawne argumenty, podnieś wyjątek `TypeError`:
        a. `str`
        b. `list[int]`
        c. `list[float]`
        d. `bool`
        e. any other type
    4. Zwracany dystans musi być float
    5. Zwracany dystans nie może być ujemny
    6. Uruchom doctesty - wszystkie muszą się powieść

Hint:
    * 1 km = 1000 m

Tests:
    >>> import sys; sys.tracebacklimit = 0
"""

def km_to_meters(kilometers):
    if type(kilometers) not in {int, float}:
        raise TypeError('Invalid argument type')
    if kilometers < 0:
        raise ValueError('Argument must be not negative')
    return float(kilometers * 1000)
Code 16.2. Solution
"""
* Assignment: Test Doctest Temperature
* Required: no
* Complexity: easy
* Lines of code: 5 lines
* Time: 13 min

English:
    1. Write doctests to `celsius_to_kelvin` function
    2. Parameter `degrees` can be:
        a. int
        b. float
        c. list[int|float]
        d. tuple[int|float,...]
        e. set[int|float]
        f. In other case raise an exception: TypeError
           with message: "Invalid argument type"
    3. Run doctests - all must succeed

Polish:
    1. Napisz doctesty do funkcji `celsius_to_kelvin`
    2. Parametr `degrees` może być:
        a. int
        b. float
        c. list[int|float]
        d. tuple[int|float,...]
        e. set[int|float]
        f. W przeciwnym wypadku podnieś wyjątek: TypeError
           z komunikatem: "Invalid argument type"
    3. Uruchom doctesty - wszystkie muszą się powieść

Tests:
    >>> import sys; sys.tracebacklimit = 0
    >>> from inspect import isfunction
"""

def celsius_to_kelvin(degrees):
    if type(degrees) in (int, float):
        return 273.15 + degrees
    if type(degrees) in (list, tuple, set):
        cls = type(degrees)
        return cls(x+273.15 for x in degrees)
    raise TypeError('Invalid argument type')