12. Testy i Jakość


Fig. 12.1. Development driven tests

12.1. Glossary

In object-oriented programming, mock objects are simulated objects that mimic the behavior of real objects in controlled ways. In a unit test, mock objects can simulate the behavior of complex, real objects and are therefore useful when a real object is impractical or impossible to incorporate into a unit test.
A method stub or simply stub in software development is a piece of code used to stand in for some other programming functionality. A stub may simulate the behavior of existing code (such as a procedure on a remote machine) or be a temporary substitute for yet-to-be-developed code. Stubs are therefore most useful in porting, distributed computing as well as general software development and testing.
In computer programming, unit testing is a software testing method by which individual units of source code, sets of one or more computer program modules together with associated control data, usage procedures, and operating procedures, are tested to determine whether they are fit for use.

12.2. Built-in frameworks

12.2.1. Doctest

Code Listing 12.1. Wykorzystanie doctest.testmod() do uruchamiania testów
def km_na_metry(ile):
    >>> km_na_metry(1)

    >>> km_na_metry(0)

    >>> km_na_metry(-1)
    Traceback (most recent call last):

    >>> km_na_metry('adas')
    Traceback (most recent call last):
    return ile * 1000

if __name__ == "__main__":
    import doctest

Code Listing 12.2. Pokrycie przypadków brzegowych doctestami

def przelicz_celsius_na_kelvin(temperatura):
    >>> przelicz_celsius_na_kelvin(1)

    >>> przelicz_celsius_na_kelvin(0)

    >>> przelicz_celsius_na_kelvin(-300)
    Traceback (most recent call last):
    ValueError: Nie może być mniejsze niż minimalna temperatura

    >>> przelicz_celsius_na_kelvin('jeden')
    Traceback (most recent call last):
    ValueError: Temperatura musi być float

    >>> przelicz_celsius_na_kelvin([1.0, 1, 0])
    Traceback (most recent call last):
    TypeError: Nie obsługiwany typ argumentu

        temperatura = float(temperatura)
    except ValueError:
        raise ValueError('Temperatura musi być float')
    except TypeError:
        raise TypeError('Nie obsługiwany typ argumentu')

    if temperatura < MINIMALNA_TEMPERATURA:
        raise ValueError('Nie może być mniejsze niż minimalna temperatura')
        return temperatura - MINIMALNA_TEMPERATURA

12.2.2. Unittest

Code Listing 12.3. Przykład pokrycia klasy za pomocą unittest.
import unittest

class Prostokat:
    def __init__(self, a, b):
        if not isinstance(a, (float, int)) or a <= 0:
            raise ValueError('Długość boku A musi być więszka od 0')

        if not isinstance(b, (float, int)) or b <= 0:
            raise ValueError('Długość boku B musi być więszka od 0')

        self.bok_a = float(a)
        self.bok_b = float(a)

    def pole(self):
        return self.bok_a * self.bok_b

    def obwod(self):
        return 2 * (self.bok_a + self.bok_b)

    def __str__(self):
        return f'Prostokat(a={self.bok_a}, b={self.bok_b})'

class ProstokatTest(unittest.TestCase):

    def setUp(self):
        self.prostokat = Prostokat(a=10, b=20)

    def test_prostokata_bok_nieprawidlowy(self):
        with self.assertRaises(ValueError):
            Prostokat(a='a', b=20)

        with self.assertRaises(ValueError):
            Prostokat(a=20, b='b')

    def test_prostokata_bok_zero(self):
        with self.assertRaises(ValueError):
            Prostokat(a=0, b=20)

        with self.assertRaises(ValueError):
            Prostokat(a=20, b=0)

    def test_prostokata_bok_ujemny(self):
        with self.assertRaises(ValueError):
            Prostokat(a=-3, b=20)

        with self.assertRaises(ValueError):
            Prostokat(a=20, b=-3)

    def test_ustawienia_bokow(self):
        with self.assertRaises(TypeError):

        with self.assertRaises(TypeError):

    def test_tworzenie_prostokata(self):
        self.assertEqual(self.prostokat.bok_a, 10)
        self.assertEqual(self.prostokat.bok_b, 20)

    def test_pole(self):
        self.assertEqual(self.prostokat.pole(), 200)

    def test_obwod(self):
        self.assertEqual(self.prostokat.obwod(), 60)

    def test_prostokat_to_string(self):
        self.assertEqual(str(self.prostokat), 'Prostokat(a=5.0, b=10.0)')

if __name__ == '__main__':
$ python -m unittest FILENAME.py

12.3. Frontend Testing

12.3.1. selenium

Selenium automates browsers. That’s it! What you do with that power is entirely up to you. Primarily, it is for automating web applications for testing purposes, but is certainly not limited to just that. Boring web-based administration tasks can (and should!) be automated as well.

Selenium has the support of some of the largest browser vendors who have taken (or are taking) steps to make Selenium a native part of their browser. It is also the core technology in countless other browser automation tools, APIs and frameworks.

Selenium 1.0 + WebDriver = Selenium 2.0

  • WebDriver is designed in a simpler and more concise programming interface along with addressing some limitations in the Selenium-RC API.
  • WebDriver is a compact Object Oriented API when compared to Selenium1.0
  • It drives the browser much more effectively and overcomes the limitations of Selenium 1.x which affected our functional test coverage, like the file upload or download, pop-ups and dialogs barrier
  • WebDriver overcomes the limitation of Selenium RC’s Single Host origin policy

WebDriver is the name of the key interface against which tests should be written in Java, the implementing classes one should use are listed as below:

  • ChromeDriver,
  • EventFiringWebDriver,
  • FirefoxDriver,
  • HtmlUnitDriver,
  • InternetExplorerDriver,
  • PhantomJSDriver,
  • RemoteWebDriver,
  • SafariDriver.

12.4. Static Code Analysis

12.4.1. pycodestyle previously known as PEP8


Python style guide checker. pycodestyle is a tool to check your Python code against some of the style conventions in PEP 8.

  • Plugin architecture: Adding new checks is easy.
  • Parseable output: Jump to error location in your editor.
  • Small: Just one Python file, requires only stdlib. You can use just the
  • pep8.py file for this purpose.
  • Comes with a comprehensive test suite.
$ pip install pycodestyle
$ pip install --upgrade pycodestyle
$ pip uninstall pycodestyle
$ pycodestyle FILENAME.py
$ pycodestyle DIRECTORY/
$ pycodestyle --statistics -qq DIRECTORY/
$ pycodestyle --show-source --show-pep8 FILENAME.py


max-line-length = 939
ignore = E402,W391

12.4.2. SonarQube


SonarQube software (previously called Sonar) is an open source quality management platform, dedicated to continuously analyze and measure technical quality, from project portfolio to method.

More information:

12.4.3. Pylint


Pylint is a Python source code analyzer which looks for programming errors, helps enforcing a coding standard and sniffs for some code smells (as defined in Martin Fowler’s Refactoring book). Pylint has many rules enabled by default, way too much to silence them all on a minimally sized program. It’s highly configurable and handle pragmas to control it from within your code. Additionally, it is possible to write plugins to add your own checks.

Coding Standard:

  • checking line-code’s length,
  • checking if variable names are well-formed according to your coding standard
  • checking if imported modules are used

Error detection:

  • checking if declared interfaces are truly implemented
  • checking if modules are imported and much more (see the complete check list)

Pylint is shipped with Pyreverse which creates UML diagrams for python code.

$ pip install pylint
$ pip install --upgrade pylint
$ pip uninstall pylint
$ pylint DIRECTORY/
$ pylint FILENAME.py
More information:

12.4.4. Pyflakes


A simple program which checks Python source files for errors. Pyflakes analyzes programs and detects various errors. It works by parsing the source file, not importing it, so it is safe to use on modules with side effects. It’s also much faster.

$ pip install pyflakes
$ pip install --upgrade pyflakes
$ pip uninstall pyflakes
$ pyflakes DIRECTORY/
$ python -m pyflakes DIRECTORY/
More information:

12.4.5. Coverage


Coverage.py measures code coverage, typically during test execution. It uses the code analysis tools and tracing hooks provided in the Python standard library to determine which lines are executable, and which have been executed.

$ pip install coverage
$ pip install --upgrade coverage
$ pip uninstall coverage
$ coverage run FILENAME.py
$ coverage report -m

Use coverage run to run your program and gather data:

$ coverage run my_program.py arg1 arg2
blah blah ..your program's output.. blah blah

Use coverage report to report on the results:

$ coverage report -m
Name                      Stmts   Miss  Cover   Missing
my_program.py                20      4    80%   33-35, 39
my_other_module.py           56      6    89%   17-23
TOTAL                        76     10    87%

For a nicer presentation, use coverage html to get annotated HTML listings detailing missed lines:

$ coverage html
More information:

12.5. Automation and Releases

12.5.1. Fabric

$ pip install fabric
$ pip install --upgrade fabric
$ pip uninstall fabric

12.7. Transifex