1. Software Engineering Conventions

1.1. PEP 8 - Style Guide for Python Code

1.1.1. Tabs or spaces?

  • 4 spacje

  • IDE zamienia tab na 4 spacje

1.1.2. Line length

  • najbardziej kontrowersyjna klauzula

  • 79 znaków kod

  • 72 znaki docstrings/comments

  • Python standard library is conservative and requires limiting lines to 79 characters (and docstrings/comments to 72)

  • soft wrap

  • co z monitorami 4k?

  • Preferred way of wrapping long lines is by using Python's implied line continuation inside parentheses, brackets and braces.

class FoodProduct(models.Model):
    vitamins_folic_acid = models.DecimalField(verbose_name=_('Folic Acid'), help_text=_('µg/100g'), decimal_places=2, max_digits=5, blank=True, null=True, default=None)
    vitamins_a = models.DecimalField(verbose_name=_('Vitamin A'), help_text=_('µg/100g'), decimal_places=2, max_digits=5, blank=True, null=True, default=None)
    vitamins_b1 = models.DecimalField(verbose_name=_('Vitamin B1'), help_text=_('mg/100g'), decimal_places=2, max_digits=5, blank=True, null=True, default=None)
    vitamins_b2 = models.DecimalField(verbose_name=_('Vitamin B2'), help_text=_('mg/100g'), decimal_places=2, max_digits=5, blank=True, null=True, default=None)
    vitamins_b6 = models.DecimalField(verbose_name=_('Vitamin B6'), help_text=_('mg/100g'), decimal_places=2, max_digits=5, blank=True, null=True, default=None)
    vitamins_b12 = models.DecimalField(verbose_name=_('Vitamin B12'), help_text=_('µg/100g'), decimal_places=2, max_digits=5, blank=True, null=True, default=None)
    vitamins_c = models.DecimalField(verbose_name=_('Vitamin C'), help_text=_('mg/100g'), decimal_places=2, max_digits=5, blank=True, null=True, default=None)
    vitamins_d = models.DecimalField(verbose_name=_('Vitamin D'), help_text=_('µg/100g'), decimal_places=2, max_digits=5, blank=True, null=True, default=None)
    vitamins_e = models.DecimalField(verbose_name=_('Vitamin E'), help_text=_('mg/100g'), decimal_places=2, max_digits=5, blank=True, null=True, default=None)
    vitamins_pp = models.DecimalField(verbose_name=_('Vitamin PP'), help_text=_('mg/100g'), decimal_places=2, max_digits=5, blank=True, null=True, default=None)

    minerals_zinc = models.DecimalField(verbose_name=_('Zinc'), help_text=_('mg/100g'), decimal_places=2, max_digits=5, blank=True, null=True, default=None)
    minerals_phosphorus = models.DecimalField(verbose_name=_('Phosphorus'), help_text=_('mg/100g'), decimal_places=2, max_digits=5, blank=True, null=True, default=None)
    minerals_iodine = models.DecimalField(verbose_name=_('Iodine'), help_text=_('µg/100g'), decimal_places=2, max_digits=5, blank=True, null=True, default=None)
    minerals_magnesium = models.DecimalField(verbose_name=_('Magnesium'), help_text=_('mg/100g'), decimal_places=2, max_digits=5, blank=True, null=True, default=None)
    minerals_copper = models.DecimalField(verbose_name=_('Copper'), help_text=_('mg/100g'), decimal_places=2, max_digits=5, blank=True, null=True, default=None)
    minerals_potasium = models.DecimalField(verbose_name=_('Potasium'), help_text=_('mg/100g'), decimal_places=2, max_digits=5, blank=True, null=True, default=None)
    minerals_selenium = models.DecimalField(verbose_name=_('Selenium'), help_text=_('µg/100g'), decimal_places=2, max_digits=5, blank=True, null=True, default=None)
    minerals_sodium = models.DecimalField(verbose_name=_('Sodium'), help_text=_('mg/100g'), decimal_places=2, max_digits=5, blank=True, null=True, default=None)
    minerals_calcium = models.DecimalField(verbose_name=_('Calcium'), help_text=_('mg/100g'), decimal_places=2, max_digits=5, blank=True, null=True, default=None)
    minerals_iron = models.DecimalField(verbose_name=_('Iron'), help_text=_('mg/100g'), decimal_places=2, max_digits=5, blank=True, null=True, default=None)

1.1.3. File encoding

  • UTF-8

  • always remember to open files for reading and writing with encoding='utf-8'

  • All identifiers in the Python standard library MUST use ASCII-only identifiers, and SHOULD use English words wherever feasible (in many cases, abbreviations and technical terms are used which aren't English).

  • String literals and comments must also be in ASCII.

  • Authors whose names are not based on the Latin alphabet (latin-1, ISO/IEC 8859-1 character set) MUST provide a transliteration of their names in this character set.

1.1.4. Comments

  • Comments that contradict the code are worse than no comments.

  • Comments should be complete sentences.

  • Block comments generally consist of one or more paragraphs built out of complete sentences

  • Each sentence ending in a period.

  • Python coders from non-English speaking countries: please write your comments in English, unless you are 120% sure that the code will never be read by people who don't speak your language.

  • Each line of a block comment starts with a # and a single space (unless it is indented text inside the comment).

1.1.5. Documentation Strings

  • PEP 257

  • Write docstrings for all public modules, functions, classes, and methods.

  • Docstrings are not necessary for non-public methods, but you should have a comment that describes what the method does.

  • For one liner docstrings, please keep the closing """ on the same line.

1.1.6. Use better names, rather than comments

def cal_var(results):
    """Calculate variance"""
    return sum((Xi-m) ** 2 for Xi in results) / len(results)

def calculate_variance(results):
    return sum((Xi-m) ** 2 for Xi in results) / len(results)
def fabs(a, b):
    return float(abs(a + b))

def float_absolute_value(a, b):
    return float(abs(a + b))

def abs(a: int, b: int) -> float:
    return float(abs(a + b))

def absolute_value(a: int, b: int) -> float:
    return float(abs(a + b))

1.1.7. Commented code?

  • NO!

  • Never commit files with commented code

1.1.8. Author name or revision version

  • Do not put author name or revision version to the files

  • Version Control System is responsible for that

1.1.9. Naming convention

1.1.9.1. Constants and Variables

  • Używanie _ w nazwach (snake_case) - // Python - snake ;)

  • variable or variable_name

    name = 'José Jiménez'
    
    first_name = 'José'
    last_name = 'Jiménez'
    
  • CONSTANT or CONSTANT_NAME

    PATH = '/etc/hosts'
    
    FILE_NAME = 'README.txt'
    

1.1.9.2. Classes

  • CamelCase

    class MyClass:
        pass
    

1.1.10. Class Attributes

  • Public attributes should have no leading underscores.

  • If your public attribute name collides with a reserved keyword, append a single trailing underscore to your attribute name.

  • cls is the preferred spelling for any variable or argument which is known to be a class, especially the first argument to a class method.

1.1.11. Methods/Functions

  • Używanie _ w nazwach (snake_case) - // Python - snake ;)

  • method_name() or function_name()

    def add_numbers(a, b):
        return a + b
    
  • Nie robimy camelCase

    def addNumbers(a, b):
        return a + b
    

1.1.12. Modules names

  • nazwymodulow

  • nazwy_modulow

  • Preferable one word

    import random
    import argparse
    

1.1.13. Function/Method argument names

  • self

    class Astronaut:
        name = 'José Jiménez'
    
        def say_hello(self):
            print(f'My name... {self.name}')
    
  • cls

    class Astronaut:
        pass
    
    class Cosmonaut:
        pass
    
    class Starman:
        pass
    
    def is_spaceman(cls):
        if instance(cls, (Astronaut, Cosmonaut)):
            return True
        else:
            return False
    
    
    is_spaceman(Cosmonaut)  # True
    is_spaceman(Astronaut)  # True
    is_spaceman(Starman)    # False
    
  • self and other

    class Vector:
        x = 0
        y = 1
    
        def __add__(self, other):
            return Vector(
                x=self.x+other.x,
                y=self.y+other.y
            )
    

1.1.14. Using __ and _ in names

  • W Pythonie nie ma private/protected/public

  • Funkcje o nazwie zaczynającej się od _ przez konwencję są traktowane jako prywatne

    from random import _ceil
    
    _ceil()
    # good IDE will display information, that you're accessing protected member
    
  • Funkcje i zmienne o nazwie zaczynającej się od __ i kończących się na __ przez konwencję są traktowane jako systemowe

    print(__file__)
    
  • _ at the end of name when name collision

    def print_(text1, text2):
        print(values, sep=';', end='\n')
    

1.1.15. Single or double quotes?

  • Python nie rozróżnia czy stosujemy pojedyncze znaki cudzysłowiu czy podwójne.

  • Ważne jest aby wybrać jedną konwencję i się jej konsekwentnie trzymać.

  • Interpreter Pythona domyślnie stosuje pojedyncze znaki cudzysłowia.

  • Z tego powodu w tej książce będziemy trzymać się powyższej konwencji.

  • Ma to znaczenie przy doctest, który zawsze korzysta z pojedynczych i rzuca errorem jak są podwójne

  • For triple-quoted strings, always use double quote characters to be consistent with the docstring convention in PEP 257

print('it\'s José\'s book')
print("it's José's book")
print('<a href="http://python.astrotech.io">Python and Machine Learning</a>')

1.1.16. Trailing Commas

Listing 262. Yes
FILES = ('setup.cfg',)
Listing 263. OK, but confusing
FILES = 'setup.cfg',

1.1.17. Indents

Listing 264. Good
# More indentation included to distinguish this from the rest.
def server(
        host='localhost', port=443, secure=True,
        username='admin', password='admin'):
    return locals()


# Aligned with opening delimiter.
connection = server(host='localhost', port=443, secure=True,
                    username='admin', password='admin')

# Hanging indents should add a level.
connection = server(
    host='localhost', port=443, secure=True,
    username='admin', password='admin')

# The best
connection = server(
    host='localhost',
    username='admin',
    password='admin',
    port=443,
    secure=True,
)
Listing 265. Bad
# Further indentation required as indentation is not distinguishable.
def Connection(
    host='localhost', port=1337,
    username='admin', password='admin'):
    return host, port, username, password


# Arguments on first line forbidden when not using vertical alignment.
connection = Connection(host='localhost', port=1337,
    username='admin', password='admin')

1.1.18. Brackets

vector = [
    1, 2, 3,
    4, 5, 6,
]

result = some_function_that_takes_arguments(
    'a', 'b', 'c',
    'd', 'e', 'f',
)

vector = [
    1, 2, 3,
    4, 5, 6]

result = some_function_that_takes_arguments(
    'a', 'b', 'c',
    'd', 'e', 'f')
TYPE_CHOICES = [
    ('custom', _('Custom Made')),
    ('brand', _('Brand Product')),
    ('gourmet', _('Gourmet Food')),
    ('restaurant', _('Restaurant'))]

FORM_CHOICES = [
    ('solid', _('Solid')),
    ('liquid', _('Liquid'))]

CATEGORY_CHOICES = [
    ('other', _('Other')),
    ('fruits', _('Fruits')),
    ('vegetables', _('Vegetables')),
    ('meat', _('Meat'))]

1.1.19. Modules

  • Modules should explicitly declare the names in their public API using the __all__ attribute.

  • Setting __all__ to an empty list indicates that the module has no public API.

1.1.20. Line continuation

Linie możemy łamać poprzez stawianie znaku ukośnika \ na końcu:

with open('/path/to/some/file/you/want/to/read') as file1, \
        open('/path/to/some/file/being/written', mode='w') as file2:
    content = file1.read()
    file2.write(content)
Listing 266. Easy to match operators with operands
income = (gross_wages
          + taxable_interest
          + (dividends - qualified_dividends)
          - ira_deduction
          - student_loan_interest)
class Server:
    def __init__(self, username, password, host='localhost'
                 port=80, secure=False):

        if not instance(username, str) or not instance(password, str) or
                not instance(host, str) or not instance(secure, bool) or
                (not instance(port, int) and 0 < port <= 65535):
            raise TypeError(f'One of your parameters is incorrect type')

     def __str__(self):
        if secure:
            protocol = 'https'
        else:
            protocol = 'http'

        return f'{protocol}://{self.username}:{self.password}@{self.host}:{self.port}/'

server = Server(
    host='localhost',
    username='admin',
    password='admin',
    port=443,
    secure=True,
)

1.1.21. Blank lines

  • Surround top-level function and class definitions with two blank lines.

  • Method definitions inside a class are surrounded by a single blank line.

  • Extra blank lines may be used (sparingly) to separate groups of related functions.

  • Use blank lines in functions, sparingly, to indicate logical sections.

class Server:
    def __init__(self, username, password, host='localhost'
                 port=80, secure=False):

        if not instance(username, str):
            raise TypeError(f'Username must be str')

        if not instance(password, str):
            raise TypeError(f'Password must be str')

        if not instance(port, int):
            raise TypeError(f'Port must be int')
        elif: 0 < port <= 65535
            raise ValueError(f'Port must be 0-65535')

    def __str__(self):
        if secure:
            protocol = 'https'
        else:
            protocol = 'http'

        return f'{protocol}://{self.username}:{self.password}@{self.host}:{self.port}/'

1.1.22. Whitespace in function calls

spam(ham[1], {eggs: 2})        # Good
spam( ham[ 1 ], { eggs: 2 } )  # Bad
spam(1)     # Good
spam (1)    # Bad
do_one()    # Good
do_two()    # Good
do_three()  # Good

do_one(); do_two(); do_three()                  # Bad

do_one(); do_two(); do_three(long, argument,    # Bad
                             list, like, this)  # Bad

1.1.23. Whitespace in slices

ham[1:9]                          # Good
ham[1:9:3]                        # Good
ham[:9:3]                         # Good
ham[1::3]                         # Good
ham[1:9:]                         # Good

ham[1: 9]                         # Bad
ham[1 :9]                         # Bad
ham[1:9 :3]                       # Bad
ham[lower:upper]                  # Good
ham[lower:upper:]                 # Good
ham[lower::step]                  # Good

ham[lower : : upper]              # Bad
ham[lower+offset : upper+offset]  # Good
ham[: upper_fn(x) : step_fn(x)]   # Good
ham[:: step_fn(x)]                # Good

ham[lower + offset:upper + offset]    # Bad
ham[:upper]             # Good
ham[ : upper]           # Bad
ham[ :upper]            # Bad

1.1.24. Whitespace in assignments

x = 1                   # Good
y = 2                   # Good
long_variable = 3       # Good

x             = 1       # Bad
y             = 2       # Bad
long_variable = 3       # Bad
i = i + 1               # Good
i=i+1                   # Bad
submitted += 1          # Good
submitted +=1           # Bad

1.1.25. Whitespace in math operators

x = x*2 - 1             # Good
x = x * 2 - 1           # Bad
hypot2 = x*x + y*y      # Good
hypot2 = x * x + y * y  # Bad
c = (a+b) * (a-b)      # Good
c = (a + b) * (a - b)  # Bad

1.1.26. Whitespace in accessors

dct['key'] = lst[index]     # Good
dct ['key'] = lst[ index ]  # Bad

1.1.27. Whitespace in functions

Good
def complex(real, imag=0.0):
    return magic(r=real, i=imag)
Bad
def complex(real, imag = 0.0):
    return magic(r = real, i = imag)
Controversial
def move(self, left: int = 0, down: int = 0, up: int = 0, right: int = 0) -> None:
    self.set_position_coordinates(
        x=self.position.x + right - left,
        y=self.position.y + down - up
    )

1.1.28. Whitespace in conditionals

Good
if foo == 'blah':
    do_blah_thing()
Bad
if foo == 'blah': do_blah_thing()

if foo == 'blah': one(); two(); three()

if foo == 'blah': do_blah_thing()
else: do_non_blah_thing()

1.1.29. Whitespace in exceptions

Good
try:
    do_something()
except Exception:
    pass
Bad
try: something()
finally: cleanup()

1.1.30. Conditionals

Good
if greeting:
    pass
Bad
if greeting == True:
    pass

if greeting is True:
    pass

1.1.31. Negative Conditionals

Good
if name is not None:
    pass
Bad
# if (! name == null) {}
if not name is None:
    pass
usernames = {'José', 'Max', 'Иван'}

# if (! usernames.contains('José')) {}
if not 'José' in usernames:
    print('I do not know you')
else:
    print('Hello my friend')

1.1.32. Checking if not empty

Good
if sequence:
    pass

if not sequence:
    pass
Bad
if len(sequence):
    pass

if not len(sequence):
    pass

1.1.33. Explicit return

Good
def foo(x):
    if x >= 0:
        return math.sqrt(x)
    else:
        return None
Bad
def foo(x):
    if x >= 0:
        return math.sqrt(x)

1.1.34. Explicit return value

Good
def bar(x):
    if x < 0:
        return None
    return math.sqrt(x)
Bad
def bar(x):
    if x < 0:
        return
    return math.sqrt(x)

1.1.35. Imports

  • Każdy z importów powinien być w osobnej linii

  • importy systemowe na górze

  • importy bibliotek zewnętrznych poniżej systemowych

  • importy własnych modułów poniżej bibliotek zewnętrznych

  • jeżeli jest dużo importów, pomiędzy grupami powinna być linia przerwy

Good
import os
import sys
import requests
import numpy as np
from datetime import date
from datetime import time
from datetime import datetime
from datetime import timezone
from datetime import date, time, datetime, timezone
from datetime import date, time, datetime, timezone
import os
import sys
from random import shuffle
from subprocess import Popen, PIPE
import requests
import numpy as np
Bad
import sys, os, requests, numpy
import sys, os
import requests, numpy

1.1.36. Whitespace with type annotations

Good
def function(first: str):
    pass

def function(first: str = None):
    pass

def function() -> None:
    pass

def function(first: str, second: str = None, limit: int = 1000) -> int:
    pass
Bad
def function(first: str=None):
    pass

def function(first:str):
    pass

def function(first: str)->None:
    pass

1.1.37. Magic number i magic string

  • NO!

1.2. Static Code Analysis

Note

Topic is covered in details in chapters: CI/CD Tools and CI/CD Pipelines.

1.2.1. pycodestyle

  • Previously known as pep8

  • Python style guide checker.

  • pycodestyle is a tool to check your Python code against some of the style conventions in PEP 8

  • Plugin architecture: Adding new checks is easy

  • Parseable output: Jump to error location in your editor

  • Small: Just one Python file, requires only stdlib

  • Comes with a comprehensive test suite

1.2.1.1. Installation

pip install pycodestyle

1.2.1.2. Usage

pycodestyle FILE.py
pycodestyle DIRECTORY/*.py
pycodestyle DIRECTORY/
pycodestyle --statistics -qq DIRECTORY/
pycodestyle --show-source --show-pep8 FILE.py

1.2.1.3. Configuration

  • setup.cfg

[pycodestyle]
max-line-length = 120
ignore = E402,W391

1.3. Assignment

1.3.1. Cleanup your file

  • Lines of code to write: 2 lines

  • Estimated time of completion: 5 min

  1. Install pycodestyle

  2. Run pycodestyle on your last script

  3. Fix all errors

  4. Run pycodestyle on directory with all of your scripts

  5. Fix all errors

The whys and wherefores
  • Umiejętność czytania komunikatów

  • Umiejętność pracy z terminalem

  • Utrzymywanie konwencji PEP8