12. Static Code Analysis

12.1. SonarLint

12.2. SonarScanner

12.3. SonarQube

SonarQube software (previously called Sonar) is an open source quality management platform, dedicated to continuously analyze and measure technical quality, from project portfolio to method.

More information

12.4. Przygotowanie środowiska statycznej analizy

Uruchomienie:

cd PROJECT_DIRECTORY
docker run --rm -d --name sonarqube -p 9000:9000 -v $(pwd):/src sonarqube
docker exec -u 0 -it sonarqube bash

    curl -sL https://deb.nodesource.com/setup_8.x -o /opt/node.sh
    bash /opt/node.sh
    apt install -y nodejs
    wget https://binaries.sonarsource.com/Distribution/sonar-scanner-cli/sonar-scanner-cli-3.3.0.1492.zip -O /opt/sonar-scanner.zip
    unzip -d /opt/ /opt/sonar-scanner.zip
    ln -s /opt/sonar-scanner-*/bin/sonar-scanner /usr/bin/sonar-scanner
    VERSION=$(cd /src/ && hg log -l 1 --template '{node}\n')

    apt install -y python-pip pylint python-coverage python-nose
    pip install -r /src/requirements.txt

Konfiguracja:

  1. Quality Profile -> Python

  2. Skopiuj profil "Sonar way" i nazwij nowy jako "PyLint"

  3. Trybik (prawy górny róg) -> Activate more rules

  4. Przefiltruj listę (lewy dolny róg) po "Repository" równym "PyLint"

  5. Bulk Change (góra ekrany) -> Activate in "PyLint" -> zaakceptuj

  6. Ustaw "PyLint jako domyślny"

  7. Uruchom analizę

Warning

Po uruchomieniu SonarQube z obrazu Docker instalacja pluginów, a następnie restart SonarQube niszczy możliwość przeprowadzania analizy

  1. Administration -> Marketplace

    • Zainstalować plugin HTML

12.4.1. Python

Listing 380. SonarScanner config for static analysis of Python code
## Sonar Server
sonar.host.url=http://localhost:9000/
sonar.login=admin
sonar.password=admin

## About Project
sonar.projectKey=python
sonar.projectName=Python
sonar.projectDescription=
sonar.links.homepage=
sonar.links.scm=
sonar.links.issue=
sonar.links.ci=

## Analysis
sonar.language=py
sonar.projectBaseDir=/src/
sonar.sources=.
sonar.sourceEncoding=UTF-8
#sonar.scm.provider=hg

## Output
sonar.verbose=false
sonar.log.level=INFO
sonar.showProfiling=false
#sonar.scanner.dumpToFile=/tmp/sonar-python.properties

## Python
sonar.inclusions=**/*.py
sonar.exclusions=**/migrations/**,**/*.pyc,**/__pycache__/**
sonar.python.pylint=/usr/bin/pylint
sonar.python.pylint_config=.pylintrc
sonar.python.xunit.skipDetails=false
sonar.python.xunit.reportPath=xunit.xml
sonar.python.coverage.reportPath=coverage.xml
sonar.core.codeCoveragePlugin=cobertura

## Turn off these rules
## python:s100: "Method names should comply with a naming convention"
## gives many false positives when overriding
## TestCase methods (such as setUp and tearDown) in test files
sonar.issue.ignore.multicriteria=e1,e2
sonar.issue.ignore.multicriteria.e1.ruleKey=python:S100
sonar.issue.ignore.multicriteria.e1.resourceKey=**/tests.py
sonar.issue.ignore.multicriteria.e2.ruleKey=python:S100
sonar.issue.ignore.multicriteria.e2.resourceKey=**/tests.py

12.4.2. CSS

Listing 381. SonarScanner config for static analysis of CSS code
## Sonar Server
sonar.host.url=http://localhost:9000/
sonar.login=admin
sonar.password=admin

## About Project
sonar.projectKey=css
sonar.projectName=CSS
sonar.projectDescription=
sonar.links.homepage=
sonar.links.scm=
sonar.links.issue=
sonar.links.ci=

## Analysis
sonar.language=css
sonar.projectBaseDir=/src/
sonar.sources=.
sonar.sourceEncoding=UTF-8
sonar.scm.provider=git

## Output
sonar.verbose=false
sonar.log.level=INFO
sonar.showProfiling=false
#sonar.scanner.dumpToFile=/tmp/sonar-css.properties

## CSS
sonar.inclusions=**/*.css,**/*.less,**/*.scss
sonar.exclusions=**/tinymce.**,**/jquery.*
sonar.css.node=/usr/bin/node
sonar.css.file.suffixes=.css,.less,.scss

12.4.3. JavaScript

Listing 382. SonarScanner config for static analysis of JavaScript code
## Sonar Server
sonar.host.url=http://localhost:9000/
sonar.login=admin
sonar.password=admin

## About Project
sonar.projectKey=javascript
sonar.projectName=JavaScript
sonar.projectDescription=
sonar.links.homepage=
sonar.links.scm=
sonar.links.issue=
sonar.links.ci=

## Analysis
sonar.language=js
sonar.projectBaseDir=/src/
sonar.sources=.
sonar.sourceEncoding=UTF-8
sonar.scm.provider=git

## Output
sonar.verbose=false
sonar.log.level=INFO
sonar.showProfiling=false
#sonar.scanner.dumpToFile=/tmp/sonar-javascript.properties

## JavaScript
sonar.inclusions=**/*.js,**/*.jsx,**/*.vue
sonar.exclusions=**/tinymce.**,**/jquery.*
sonar.javascript.jQueryObjectAliases=$,jQuery
sonar.javascript.environments=amd,applescript,atomtest,browser,commonjs,couch,embertest,greasemonkey,jasmine,jest,jquery,meteor,mocha,mongo,nashorn,node,phantomjs,prototypejs,protractor,qunit,rhino,serviceworker,shared-node-browser,shelljs,webextensions,worker,wsh,yui
sonar.javascript.globals=angular,goog,google,OpenLayers,d3,dojo,dojox,dijit,Backbone,moment,casper
sonar.javascript.exclusions=**/node_modules/**,**/bower_components/**
sonar.nodejs.executable=/usr/bin/node

12.4.4. Multi-language

Listing 383. SonarScanner config for static analysis of Multilanguage code
## Sonar Server
sonar.host.url=http://localhost:9000/
sonar.login=admin
sonar.password=admin

## About Project
sonar.projectKey=multilanguage
sonar.projectName=Multi-language
sonar.projectDescription=
sonar.links.homepage=
sonar.links.scm=
sonar.links.issue=
sonar.links.ci=

## Analysis
#sonar.language=
sonar.projectBaseDir=/src/
sonar.sources=.
sonar.sourceEncoding=UTF-8
sonar.scm.provider=git

## Output
sonar.verbose=false
sonar.log.level=INFO
sonar.showProfiling=false
#sonar.scanner.dumpToFile=/tmp/sonar-multilanguage.properties

sonar.inclusions=**/*.css,**/*.less,**/*.scss,**/*.html,**/*.xhtml,**/*.jspf,**/*.jspx,**/*.cshtml,**/*.vbhtml,**/*.aspx,**/*.ascx,**/*.rhtml,**/*.erb,**/*.shtm,**/*.shtml,**/*.js,**/*.jsx,**/*.vue,**/*.py
sonar.exclusions=**/tinymce.**,**/jquery.*,**/sitemap.xml,**/migrations/**,**/*.pyc,**/__pycache__/**

## CSS
sonar.css.node=/usr/bin/node
sonar.css.file.suffixes=.css,.less,.scss

## JavaScript
sonar.javascript.jQueryObjectAliases=$,jQuery
sonar.javascript.environments=amd,applescript,atomtest,browser,commonjs,couch,embertest,greasemonkey,jasmine,jest,jquery,meteor,mocha,mongo,nashorn,node,phantomjs,prototypejs,protractor,qunit,rhino,serviceworker,shared-node-browser,shelljs,webextensions,worker,wsh,yui
sonar.javascript.globals=angular,goog,google,OpenLayers,d3,dojo,dojox,dijit,Backbone,moment,casper
sonar.javascript.exclusions=**/node_modules/**,**/bower_components/**
sonar.nodejs.executable=/usr/bin/node

## Python
sonar.python.pylint=/usr/bin/pylint
sonar.python.pylint_config=.pylintrc
sonar.python.xunit.skipDetails=false
sonar.python.xunit.reportPath=xunit.xml
sonar.python.coverage.reportPath=coverage.xml
sonar.core.codeCoveragePlugin=cobertura

## Turn off these rules
## python:s100: "Method names should comply with a naming convention"
## gives many false positives when overriding
## TestCase methods (such as setUp and tearDown) in test files
sonar.issue.ignore.multicriteria=e1,e2
sonar.issue.ignore.multicriteria.e1.ruleKey=python:S100
sonar.issue.ignore.multicriteria.e1.resourceKey=**/tests.py
sonar.issue.ignore.multicriteria.e2.ruleKey=python:S100
sonar.issue.ignore.multicriteria.e2.resourceKey=**/tests.py

12.4.5. Coverage

About

Coverage.py measures code coverage, typically during test execution. It uses the code analysis tools and tracing hooks provided in the Python standard library to determine which lines are executable, and which have been executed.

Install
$ pip install coverage
Usage
$ coverage run FILE.py
$ coverage report -m
$ nosetests --with-coverage --cover-erase --cover-xml --cover-inclusive --with-xunit --xunit-file=xunit.xml --cover-xml-file=coverage.xml

Use coverage run to run your program and gather data:

$ coverage run my_program.py arg1 arg2
blah blah ..your program's output.. blah blah

Use coverage report to report on the results:

$ coverage report -m
Name                      Stmts   Miss  Cover   Missing
-------------------------------------------------------
my_program.py                20      4    80%   33-35, 39
my_other_module.py           56      6    89%   17-23
-------------------------------------------------------
TOTAL                        76     10    87%

For a nicer presentation, use coverage html to get annotated HTML listings detailing missed lines:

$ coverage html
More information

12.4.6. pylama

pylama --linters pylint --skip='*/migrations/*' --abspath /src
Listing 384. setup.cfg
[pylama]
format = pylint
skip = */.tox/*,*/.env/*
linters = pylint,mccabe,eradicate,isort,pyflakes,pydocstyle,pycodestyle
ignore = F0401,C0111,E731

[pylama:pycodestyle]
max_line_length = 300

[pylama:pyflakes]
builtins = _

[pylama:pylint]
max_line_length = 100
disable = R

[pylama:*/pylama/main.py]
ignore = C901,R0914,W0212
select = R

[pylama:*/tests.py]
ignore = C0110

[pylama:*/setup.py]
skip = 1

12.4.7. bandit

  • Sprawdzanie kodu pod kątem podatności bezpieczeństwa

pip install bandit
bandit --recursive /src/

12.4.8. pycodestyle

pip install pycodestyle
pycodestyle --max-line-length=79 --exclude=*/migrations/* .

12.4.9. safety

pip install safety
safety check -r /src/requirements.txt