Code quality
There are two types of software quality - external and internal. External are the ones that are important to the users of the system. They may include:
- correctness - software behaves as users expect
- usability - how easy is it to use
- reliability - the ability to function under any circumstances
Internal quality characteristics are what developers care about:
- maintainability - how easy the software can be modified
- readability - how easy new developers can understand what code is doing by reading it
- testability - how easy the systems could be tested to verify that it satisfies the requirements
The internal characteristics relate closely with the quality of the code and design.
Does code quality matter?
When I’m working on my pet projects I’m not sure if someone else will see the code. Not that the code is a mess - it’s ok. But I’ve spent an hour putting it all together, so it’s maybe not my best one.
I work full time as a software developer. I write software that generates revenue for the company I work for. The owner may not care about code quality as long as users don’t complain (external software quality characteristics are met) and pay. But from CTO point of view code quality matters:
- our team productivity fill drop as we won’t be effective when working with smelly code if we don’t keep the plank high enough from the beginning
- onboarding for the new members of the team will take longer
- engineers will become dissatisfied with the job
- we won’t be able to implement new revenue-generating features on time
So there are situations when code quality doesn’t matter, and when it may cost a lot
Python linters
There are many methods to ensure quality: having a code style, doing code reviews, etc. And one of them is using automated tools such as linters. They analyze the code for known issues - for example, violations of PEP8 code style. They can be incorporated into CI/CD pipelines and stop deployments if report problems are reported (or the number of issues increased from the last run).
So there are several python linters available:
Flake8
It’s a wrapper around code style checker, error checker and complexity checker
- pycodestyle - style checker for PEP8 compliance. PEP8 is a set of rules for how to format your Python code to maximize its readability. It acts as a base for a company/project/team-specific code style. Error codes are E***/W***/N8**
- pyflakes - checks for errors or violation of common non-stylistic practices. For example, things like “module imported but unused”. See full list here
- McCabe complexity checker. Complexity is a quantitative measure of the number of linearly independent paths through a program’s source code. It was developed by Thomas J. McCabe, Sr. in 1976. So basically nested if statements inside for loops will blow up this number when the threshold for a good code is less than 10
Flake8 has a lot of configuration options, but its killer feature is an ecosystem of plugins. Take for example https://pypi.org/project/flake8-django/. It looks for bad practices when using Django web framework.
Other wrappers
Coala - wraps all popular Python linters: bandit, isort, mypy, pycodestyle, pydocstyle, pyflakes, pylint, pyroma, radon, vulture, yapf. Has 2.9k stars on github and is actively maintained. Also, it supports Jupyter notebooks.
Pylama - wraps mccabe, pycodestyle, pydocstyle, pyflakes, pylint, radon. Has 626 stars with last commit more than a year ago (as of November 2019).
prospector - wraps dodgy, frosted, mccabe, pycodestyle, pydocstyle, pyflakes, pylint, pyroma, vulture. Same authors as flake8 with 1.2k stars.
wemake-python-styleguide is a flake8 with some other plugins as dependencies. 535 stars.
Ciocheck wraps autopep8, flake8, isort, pep8, pydocstyle, pylint, pytest-cov, yapf.Some of them - like autopep8 - are formatters. They will automatically format code to remove found issues.
pylint
From the authors of flake8 - pylint, checks for PEP8-like code style, some code smells and type errors. It intersects with flake8 in many regards, but doesn’t check for complexity (can be enabled through the plugin pylint.extensions.mccabe).
Some report that pylint is more thorough than flake8 and whinier. It will take more time to reach the state “no linter issues” from “not using linter at all” state with pylint than with flake8.
Security linters
safety - check dependencies for known vulnerabilities. The vulnerability database is owned by pyupio. The company also provides CI service that integrates with GitHub for as low as $29 p.m. for private repos (free for open source).
bandit - builds a tree of python objects and run it against known unsafe coding practices - things like usage exec or assert, pickle or debug=True for flask app.
dodgy - looks for dodgy code. It intersects a bit with bandit. It can check for hardcoded passwords and such.
black
black - the popularity of this tool exploded recently. Black doesn’t just report on issues - it formats the code the way it likes. Very nice tool with no configuration - it just works and everybody is ok with that.
Experiment
I’ll try to use some of the above-mentioned linters with default settings on requests source code.
Pylint gave a lot of warnings and errors
~ pylint -f parseable requests/requests | grep -c "^requests"
No config file found, using default configuration
368
It also gave the library 7.70/10 score.
Default flake8:
~ flake8 requests/requests | grep -c "^requests"
247
While most of the are line too long (>79 chars). Otherwise, it’s:
~ flake8 --ignore E501 requests/requests | grep -c "^requests"
111
Bandit found 11 issues - insecure hash functions and using asserts
~ bandit -r requests/requests | grep -c "^>>"
[main] INFO profile include tests: None
[main] INFO profile exclude tests: None
[main] INFO cli include tests: None
[main] INFO cli exclude tests: None
[main] INFO running on Python 3.6.8
11
Do linters increase code quality?
I think the linters cover such internal quality characteristics as:
- readability through checking code style
- maintainability through finding known bad practices and anti-patterns
Security linters also scratch the surface of external characteristics - reliability by increasing security.
Do you use linters? Drop me a message in LinkedIn