Tool to convert Python code to be PEP8 compliant

Question

I know there are tools which validate whether your Python code is compliant with PEP8, for example there is both an online service and a python module.

However, I c

Answer 1

There are many.

IDEs usually have some formatting capability built in. IntelliJ Idea / PyCharm does, same goes for the Python plugin for Eclipse, and so on.

There are formatters/linters that can target multiple languages. https://coala.io is a good example of those.

Then there are the single purpose tools, of which many are mentioned in other answers.

One specific method of automatic reformatting is to parse the file into AST tree (without dropping comments) and then dump it back to text (meaning nothing of the original formatting is preserved). Example of that would be https://github.com/python/black.

Answer 2

You can use autopep8! Whilst you make yourself a cup of coffee this tool happily removes all those pesky PEP8 violations which don't change the meaning of the code.

Install it via pip:

pip install autopep8

Apply this to a specific file:

autopep8 py_file --in-place

or to your project (recursively), the verbose option gives you some feedback of how it's going:

autopep8 project_dir --recursive --in-place --pep8-passes 2000 --verbose

Note: Sometimes the default of 100 passes isn't enough, I set it to 2000 as it's reasonably high and will catch all but the most troublesome files (it stops passing once it finds no resolvable pep8 infractions)...

At this point I suggest retesting and doing a commit!

If you want "full" PEP8 compliance: one tactic I've used is to run autopep8 as above, then run PEP8, which prints the remaining violations (file, line number, and what):

pep8 project_dir --ignore=E501

and manually change these individually (e.g. E712s - comparison with boolean).

Note: autopep8 offers an --aggressive argument (to ruthlessly "fix" these meaning-changing violations), but beware if you do use aggressive you may have to debug... (e.g. in numpy/pandas True == np.bool_(True) but not True is np.bool_(True)!)

You can check how many violations of each type (before and after):

pep8 --quiet --statistics .

Note: I consider E501s (line too long) are a special case as there will probably be a lot of these in your code and sometimes these are not corrected by autopep8.

As an example, I applied this technique to the pandas code base.

Answer 3

If you're using eclipse + PyDev you can simply activate autopep8 from PyDev's settings: Windows -> Preferences -> type 'autopep8' in the search filter.

Check the 'use autopep8.py for code formatting?' -> OK

Now eclipse's CTRL-SHIFT-F code formatting should format your code using autopep8 :)

Answer 4

@Andy Hayden gave good overview of autopep8. In addition to that there is one more package called pep8ify which also does the same thing.

However both packages can remove only lint errors but they cannot format code.

little = more[3:   5]

Above code remains same after pep8ifying also. But the code doesn't look good yet. You can use formatters like yapf, which will format the code even if the code is PEP8 compliant. Above code will be formatted to

little = more[3:5]

Some times this even destroys Your manual formatting. For example

BAZ = {
    [1, 2, 3, 4],
    [5, 6, 7, 8],
    [9, 10, 11, 12]
}

will be converted to

BAZ = {[1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 11, 12]}

But You can tell it to ignore some parts.

BAZ = {
   [1, 2, 3, 4],
   [5, 6, 7, 8],
   [9, 10, 11, 12]
}  # yapf: disable

Taken from my old blog post: Automatically PEP8 & Format Your Python Code!

Answer 5

Unfortunately "pep8 storming" (the entire project) has several negative side-effects:

• lots of merge-conflicts
• break git blame
• make code review difficult

As an alternative (and thanks to @y-p for the idea), I wrote a small package which autopep8s only those lines which you have been working on since the last commit/branch:

Basically leaving the project a little better than you found it:

pip install pep8radius

Suppose you've done your work off of master and are ready to commit:

# be somewhere in your project directory
# see the diff with pep, see the changes you've made since master
pep8radius master --diff
# make those changes
pep8radius master --diff --in-place

Or to clean the new lines you've commited since the last commit:

pep8radius --diff
pep8radius --diff --in-place

# the lines which changed since a specific commit `git diff 98f51f`
pep8radius 98f51f --diff

Basically pep8radius is applying autopep8 to lines in the output of git/hg diff (from the last shared commit).

This script currently works with git and hg, if your using something else and want this to work please post a comment/issue/PR!

Answer 6

I made wide research about different instruments for python and code style. There are two types of instruments: linters - analyzing your code and give some warnings about bad used code styles and showing advices how to fix it, and code formatters - when you save your file it re-format your document using PEP style.

Because re-formatting must be more accurate - if it remorfat something that you don't want it became useless - they cover less part of PEP, linters show much more.

All of them have different permissions for configuring - for example, pylinter configurable in all its rules (you can turn on/off every type of warnings), black unconfigurable at all.

Here are some useful links and tutorials:

Documentation:

• PEP-257 Docstring Conventions: https://www.python.org/dev/peps/pep-0257/
• PEP-484 Type Hint: https://www.python.org/dev/peps/pep-0484
• Chromium Style Guide https://chromium.googlesource.com/chromiumos/docs/+/master/styleguide/python.md
• Code Style for autotest https://chromium.googlesource.com/chromiumos/third_party/autotest/+/master/docs/coding-style.md
• Khan Academy Coding Style Guide https://github.com/Khan/style-guides/blob/master/style/python.md
• The hitchhiker's Guide to Python https://docs.python-guide.org/
• EdX Python Style Guide https://edx.readthedocs.io/projects/edx-developer-guide/en/latest/style_guides/python-guidelines.html
• Code Style Article on RealPython https://realpython.com/python-pep8/

Linters (in order of popularity):

• mypy https://github.com/python/mypy linter for type checks (PEP-484)
• pycodestyle https://github.com/PyCQA/pycodestyle - good one using PEP-8, very popular. Often used alongside of pylint and flake8 (simultaniously)
• pylint https://github.com/PyCQA/pylint very configurable, actively supported
• bandit https://github.com/PyCQA/bandit линтер по безопасности
• prospector https://github.com/PyCQA/prospector pylint+code difficulty check
• flake8 https://github.com/PyCQA/flake8 pycodestyle wrapper with ability to turn on plugins. Very big list of different configurable plugins. Here is awesome flake8 repo: https://github.com/DmytroLitvinov/awesome-flake8-extensions
• wemake https://github.com/wemake-services/wemake-python-styleguide - trying to combine a lot of different linters in one project (really it is a flake8 plugin combining styles from several other linters)
• pylama https://github.com/klen/pylama trying to combine 10 another linters in one (mypy, pylint, pycodeestyle, pydocstyle и др.). I can see the only one problem here - old version (no updates in github repo for about 10 months.)
• pydocstyle https://github.com/PyCQA/pydocstyle docstrings linter (PEP-257)

Code formatters (in order of popularity):

• black https://github.com/psf/black most populat formatter, used in several big companies. Was created later than yapf, but already has more starts at GitHub
• yapf https://github.com/google/yapf Google code formatter
• autopep8 https://github.com/hhatto/autopep8 build upon the pycodestyle