How to write code that works in both Python 2 and Python 3?

Question

A Django website I maintain currently uses Python 2.7 but I know that I\'ll have to upgrade it to Python 3 in a couple of months. If I\'m writing code right now that has to

Answer 1

you can import the future package

from future import ....

nested_scopes 2.1.0b1 2.2 PEP 227: Statically Nested Scopes

generators 2.2.0a1 2.3 PEP 255: Simple Generators

division 2.2.0a2 3.0 PEP 238: Changing the Division Operator

absolute_import 2.5.0a1 3.0 PEP 328: Imports: Multi-Line and Absolute/Relative

with_statement 2.5.0a1 2.6 PEP 343: The “with” Statement

print_function 2.6.0a2 3.0 PEP 3105: Make print a function

unicode_literals 2.6.0a2 3.0 PEP 3112: Bytes literals in Python 3000

Answer 2

There are a few different tools that will help you make sure you are writing python2/3 compatible code.

If you are interested in porting python2 code into python3, then the 2to3 program that comes with the standard library will try to convert a python 2 program to python 3.

https://docs.python.org/2/library/2to3.html

Another great tool is pylint. pylint is a python linter that will describe issues to you without fixing them. If you pip install pylint on a python3 environment, then it will analyze your code based on python 3's rules. If you use python 2 to install pylint, it will do the same but with python 2's rules.

There are other popular and similar tools like flake8 or autopep8, but I am not familiar with them enough to advertise them.

Answer 3

six and future is a golden rule, enough to make easy a coming migration

add to every python2 file, this as first line:

from __future__ import absolute_import, unicode_literals

use below working with strings, iteration, metaclasses, ...

isinstance(sth, six.string_types)

six.iteritems(dict)

@six.add_metaclass(Meta)

and so on six reference

Answer 4

Making your Django project compatible with both Python versions consists of the following steps:

1. Add from __future__ import unicode_literals at the top of each module and then use usual quotes without a u prefix for Unicode strings and a b prefix for bytestrings.

2. To ensure that a value is bytestring, use the django.utils.encoding.smart_bytes function. To ensure that a value is Unicode, use the django.utils.encoding.smart_text or django.utils.encoding.force_text function.

3. In your models use __str__ method instead of __unicode__ and add the python_2_unicode_compatible decorator.

   # models.py
   # -*- coding: UTF-8 -*-
   from __future__ import unicode_literals
   from django.db import models
   from django.utils.translation import ugettext_lazy as _
   from django.utils.encoding import python_2_unicode_compatible
   
   @python_2_unicode_compatible
   class NewsArticle(models.Model):
       title = models.CharField(_("Title"), max_length=200)
        content = models.TextField(_("Content"))
   
       def __str__(self):
           return self.title
   
       class Meta:
           verbose_name = _("News Article")
           verbose_name_plural = _("News Articles")
   

4. To iterate through dictionaries, use iteritems() , iterkeys() , and itervalues() from django.utils.six . Take a look at the following:

   from django.utils.six import iteritems
   
   d = {"imported": 25, "skipped": 12, "deleted": 3}
   for k, v in iteritems(d):
       print("{0}: {1}".format(k, v))
   

5. At the time of capturing exceptions, use the as keyword, as follows:

   try:
       article = NewsArticle.objects.get(slug="hello-world")
   except NewsArticle.DoesNotExist as exc:
       pass
   except NewsArticle.MultipleObjectsReturned as exc:
       pass
   

6. Use django.utils.six to check the type of a value as shown in the following:

   from django.utils import six
   
   isinstance(val, six.string_types) # previously basestring
   isinstance(val, six.text_type) # previously unicode
   isinstance(val, bytes) # previously str
   isinstance(val, six.integer_types) # previously (int, long)
   

7. Use range from django.utils.six.moves ,Instead of xrange , as follows:

   from django.utils.six.moves import range
   
   for i in range(1, 11):
      print(i)
   

Source link

Answer 5

There is official documentation suggesting ways to do this. That documentation has changed over time as the situation has changed, so it's worth going directly to the source (especially if you're reading this answer a year or two after it was written).

It's also worth reading the Conservative Python 3 Porting Guide and skimming Nick Coghlan's Python 3 Q&A, especially this section.

Going back in time from the early 2018:

futurize

The current official suggestions are:

• Only worry about supporting Python 2.7
• Make sure you have good test coverage (coverage.py can help; pip install coverage)
• Learn the differences between Python 2 & 3
• Use Futurize (or Modernize) to update your code (e.g. pip install future)
• Use Pylint to help make sure you don’t regress on your Python 3 support (pip install pylint)
• Use caniusepython3 to find out which of your dependencies are blocking your use of Python 3 (pip install caniusepython3)
• Once your dependencies are no longer blocking you, use continuous integration to make sure you stay compatible with Python 2 & 3 (tox can help test against multiple versions of Python; pip install tox)
• Consider using optional static type checking to make sure your type usage works in both Python 2 & 3 (e.g. use mypy to check your typing under both Python 2 & Python 3).

Notice the last suggestion. Guido and another of the core devs have both been heavily involved in leading large teams to port large 2.7 codebases to 3.x, and found mypy to be very helpful (especially in dealing with bytes-vs.-unicode issues). In fact, that's a large part of the reason gradual static typing is now an official part of the language.

You also almost certainly want to use all of the future statements available in 2.7. This is so obvious that they seem to have forgotten to leave it out of the docs, but, besides making your life easier (e.g., you can write print function calls), futurize and modernize (and six and sixer) require it.

six

The documentation is aimed at people making an irreversible transition to Python 3 in the near future. If you're planning to stick with dual-version code for a long time, you might be better off following the previous recommendations, which largely revolved around using six instead of futurize. Six covers more of the differences between the two languages, and also makes you write code that's explicit about being dual-version instead of being as close to Python 3 as possible while still running in 2.7. But the downside is that you're effectively doing two ports—one from 2.7 to six-based dual-version code, and then, later, from 3.x-only six code to 3.x-only "native" code.

2to3

The original recommended answer was to use 2to3, a tool that can automatically convert Python 2 code to Python 3 code, or guide you in doing so. If you want your code to work in both, you need to deliver Python 2 code, then run 2to3 at installation time to port it to Python 3. Which means you need to test your code both ways, and usually modify it so that it still works in 2.7 but also works in 3.x after 2to3, which isn't always easy to work out. This turns out to not be feasible for most non-trivial projects, so it's no longer recommended by the core devs—but it is still built in with Python 2.7 and 3.x, and getting updates.

There are also two variations on 2to3: sixer auto-ports your Python 2.7 code to dual-version code that uses six, and 3to2 lets you write your code for Python 3 and auto-port back to 2.7 at install time. Both of these were popular for a time, but don't seem to be used much anymore; modernize and futurize, respectively, are their main successors.

---

For your specific question,

• kwargs.items() will work on both, if you don't mind a minor performance cost in 2.7.
• 2to3 can automatically change that iteritems to items at install time on 3.x.
• futurize can be used to do either of the above.
• six will allow you to write six.iteritems(kwargs), which will do iteritems in 2.7 and items in 3.x.
• six will also allow you to write six.viewitems(kwargs), which will do viewitems in 2.7 (which is identical to what items does in 3.x, rather than just similar).
• modernize and sixer will automatically change that kwargs.iteritems() to six.iteritems(kwargs).
• 3to2 will let you write kwargs.items() and autmatically convert it to viewitems at install time on 2.x.
• mypy can verify that you're just using the result as a general iterable (rather than specifically as an iterator), so changing to viewitems or items leaves your code still correctly typed.

Answer 6

In addition to importing from future, there is also the six project that aims to provide api compatibility between Python 2 and Python 3: https://pypi.org/project/six/.

Your example code could be made compatible between version 2 and 3:

import six

for key, value in six.iteritems(dict):
    log.info("{0}: {1}".format(key, value))

There are still things that won't be compatible between 2 and 3 like f-strings.