pathlib Path and py.test LocalPath

Question

I have started using pathlib.Path some time ago and I like using it. Now that I have gotten used to it, I have gotten sloppy and forget to cast arguments to str.

This often happens when using tox + py.test with temporary directories based on tmpdir (which is a py._path.local.LocalPath):

from pathlib import Path
import pytest

def test_tmpdir(tmpdir):
    p = Path(tmpdir) / 'testfile.csv'

Instead of inserting str() every time, I looked at solving this more generally, but could not.

First I tried to make my own Path class that has an adapted _parse_args:

import pytest
from py._path.local import LocalPath
from pathlib import Path, PurePath

def Path(Path):
    @classmethod
    def _parse_args(cls, args):
        parts = []
        for a in args:
            if isinstance(a, PurePath):
                parts += a._parts
            elif isinstance(a, str):
                # Force-cast str subclasses to str (issue #21127)
                parts.append(str(a))
            elif isinstance(a, LocalPath):
                parts.append(str(a))
            else:
                raise TypeError(
                    "argument should be a path or str object, not %r"
                    % type(a))
        return cls._flavour.parse_parts(parts)

def test_subclass(tmpdir):
    p = Path(tmpdir) / 'testfile.csv'

This throws a TypeError: unsupported operand type(s) for /: 'NoneType' and 'str' (tried with PosixPath as well, same result, would prefer not to be Linux specific).

The I tried to monkey-patch Path:

import pytest
from pathlib import Path

def add_tmpdir():
    from py._path.local import LocalPath

    org_attr = '_parse_args'
    stow_attr = '_org_parse_args'

    def parse_args_localpath(cls, args):
        args = list(args)
        for idx, a in enumerate(args):
            if isinstance(a, LocalPath):
                args[idx] = str(a)
        return getattr(cls, stow_attr)(args)

    if hasattr(Path, stow_attr):
        return  # already done
    setattr(Path, stow_attr, getattr(Path, org_attr))
    setattr(Path, org_attr, parse_args_localpath)

add_tmpdir()

def test_monkeypatch_path(tmpdir):
    p = Path(tmpdir) / 'testfile.csv'

This throws a AttributeError: type object 'Path' has no attribute '_flavour' (also when monkey-patching PurePath).

And finally I tried just wrapping Path:

import pytest
import pathlib

def Path(*args):
    from py._path.local import LocalPath
    args = list(args)
    for idx, a in enumerate(args):
        if isinstance(a, LocalPath):
            args[idx] = str(a)
    return pathlib.Path(*args)

def test_tmpdir_path(tmpdir):
    p = Path(tmpdir) / 'testfile.csv'

Which also gives the AttributeError: type object 'Path' has no attribute '_flavour'

I thought at some point this last one worked, but I cannot reproduce that.
Am I doing something wrong? Why is this so hard?

Answer 1

That last one (wrapping) should work, I suspect you actually test all of these in one py.test/tox run and that monkey-patch is still in effect (that might explain why it worked at some point, the order of the test files etc matters if you start to change things on global classes).

That this is hard, is because of Path essentially being a generator, that on the fly decides whether you are on Windows or Linux, and creates a WindowsPath resp. PosixPath accordingly.

BDFL Guido van Rossum already indicated in May 2015:

It does sound like subclassing Path should be made easier.

but nothing happened. Support for pathlib in 3.6 within other standard libraries has increased, but pathlib itself still has the same problems.

Answer 2

In case anyone else is researching whether pytest's tmpdir paths play nicely with pathlib.Path:

Using python 3.6.5 and pytest 3.2.1, the code posted in the question works perfectly fine without explicitly casting to str:

from pathlib import Path

def test_tmpdir(tmpdir):
    p = Path(tmpdir) / 'testfile.csv'