1. 技术文章
  2. Pants includes OS X specific Python wheels

Pants includes OS X specific Python wheels

由 早过忘川 提交于 2019-12-03 03:46:52

One of the nice things about distributing your project as a PEX file is that you can prepare it to run on multiple platforms. For example, one PEX can run on both Linux and Mac platforms. For many projects, there is nothing special to do other than build a PEX. But when your project has dependencies on platform specific binary code, you will need to perform some extra steps.

One example of a library that contains platform specific code is the psutil library. It contains C code that it compiled into a shared library when the module is installed. To create a PEX file that contains such dependencies, you must first provide a pre-built version of that library for all platforms other than the one where you are running pants.

The easiest way to pre-build libraries is to use the pip tool to build wheels.

This recipe assumes the following:

  • You want to build a multi-platform pex to run on both Linux and mac You are going to pre-build the libraries in the Linux environment, then build the PEX on the mac environment.
  • Your project directory lives under ~/src/cookbook

Let’s take a simple program that references a library and create a pex from it. 

#  src/python/ps_example/main.py
import psutil

for proc in psutil.process_iter():
    try:
        pinfo = proc.as_dict(attrs=['pid', 'name'])
    except psutil.NoSuchProcess:
        pass
    else:
        print(pinfo)

With Pants, you can define an executable by defining a python_binary target in a BUILD file:

# src/python/ps_example/BUILD
python_binary(name='ps_example',
  source = 'main.py',
  dependencies = [
    ':psutil',  # defined in requirements.txt
  ],
)

# Defines targets from specifications in requirements.txt
python_requirements()

In the same directory, list the python libraries in a requirements.txt file:

# src/python/ps_example/requirements.txt 
psutil==3.1.1

Now, to to make the multi-platform pex, you'll need access to a Linux box to create the linux version of psutil wheel. Copy the requirements.txt file to the linux machine, then, execute the pip tool:

linux $ mkdir ~/src/cookbook/wheelhouse
linux $ pip wheel -r src/python/multi-platform/requirements.txt  \
    --wheel-dir=~/src/cookbook/wheelhouse

This will create a platform specific wheel file.

linux $ ls ~/src/cookbook/wheelhouse/
psutil-3.1.1-cp27-none-linux_x86_64.whl

Now, you will need to copy the platform specific wheel over to the machine where you want to build your multi-platform pex (in this case, your mac laptop). If you use this recipe on a regular basis, you will probably want to configure a Python Respository to store your pre-built libraries.

We’ll use the same BUILD file setup as in above, but modify python_binary to specify the platforms= parameter.

# src/python/ps_example/BUILD
python_binary(name='ps_example',
  source = 'main.py',
  dependencies = [
    ':psutil',  # defined in requirements.txt
  ],
  platforms=[
    'linux-x86_64',
    'macosx-10.7-x86_64',
  ],
)

# Defines targets from specifications in requirements.txt
python_requirements()

You will also need to tell pants where to find the pre-built python packages. Edit pants.ini and add:

[python-repos]
repos: [
    "%(buildroot)s/wheelhouse/"
  ]

Now, copy the file psutil-3.1.1-cp27-none-linux_x86_64.whl over to the mac workstation and place it in a directory named wheelhouse/ under the root of your repo.

Once this is done you can now build the multi-platform pex with

mac $ ./pants binary src/python/py_example

You can verify that libraries for both mac and Linux are included in the pex by unzipping it:

mac $ unzip -l dist/ps_example.pex | grep psutil
    17290  12-21-15 22:09   .deps/psutil-3.1.1-cp27-none-linux_x86_64.whl/psutil-3.1.1.dist-info/DESCRIPTION.rst
    19671  12-21-15 22:09   .deps/psutil-3.1.1-cp27-none-linux_x86_64.whl/psutil-3.1.1.dist-info/METADATA
     1340  12-21-15 22:09   .deps/psutil-3.1.1-cp27-none-linux_x86_64.whl/psutil-3.1.1.dist-info/RECORD
      103  12-21-15 22:09  
...   .deps/psutil-3.1.1-cp27-none-macosx_10_11_intel.whl/psutil-3.1.1.dist-info/DESCRIPTION.rst
    19671  12-21-15 22:09   .deps/psutil-3.1.1-cp27-none-macosx_10_11_intel.whl/psutil-3.1.1.dist-info/METADATA
     1338  12-21-15 22:09   .deps/psutil-3.1.1-cp27-none-macosx_10_11_intel.whl/psutil-3.1.1.dist-info/RECORD
      109  12-21-15 22:09   
...
标签
pants
易学教程内所有资源均来自网络或用户发布的内容，如有违反法律规定的内容欢迎反馈！
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!