How do you override Google AI platform's standard library's (i.e upgrade scikit-learn) and install other libraries for custom prediction routines?

问题

I'm currently building a pipeline and trying to see if I can get an ML model deployed in AI platform's prediction service, then use it later on in other projects via the HTTP request that the prediction service offers.

However the model that is being used was built using an scikit-learn library that is of a higher version than offered for the prediction runtime version 1.15 (this is the current version supported by google for scikit-learn predictions). This runtime version only supports scikit-learn version 0.20.4 and my model requires 0.23.1. As far as I know, everything else in the custom prediction routine works as intended, but the error received when loading the model () is only ever encountered when the scikit-learn version is older than the model needs.

So, I need a way to force the prediction routine to use a particular version of scikit-learn via a pip install or some equivalent - in the past I have done this in Google Dataflow via custom installs in the setup.py file but have yet to succeed achieving this in AI platform custom prediction routines. I assume it can be done?

non-working 'setup.py'

from setuptools import setup
from setuptools import find_packages

REQUIRED_PACKAGES = ['scikit-learn>=0.23.1',
                 'mlxtend>=0.17.2']

setup(
    name='my_custom_code',
    version='0.1',
    install_requires=REQUIRED_PACKAGES,
    packages=find_packages(),
    include_package_data=True,
    scripts=['predictor.py']
)

回答1:

This is a bit of a guessed idea since I haven't hit this problem before, but let me know if it works: could you specify an additional package uri via --package_uris that contains the version of scikit-learn that you would like?

You can get the tarball of your desired scikit-learn version via:

pip download scikit-learn==0.23.1 --no-binary=:all:

Shameless plug: You should check out my platform. It supports direct scikit-learn model -> HTTP endpoint deployment and is way easier to use :) Shoot me an email at contact@modelzoo.dev and I can put you on my beta list.

回答2:

So it turns out google currently does not support this capability. There is a closed alpha at this stage for AI Platform Prediction Custom Containers Alpha - but for the time being I've achieved the same result using Dataflow with a setup.py file using custom pip install commands.

来源：https://stackoverflow.com/questions/62816129/how-do-you-override-google-ai-platforms-standard-librarys-i-e-upgrade-scikit