问题
In my tasks.py file, when I import hmmlearn,
from hmmlearn import hmm
and start my celery workers, I get the following error
[2017-06-14 09:18:27,638: INFO/MainProcess] Received task:
sm.tasks.mytask[4e46806e-6f0f-420f-baac-c727c2a382d4]
[2017-06-14 09:18:27,716: ERROR/MainProcess] Process 'Worker-4' pid:5264
exited with 'exitcode 1'
[2017-06-14 09:18:29,857: ERROR/MainProcess] Process 'Worker-7' pid:3172
exited with 'exitcode 1'
[2017-06-14 09:18:29,857: ERROR/MainProcess] Process 'Worker-6' pid:5768
exited with 'exitcode 1'
[2017-06-14 09:18:29,857: ERROR/MainProcess] Process 'Worker-5' pid:5236
exited with 'exitcode 1'
[2017-06-14 09:18:31,450: ERROR/MainProcess] Process 'Worker-8' pid:5876
exited with 'exitcode 1'
And after I shutdown the worker,
[2017-06-14 09:19:28,545: WARNING/MainProcess] c:\anaconda3\lib\site-
packages\celery\apps\worker.py:161: CDeprecationWarning:
Starting from version 3.2 Celery will refuse to accept pickle by default.
If I just comment out that import and code using that import, everything works fine. But, I'm able to execute all the tasks(including the hmm code) as standalone python code on ipython without any issues.
I'm using the conda distribution with following details
Current conda install:
platform : win-64
conda version : 4.3.21
conda is private : False
conda-env version : 4.3.21
conda-build version : 1.21.3
python version : 3.5.2.final.0
requests version : 2.14.2
λ conda list | grep celery
celery 3.1.18 <pip>
λ conda list | grep kombu
kombu 3.0.37 <pip>
λ conda list | grep hmmlearn
hmmlearn 0.1.1 np111py35_0 omnia
What should I do?
回答1:
This might be because celery 3.1.xx comes bundled with billiard 3.3.
If you upgrade that package (to 3.5 at time of writing), the service might work again.
pip install --upgrade billiard
回答2:
I just stumbled upon a similar situation. Upgrading billiard
to 3.5, as suggested in a different answer, was not an option (because it conflicts with Celery==3.1.25
and I prefer that particular version for Windows its support).
I figured out, however, that in my case the problem was most probably due to this issue - it occurred only when I tried to import anything from sklearn
in the Worker's process.
The problem was resolved by adding this snippet before the import
s from sklearn
:
from multiprocessing import current_process
try:
current_process()._config
except AttributeError:
current_process()._config = {'semprefix': '/mp'}
来源:https://stackoverflow.com/questions/44550407/celery-worker-n-pidxxxx-exited-with-exitcode-1-when-i-import-hmmlearn