Luigi - Unfulfilled %s at run time

做~自己de王妃 提交于 2019-12-09 16:32:15

问题


I am trying to learn in a very simple way how luigi works. Just as a newbie I came up with this code

import luigi

class class1(luigi.Task):

  def requires(self):
     return class2()

  def output(self):
    return luigi.LocalTarget('class1.txt')

 def run(self):
    print 'IN class A'


class class2(luigi.Task): 

  def requires(self):
     return []

  def output(self):
     return luigi.LocalTarget('class2.txt')


if __name__ == '__main__':
  luigi.run()

Running this in command prompt gives error saying

raise RuntimeError('Unfulfilled %s at run time: %s' % (deps, ',     

回答1:


This happens because you define an output for class2 but never create it.

Let's break it down...

When running

python file.py class2 --local-scheduler

luigi will ask:

  • is the output of class2 already on disk? NO
  • check dependencies of class2: NONE
  • execute the run method (by default it's and empty method pass)
  • run method didn't return errors, so job finishes successfully.

However, when running

python file.py class1 --local-scheduler

luigi will:

  • is the output of class1 already on disk? NO
  • check task dependencies: YES: class2
  • pause to check status of class2
    • is the output of class2 on disk? NO
    • run class2 -> running -> done without errors
    • is the output of class2 on disk? NO -> raise error

luigi never runs a task unless all of its previous dependencies are met. (i.e. their output is on the file system)



来源:https://stackoverflow.com/questions/43107177/luigi-unfulfilled-s-at-run-time

标签
易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!