airbnb

Creating connection outside of Airflow GUI

瘦欲@ 提交于 2019-12-05 23:37:10
问题 I would like to create S3 connection without interacting Airflow GUI. Is it possible through airflow.cfg or command line? We are using AWS role and following connection parameter works for us: {"aws_account_id":"xxxx","role_arn":"yyyyy"} So, manually creating connection on GUI for S3 is working, now we want to automate this process and want to add it as part of the Airflow deployment process. Any work around? 回答1: You can use the airflow CLI. Unfortunately there is no support for editing

Creating connection outside of Airflow GUI

纵饮孤独 提交于 2019-12-04 06:22:06
I would like to create S3 connection without interacting Airflow GUI. Is it possible through airflow.cfg or command line? We are using AWS role and following connection parameter works for us: {"aws_account_id":"xxxx","role_arn":"yyyyy"} So, manually creating connection on GUI for S3 is working, now we want to automate this process and want to add it as part of the Airflow deployment process. Any work around? You can use the airflow CLI. Unfortunately there is no support for editing connections, so you would have to remove and add as part of your deployment process, e.g.: airflow connections

Airflow: Tasks queued but not running

孤街浪徒 提交于 2019-12-04 03:17:47
问题 I am new to airflow and trying to setup airflow to run ETL pipelines. I was able to install airflow postgres celery rabbitmq I am able to test run the turtorial dag. When i try to schedule the jobs, scheduler is able to pick it up and queue the jobs which i could see on the UI but tasks are not running. Could somebody help me fix ths issue? I believe i am missing most basic airflow concept here. below is the airflow.cfg Here is my config file: [core] airflow_home = /root/airflow dags_folder =

How to acquire API key for Airbnb

♀尐吖头ヾ 提交于 2019-12-03 10:13:53
I know that Airbnb haven't opened their API to the public yet, but searching the internet I found some people are using it. I tried to contact them, and also Airbnb, but without any answers. Does anyone here knows any contact email, page, or phone number that I can refer to? I have read here that you can find your API key by looking at the requests that AirBNB uses in their own website. So use the web-developer tools in chrome, or firefox, or firebug in firefox and search trough the requests in the network panel until you find the key being used in any JSON request. Some urls contain this key

Trailing comma in JavaScript function call

你。 提交于 2019-12-02 04:03:41
问题 I'm trying to follow the JS code style defined by Airbnb. The rule on trailing commas for function call arguments states: 7.15 Functions with multiline signatures, or invocations, should be indented just like every other multiline list in this guide: with each item on a line by itself, with a trailing comma on the last item. But when I do the following: /* THREE.js constructor for PerspectiveCamera */ const camera = new THREE.PerspectiveCamera( 75, window.innerWidth / window.innerHeight, 0.1,

Trailing comma in JavaScript function call

跟風遠走 提交于 2019-12-02 00:34:21
I'm trying to follow the JS code style defined by Airbnb . The rule on trailing commas for function call arguments states: 7.15 Functions with multiline signatures, or invocations, should be indented just like every other multiline list in this guide: with each item on a line by itself, with a trailing comma on the last item. But when I do the following: /* THREE.js constructor for PerspectiveCamera */ const camera = new THREE.PerspectiveCamera( 75, window.innerWidth / window.innerHeight, 0.1, 1000, ); Google Chrome complains with the following error: app.js:11 Uncaught SyntaxError: Unexpected

Airflow: Tasks queued but not running

泄露秘密 提交于 2019-12-01 17:25:18
I am new to airflow and trying to setup airflow to run ETL pipelines. I was able to install airflow postgres celery rabbitmq I am able to test run the turtorial dag. When i try to schedule the jobs, scheduler is able to pick it up and queue the jobs which i could see on the UI but tasks are not running. Could somebody help me fix ths issue? I believe i am missing most basic airflow concept here. below is the airflow.cfg Here is my config file: [core] airflow_home = /root/airflow dags_folder = /root/airflow/dags base_log_folder = /root/airflow/logs executor = CeleryExecutor sql_alchemy_conn =

Airflow does not backfill latest run

不羁岁月 提交于 2019-12-01 11:20:10
For some reason, Airflow doesn't seem to trigger the latest run for a dag with a weekly schedule interval. Current Date: $ date $ Tue Aug 9 17:09:55 UTC 2016 DAG: from datetime import datetime from datetime import timedelta from airflow import DAG from airflow.operators.bash_operator import BashOperator dag = DAG( dag_id='superdag', start_date=datetime(2016, 7, 18), schedule_interval=timedelta(days=7), default_args={ 'owner': 'Jon Doe', 'depends_on_past': False } ) BashOperator( task_id='print_date', bash_command='date', dag=dag ) Run scheduler $ airflow scheduler -d superdag You'd expect a

Setting up Airbnb ESLint with React and Webpack

不羁岁月 提交于 2019-11-30 08:42:43
I am trying to set up linting with Airbnb's Javascript standards on my React project, it uses webpack. Updated with latest packages based on comments. "babel-eslint": "^6.1.2", "eslint": "^3.2.2", "eslint-config-airbnb": "^10.0.0", "eslint-plugin-import": "^1.12.0", "eslint-plugin-jsx-a11y": "^2.0.1", "eslint-plugin-react": "^6.0.0", "jshint": "^2.9.2", "jshint-loader": "^0.8.3", "json-loader": "^0.5.4", I also have a preloader setup in my webpack config preLoaders: [ { test: /\.jsx?$/, loaders: ['eslint'], // define an include so we check just the files we need include: PATHS.app } ], And the

Setting up Airbnb ESLint with React and Webpack

故事扮演 提交于 2019-11-29 12:07:27
问题 I am trying to set up linting with Airbnb's Javascript standards on my React project, it uses webpack. Updated with latest packages based on comments. "babel-eslint": "^6.1.2", "eslint": "^3.2.2", "eslint-config-airbnb": "^10.0.0", "eslint-plugin-import": "^1.12.0", "eslint-plugin-jsx-a11y": "^2.0.1", "eslint-plugin-react": "^6.0.0", "jshint": "^2.9.2", "jshint-loader": "^0.8.3", "json-loader": "^0.5.4", I also have a preloader setup in my webpack config preLoaders: [ { test: /\.jsx?$/,