in hql scripts we use “!sh echo ---new line---” for the same . Want to know the alternative for this in impala to print any line in impala scripts?

问题

in hql scripts we use "!sh echo ---new line---" for the same . Want to know the alternative for this in impala to print any line in impala scripts?

回答1:

You can invoke the shell command line from impala scripts.

As an example of how it works.

script_impala.sql

-- set a variable containing the of the game
SET hivevar:game=Monopoly;
-- return the list of the game
SELECT list_price FROM fun.games WHERE name = '${hivevar:game}';
-- return the prices of the game ate game shops
SELECT shop, price FROM fun.inventory WHERE game = '${hivevar:game}';

shell hdfs dfs -ls /user;
shell ls -ltr;
shell echo I can echo from impala scripts;
shell cat hex_color.sql

$ impala-shell -f script_impala.sql

output

Starting Impala Shell without Kerberos authentication
Connected to localhost.localdomain:21000
Server version: impalad version 2.10.0-cdh5.13.3 RELEASE (build 15a453e15865344e75ce0fc6c4c760696d50f626)
Variable GAME set to Monopoly
Query: -- return the list of the game
SELECT list_price FROM fun.games WHERE name = 'Monopoly'
Query submitted at: 2020-06-04 23:29:19 (Coordinator: http://localhost.localdomain:25000)
Query progress can be monitored at: http://localhost.localdomain:25000/query_plan?query_id=a94afc7556843cf7:8b19809d00000000
+------------+
| list_price |
+------------+
| 19.99      |
+------------+
Fetched 1 row(s) in 0.15s
Query: -- return the prices of the game ate game shops
SELECT shop, price FROM fun.inventory WHERE game = 'Monopoly'
Query submitted at: 2020-06-04 23:29:19 (Coordinator: http://localhost.localdomain:25000)
Query progress can be monitored at: http://localhost.localdomain:25000/query_plan?query_id=c64d1c8950e5e1f8:15d410cb00000000
+-----------+-------+
| shop      | price |
+-----------+-------+
| Dicey     | 17.99 |
| Board 'Em | 25.00 |
+-----------+-------+
Fetched 2 row(s) in 0.14s
Found 5 items
drwxrwxrwt   - mapred   hadoop              0 2019-04-29 18:19 /user/history
drwxrwxrwx   - hive     hive                0 2019-04-29 18:19 /user/hive
drwxr-xr-x   - hue      hue                 0 2019-11-25 10:19 /user/hue
drwxr-xr-x   - spark    spark               0 2019-04-29 18:19 /user/spark
drwxrwxrwx   - training supergroup          0 2020-05-28 11:33 /user/training
--------
Executed in 2.10s
total 56
-rw-rw-r-- 1 training training   61 Sep 25  2019 hex_color.sql
-rw-rw-r-- 1 training training  115 Sep 25  2019 color_from_rgb.sql~
-rw-rw-r-- 1 training training   58 Sep 25  2019 hex_color_impala.sql
-rwxr-xr-x 1 training training  449 Sep 25  2019 email_results.sh
-rw-rw-r-- 1 training training 1166 Sep 25  2019 zero_air_time.csv
-rw-r--r-- 1 training training  261 Sep 26  2019 change_background.sh~
-rwxr-xr-x 1 training training  262 Sep 26  2019 change_background.sh
-rw------- 1 training training 2966 Sep 26  2019 ChangeVMDesktopColor.txt
-rw------- 1 training training 3279 Sep 26  2019 Hive&ImpalaInScripts&Applications.txt~
-rw------- 1 training training 3278 Sep 26  2019 Hive&ImpalaInScripts&Applications.txt
-rw-rw-r-- 1 training training  449 Sep 26  2019 email_resuts.sh
-rw-rw-r-- 1 training training  120 Apr 27 11:38 color_from_rgb.sql
-rw-rw-r-- 1 training training  397 Jun  4 23:24 game_prices.sql~
-rw-rw-r-- 1 training training  395 Jun  4 23:29 game_prices.sql
--------
Executed in 0.00s
I can echo from impala scripts
--------
Executed in 0.00s
SELECT hex FROM wax.crayons WHERE color = '${hivevar:color}'
--------
Executed in 0.01s

As you can see, you only have to write in your script

shell <command>;

You can even invoke hdfs dfs commands

shell hdfs dfs -<command>;

The above example prints the output to the console, if you want to print the output to a file you have to do as following to redirect all the output to a file:

$ impala-shell -f script_impala.sql >> /home/..../my_file.csv

Other option would be to redirect only the echo commands from inside the script. This would be as following: script_impala.sql

-- set a variable containing the of the game
SET hivevar:game=Monopoly;
-- return the list of the game
SELECT list_price FROM fun.games WHERE name = '${hivevar:game}';
-- return the prices of the game ate game shops
SELECT shop, price FROM fun.inventory WHERE game = '${hivevar:game}';

shell hdfs dfs -ls /user;
shell ls -ltr;
shell echo I can echo from impala scripts >> /home/..../my_file.csv;
shell echo How are you? >> /home/..../my_file.csv;
shell echo I am feeling very good, how about you? >> /home/..../my_file.csv;
shell cat hex_color.sql

and don't forget the semicolon at the end of the line ";". I hope it helps.

来源：https://stackoverflow.com/questions/62174395/in-hql-scripts-we-use-sh-echo-new-line-for-the-same-want-to-know-the

标签

hql

impala