sqoop2

来源： https://stackoverflow.com/questions/55294932/sqoop-mysql-error-communications-link-failure

来源： https://stackoverflow.com/questions/55294932/sqoop-mysql-error-communications-link-failure

问题 I'm currently importing postgres data to hdfs. I'm planning to move the storage from hdfs to S3. When i'm trying to provide S3 location, the sqoop job is failing. I'm running it on EMR(emr-5.27.0) cluster and I've read/write access to that s3 bucket from all nodes in the cluster. sqoop import \ --connect "jdbc:postgresql://<machine_ip>:<port>/<database>?sslfactory=org.postgresql.ssl.NonValidatingFactory&ssl=true" \ --username <username> \ --password-file <password_file_path> \ --table

问题 I'm currently importing postgres data to hdfs. I'm planning to move the storage from hdfs to S3. When i'm trying to provide S3 location, the sqoop job is failing. I'm running it on EMR(emr-5.27.0) cluster and I've read/write access to that s3 bucket from all nodes in the cluster. sqoop import \ --connect "jdbc:postgresql://<machine_ip>:<port>/<database>?sslfactory=org.postgresql.ssl.NonValidatingFactory&ssl=true" \ --username <username> \ --password-file <password_file_path> \ --table

问题 I'm trying to install sqoop 2 (version 1.99.3) on an Amazon EMR cluster (AMI version 3.2.0 / Hadoop version 2.4.0). When I start the sqoop server, I see this error in localhost.log: Sep 10, 2014 4:55:56 PM org.apache.catalina.core.StandardContext listenerStart SEVERE: Exception sending context initialized event to listener instance of class org.apache.sqoop.server.ServerInitializer java.lang.RuntimeException: Failure in server initialization at org.apache.sqoop.core.SqoopServer.initialize

问题 I am new in word of hadoop and sqoop. I installed hadoop 2.7.3 (pseudo mode) and its working fine on my system. I want integration with sqoop. I am using sqoop sqoop-1.99.7-bin-hadoop200. 1) I extract tar file and move extracted content into /usr/local/sqoop 2) Set Sqoop path into .bashrc file. 3) go to /usr/local/sqoop/server/lib/sqoop.sh server start and get following error message. hadoop_usr@sawai-Lenovo-G580:/usr/local/sqoop/server/lib$ sqoop.sh server start Setting conf dir: /usr/local

问题 I am using sqoop to import Oracle data to HDFS directory. I have created the sqoop job for the same. I have used follwoing command to create sqoop job- sqoop job --create TABLE_NAME -- import --connect jdbc:oracle:thin:/system@HOST_NAME:PORT:SERVICE --username USERNAME --password-file /MYPASSWORD.txt --fields-terminated-by ',' --enclosed-by '"' --table USERNAME.TABLE_NAME --target-dir /TABLE_NAME/$(date --date "-1 days" +%F)/ -m 1 --incremental append --check-column DATE_COLUMN --last-value

问题 I am trying to import dataset from Vertica to HDFS using sqoop2. I a running following query on sqoop machines to import data into hdfs from Vertica v6.0.1-7 sqoop import -m 1 --driver com.vertica.jdbc.Driver --connect "jdbc:vertica://10.10.10.10:5433/MYDB" --password dbpassword --username dbusername --target-dir "/user/my/hdfs/dir" --verbose --query 'SELECT * FROM ORDER_V2 LIMIT 10;' but i am getting some error here, 16/02/03 10:33:17 ERROR tool.ImportTool: Encountered IOException running

问题 I'm working on testing out using Hadoop with the latest version of Sqoop2 (1.99.7), and when running the sqoop2-server, I get the following error: Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/conf/Configuration at org.apache.sqoop.security.authentication.SimpleAuthenticationHandler.secureLogin(SimpleAuthenticationHandler.java:36) at org.apache.sqoop.security.AuthenticationManager.initialize(AuthenticationManager.java:98) at org.apache.sqoop.core.SqoopServer

问题 So with mapreduce v2 you can use binding to certain YARN queues to manage resources and prioritization. Basically by using "hadoop jar /xyz.jar -D mapreduce.job.queuename=QUEUE1 /input /output" which works perfectly. How can integrate Yarn queue binding with Sqoop when you run a sqoop query? ie. sqoop import \ --connect 'jdbc://server' \ --target-dir \ and what ? 回答1: Use the same method for Sqoop as well i.e sqoop import -Dmapreduce.job.queuename=NameOfTheQueue\ --connect 'jdbc://server' \ -