问题
when I run the following sqoop merge import to update an existing Hive table that I already have:
sudo sqoop import \
--connect 'jdbc:sqlserver://1.1.1.1\test_server;database=Training' \
--username Training_user --password Training_user \
--table BigDataTest -m 1 \
--check-column lastmodified \
--merge-key id \
--incremental lastmodified \
--compression-codec=snappy \
--as-parquetfile \
--target-dir /user/hive/warehouse \
--hive-table bigDataTest \
--last-value '2019-05-06 15:07:49.917'
I get this
.
.
.
19/05/09 11:00:50 INFO tool.ImportTool: Final destination exists, will run merge job.
19/05/09 11:00:50 ERROR tool.ImportTool: Import failed: java.io.IOException: Could not load jar /tmp/sqoop-root/compile/e913f7c459cf4e1cdb8a8d5802f1dac2/codegen_BigDataTest.jar into JVM. (Could not find class BigDataTest.)
at org.apache.sqoop.util.ClassLoaderStack.addJarFile(ClassLoaderStack.java:92)
at com.cloudera.sqoop.util.ClassLoaderStack.addJarFile(ClassLoaderStack.java:36)
at org.apache.sqoop.tool.ImportTool.loadJars(ImportTool.java:120)
at org.apache.sqoop.tool.ImportTool.lastModifiedMerge(ImportTool.java:456)
at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:522)
at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:621)
at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243)
at org.apache.sqoop.Sqoop.main(Sqoop.java:252)
Caused by: java.lang.ClassNotFoundException: BigDataTest
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:789)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:270)
at org.apache.sqoop.util.ClassLoaderStack.addJarFile(ClassLoaderStack.java:88)
... 11 more
what does "ERROR tool.ImportTool: Import failed: java.io.IOException: Could not load jar" mean? and how I can fix it ?
回答1:
Why codegen_BigDataTest.jar is generated in your import try to check that and extract the jar codegen_BigDataTest.jar to check the class generated which could be codegen_BigDataTest.class?
java.io.IOException: Could not load jar /tmp/sqoop-root/compile/e913f7c459cf4e1cdb8a8d5802f1dac2/codegen_BigDataTest.jar into JVM. (Could not find class BigDataTest.)
I wrote the below after checking my class name in generated jar file which was not same as the jar generated that caused error in my run.
**Sample command:**
**<sqoop import>:**
sqoop import --connect jdbc:oracle:thin:@hostname:port:db --username test --password test123 --table TEST --incremental lastmodified --split-by COLUMN_NAME --merge-key COLUMN_NAME --check-column MODIFIED_DATE --target-dir /sqoop1/TEST
**<sqoop create import job>:**
sqoop job --create job_test01 -- import --connect jdbc:oracle:thin:@hostname:port:db --username test --password test123 --table TEST --incremental lastmodified --split-by COLUMN_NAME --merge-key COLUMN_NAME --check-column MODIFIED_DATE --target-dir /sqoop1/TEST
**<sqoop execute import job>:** This can be executed whenever you need to import data and can be scheduled for periodic execution.
sqoop job --exec job_test01
**<sqoop show import job parameter value>**: Below is shown the output of few top lines but it has very detailed information of your job.
sqoop job --show job_test01
...
...
19/08/11 02:50:28 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6-cdh5.10.2
Enter password:
Job: job_test01
Tool: import
Options:
----------------------------
verbose = false
hcatalog.drop.and.create.table = false
incremental.last.value = 2019-08-11 02:27:43.0
Here if you see sqoop job maintains the last.value in own metadata store which is used in next incremental run. You need not to take care of last.value when you create a sqoop job.
来源：https://stackoverflow.com/questions/56088078/error-tool-importtool-import-failed-java-io-ioexception-could-not-load-jar-t