Hadoop on Windows. YARN fails to start with java.lang.UnsatisfiedLinkError

Question

I have installed/configured Hadoop on windows hadoop-2.7.0

I could successfully start \"sbin\\start-dfs\" run command. DataNode and NameNode started. I could create

Answer 1

If you look at the history of the native method that is missing here you will see that it was added recently.

So this error means that you use a newer version of Hadoop, but your hadoop.dll is from an older version of Hadoop.

So either retrieving/building a newer hadoop.dll or downgrading Hadoop should avoid this problem.

For me downgrading to Hadoop 2.3.0 did the trick.

Answer 2

In my case it was an exception because Hadoop didn't find precompiled 'hadoop' DLL. I've put a path of the hadoop.dll's folder into PATH environment variable and it worked out.

The description of the exception you got is misleading, the original exception is thrown in java.lang.ClassLoader class: throw new UnsatisfiedLinkError("no " + name + " in java.library.path");

Answer 3

To me setting VM Argument -Djava.library.path=C:\winutils-master\hadoop-3.0.0 resolved the issue.

Answer 4

In my case I have the 3.1.1 Hadoop version and I changed bin folder with another one from the following address https://github.com/s911415/apache-hadoop-3.1.0-winutils. I managed to start yarn with this dll and one node (Yarn cluster with one node). You can find the settings which I followed here. Also you have to set-up the hdfs-site.xml like this:

<configuration>

<property> 
    <name>dfs.replication</name>
    <value>1</value>
</property> 

<property> 
    <name>dfs.namenode.name.dir</name> 
    <value>/hadoop-3.1.1/data/namenode</value>
</property>

<property> 
    <name>dfs.datanode.data.dir</name> 
    <value>/hadoop-3.1.1/data/datanode</value>
</property>

</configuration>

Answer 5

there should be a %HADOOP_HOME%\bin\hadoop.dll which contains the native method. %HADOOP_HOME%\bin should be in the path. If you built from source, make sure the hadoop.dll was built and placed.

Answer 6

In my case , there was the hadoop path added in path variable and it was giving out and error "Another instance of Derby has already booted"