Default Namenode port of HDFS is 50070.But I have come across at some places 8020 or 9000

Question

When I setup the hadoop cluster, I read the namenode runs on 50070 and I set up accordingly and it\'s running fine.

But in some books I have come across name node ad

Answer 1

9000 is the default HDFS service port.This does not have a web UI.50070 is the default NameNode web UI port (Although, in hadoop 3.0 onwards 50070 is updated to 9870)

Answer 2

You can check what ports each daemon is listening on if you’re having trouble finding the web interface. For example, to check ports the NameNode is listening on:

lsof -Pan -iTCP -sTCP:LISTEN -p `jps | grep "\sNameNode" | cut -d " " -f1`

This will give you output similar to

COMMAND  PID USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
java    4053   pi  275u  IPv6  45474      0t0  TCP *:9870 (LISTEN)
java    4053   pi  288u  IPv6  43476      0t0  TCP 127.0.0.1:9000 (LISTEN)

As you can see *:9870 is listed, which is the web interface.

---

Explanation:

• lsof -Pan -iTCP -sTCP:LISTEN -p <pid> lists all network files with TCP state LISTEN. -p filters the list by process id. So by plugging in a process id after this command, you can see all the ports a process is listening on.
• jps | grep "\sNameNode" | cut -d " " -f1 gets the process id of the NameNode.

Answer 3

That is because default is different for different hadoop configurations and distributions. We can always configure port by changing fs.default.name or fs.defaultFS properties as below in core-site.xml

<configuration>
 <property>
     <name>fs.default.name</name>
     <value>hdfs://localhost:9000</value>
 </property>
</configuration>

For Hadoop 1.0.4 if I dont mention port number like below

<value>hdfs://localhost</value>

then default port taken is 8020. But for some of the version like .20 i read it is 9000. So it is dependent on the version of hadoop you are using.

But all the configuration and distributation are using 50070 as standard port number for HDFS ui.

Answer 4

The default Hadoop ports are as follows: (HTTP ports, they have WEB UI):

Daemon                   Default Port  Configuration Parameter
-----------------------  ------------ ----------------------------------
Namenode                 50070        dfs.http.address
Datanodes                50075        dfs.datanode.http.address
Secondarynamenode        50090        dfs.secondary.http.address
Backup/Checkpoint node?  50105        dfs.backup.http.address
Jobracker                50030        mapred.job.tracker.http.address
Tasktrackers             50060        mapred.task.tracker.http.address

Internally, Hadoop mostly uses Hadoop IPC, which stands for Inter Process Communicator, to communicate amongst servers. The following table presents the ports and protocols that Hadoop uses. This table does not include the HTTP ports mentioned above.

Daemon      Default Port        Configuration Parameter     
------------------------------------------------------------
Namenode    8020                fs.default.name         
Datanode    50010               dfs.datanode.address        
Datanode    50020               dfs.datanode.ipc.address                                    
Backupnode  50100               dfs.backup.address          

check out this link For more info: http://blog.cloudera.com/blog/2009/08/hadoop-default-ports-quick-reference/

Answer 5

50070 is the default UI port for namenode . while 8020/9000 is the Inter Process Communicator port (IPC) for namenode.

Reference to IPC port : https://en.wikipedia.org/wiki/Inter-process_communication

Answer 6

To access Hadoop WEB UI , you need to type http://localhost:50075/ though your core-site.xml is having http://localhost:9000 because it is for hdfs requests and 50075 is the default port for WEB UI.