我们访问namenode机器，namenode会给我们数据所在datanode的ip地址供我们访问数据传输服务,
但是返回的的是datanode内网的ip,我们无法根据该IP访问datanode服务器.
报错如下
2018-06-06 17:01:44,555 [main] WARN [org.apache.hadoop.hdfs.BlockReaderFactory] - I/O error constructing remote block reader.java.net.ConnectException: Connection timed out: no further informationat sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:531)at org.apache.hadoop.hdfs.DFSClient.newConnectedPeer(DFSClient.java:3450)at org.apache.hadoop.hdfs.BlockReaderFactory.nextTcpPeer(BlockReaderFactory.java:777)at org.apache.hadoop.hdfs.BlockReaderFactory.getRemoteBlockReaderFromTcp(BlockReaderFactory.java:694)at org.apache.hadoop.hdfs.BlockReaderFactory.build(BlockReaderFactory.java:355)at org.apache.hadoop.hdfs.DFSInputStream.blockSeekTo(DFSInputStream.java:665)at org.apache.hadoop.hdfs.DFSInputStream.readWithStrategy(DFSInputStream.java:874)at org.apache.hadoop.hdfs.DFSInputStream.read(DFSInputStream.java:926)at java.io.DataInputStream.read(DataInputStream.java:149)at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:284)at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:326)at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178)at java.io.InputStreamReader.read(InputStreamReader.java:184)at java.io.BufferedReader.fill(BufferedReader.java:161)at java.io.BufferedReader.readLine(BufferedReader.java:324)at java.io.BufferedReader.readLine(BufferedReader.java:389)at com.feiyangshop.recommendation.HdfsHandler.main(HdfsHandler.java:36)2018-06-06 17:01:44,560 [main] WARN [org.apache.hadoop.hdfs.DFSClient] - Failed to connect to /192.168.1.219:50010 for block, add to deadNodes and continue、java.net.ConnectException: Connection timed out: no further informationjava.net.ConnectException: Connection timed out: no further informationat sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:531)at org.apache.hadoop.hdfs.DFSClient.newConnectedPeer(DFSClient.java:3450)at org.apache.hadoop.hdfs.BlockReaderFactory.nextTcpPeer(BlockReaderFactory.java:777)
为了能够让开发机器访问到hdfs,我们可以让namenode返回给我们datanode的域名
在开发机器的hosts文件中配置datanode对应的外网ip和域名,并且在与hdfs交互的程序中添加如下代码即可
import org.apache.hadoop.conf.Configuration;Configuration conf = new Configuration();//设置通过域名访问datanodeconf.set("dfs.client.use.datanode.hostname", "true");