Hdfs datanode啓動問題

場景:

項目組6個節點中,某一節點硬盤壽命到了,由於當時硬盤做的raid0,沒有備份,更換新硬盤後,啓動datanode遇到如下問題:

1. 更換硬盤前,停止datanode,停止在此節點上應用;

2. 停止機器,更換硬盤

3. 重啓機器,重啓datanode,遇到問題

3.1 

WARN org.apache.hadoop.hdfs.server.datanode.DataNode: Invalid dfs.datanode.data.dir /data :

WARN org.apache.hadoop.hdfs.server.datanode.DataNode: Invalid dfs.datanode.data.dir /data :
EPERM: Operation not permitted
        at org.apache.hadoop.io.nativeio.NativeIO$POSIX.chmodImpl(Native Method)
        at org.apache.hadoop.io.nativeio.NativeIO$POSIX.chmod(NativeIO.java:230)
        at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:729)
        at org.apache.hadoop.fs.ChecksumFileSystem$1.apply(ChecksumFileSystem.java:505)
        at org.apache.hadoop.fs.ChecksumFileSystem$FsOperation.run(ChecksumFileSystem.java:486)
        at org.apache.hadoop.fs.ChecksumFileSystem.setPermission(ChecksumFileSystem.java:502)
        at org.apache.hadoop.util.DiskChecker.mkdirsWithExistsAndPermissionCheck(DiskChecker.java:140)
        at org.apache.hadoop.util.DiskChecker.checkDir(DiskChecker.java:156)
        at org.apache.hadoop.hdfs.server.datanode.DataNode$DataNodeDiskChecker.checkDir(DataNode.java:2345)
        at org.apache.hadoop.hdfs.server.datanode.DataNode.checkStorageLocations(DataNode.java:2387)
        at org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:2369)
        at org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:2261)
        at org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:2308)
        at org.apache.hadoop.hdfs.server.datanode.DataNode.secureMain(DataNode.java:2485)
        at org.apache.hadoop.hdfs.server.datanode.DataNode.main(DataNode.java:2509)
2019-03-05 21:01:10,707 FATAL org.apache.hadoop.hdfs.server.datanode.DataNode: Exception in secureMain
java.io.IOException: All directories in dfs.datanode.data.dir are invalid: "/data/"
        at org.apache.hadoop.hdfs.server.datanode.DataNode.checkStorageLocations(DataNode.java:2396)
        at org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:2369)
        at org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:2261)
        at org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:2308)
        at org.apache.hadoop.hdfs.server.datanode.DataNode.secureMain(DataNode.java:2485)
        at org.apache.hadoop.hdfs.server.datanode.DataNode.main(DataNode.java:2509)
2019-03-05 21:01:10,708 INFO org.apache.hadoop.util.ExitUtil: Exiting with status 1

原因:

由於更換硬盤,針對/data目錄的權限變成了root,需要對當前用戶賦予權限

解決辦法:

chown -R hadoop:hadoop /data

 

3.2 

2019-03-05 21:25:47,891 FATAL org.apache.hadoop.hdfs.server.datanode.DataNode: Exception in secureMain
java.io.IOException: failed to stat a path component: '/var/run/hdfs-sockets'

2019-03-05 21:25:47,891 FATAL org.apache.hadoop.hdfs.server.datanode.DataNode: Exception in secureMain
java.io.IOException: failed to stat a path component: '/var/run/hdfs-sockets'.  error code 2 (No such file or directory)
        at org.apache.hadoop.net.unix.DomainSocket.validateSocketPathSecurity0(Native Method)
        at org.apache.hadoop.net.unix.DomainSocket.bindAndListen(DomainSocket.java:189)
        at org.apache.hadoop.hdfs.net.DomainPeerServer.<init>(DomainPeerServer.java:40)
        at org.apache.hadoop.hdfs.server.datanode.DataNode.getDomainPeerServer(DataNode.java:946)
        at org.apache.hadoop.hdfs.server.datanode.DataNode.initDataXceiver(DataNode.java:912)
        at org.apache.hadoop.hdfs.server.datanode.DataNode.startDataNode(DataNode.java:1111)
        at org.apache.hadoop.hdfs.server.datanode.DataNode.<init>(DataNode.java:429)
        at org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:2374)
        at org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:2261)
        at org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:2308)
        at org.apache.hadoop.hdfs.server.datanode.DataNode.secureMain(DataNode.java:2485)
        at org.apache.hadoop.hdfs.server.datanode.DataNode.main(DataNode.java:2509)
2019-03-05 21:25:47,893 INFO org.apache.hadoop.util.ExitUtil: Exiting with status 1
2019-03-05 21:25:47,894 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: SHUTDOWN_MSG:

原因:

沒有/var/run/hdfs-sockets 這個路徑

解決辦法:

創建這個目錄即可

發表評論
所有評論
還沒有人評論,想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章