hadoop集羣安裝配置


1、環境介紹

(1)軟件版本
OS:rhel6.3-x86_64
JDK:jdk1.6.0_41
Hadoop:hadoop-1.1.1

(2)IP規劃
hserver  192.168.183.130 NameNode
hclient1 192.168.183.131 DataNode
hclient2 192.168.183.132 DataNode

2、清空iptables

# iptables -F
# /etc/init.d/iptables stop
# chkconfig iptables off

3、配置host解析文件

# vim /etc/hosts
192.168.183.130 hserver
192.168.183.131 hclient1
192.168.183.132 hclient2

4、設置專門運行hadoop的用戶

# useradd hadoop -d /hadoop
# echo "redhat" |passwd hadoop --stdin

5、配置hadoop用戶的SSH無密登陸
$ ssh-keygen(一路回車)
$ ssh-copy-id -i .ssh/id_rsa.pub [hostname](回答yes,輸入密碼)

6、安裝JDK
$ chmod +x jdk-6u41-linux-x64.bin
$ ./jdk-6u41-linux-x64.bin

7、安裝 Hadoop
下載鏈接:http://archive.apache.org/dist/hadoop/core/hadoop-1.1.1/hadoop-1.1.1.tar.gz
解壓安裝:$ tar zvxf hadoop-1.1.1.tar.gz
hadoop安裝目錄:/hadoop/hadoop-1.1.1

8、配置 Hadoop

修改配置文件(hadoop-env.sh、core-site.xml、hdfs-site.xml、mapred-site.xml、masters、slaves)

(1)$ vim hadoop-env.sh

export JAVA_HOME=/hadoop/jdk1.6.0_41

(2)$ vim core-site.xml

<configuration>
  <property>
    <name>fs.default.name</name>
    <value>hdfs://hserver:9000</value>
  </property>
  <property>
    <name>hadoop.tmp.dir</name>
    <value>/hadoop/hadoop</value>
  </property>
</configuration>

(3)$ vim hdfs-site.xml

<configuration>
  <property>
    <name>dfs.replication</name>
    <value>2</value>
  </property>
</configuration>

(4)$ vim mapred-site.xml

<configuration>
  <property>
    <name>mapred.job.tracker</name>
    <value>hserver:9001</value>
  </property>
</configuration>

(5)$ vim masters

hserver

(6)$ vim slaves

hclient1
hclient2

9、格式化NameNode(hserver節點)

$ bin/hadoop namenode -format

10、啓動 Hadoop

(1)NameNode(hserver節點)上啓動守護進程

$ bin/start-all.sh

(2)jps檢查守護進程啓動情況

NameNode(hserver節點)

4389 JobTracker
4116 NameNode
4297 SecondaryNameNode
10436 Jps

DataNode(hclient1節點)

6794 Jps
2784 DataNode
2894 TaskTracker


11、操作Hadoop

(1)Web方式
NameNode 訪問地址:http://hserver:50070
JobTracker 訪問地址:http://hserver:50030

(2)Command方式
$ hadoop dfs -ls /
Found 1 items
drwxr-xr-x   - hadoop supergroup          0 2013-03-07 04:17 /hadoop



發表評論
所有評論
還沒有人評論,想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章