集羣擴展：Hadoop環境搭建

hadoop安裝步驟:

1. 系統需要安裝的包
   ssh，rpm，gcc，make
   yum -y install openssh-server openssh-clients

2. 檢測ssh安裝和rpm安裝
   a. ssh localhost
   b. yum install rpm

3. 安裝java
   a. 安裝java-7
   rpm -Uvh java

   b. 添加並配置新的jdk系統命令

   update-alternatives –install “/usr/bin/java” “java” “/usr/java/jdk1.7.0_45/bin/java” 16001
   update-alternatives –install “/usr/bin/java” “javac” “/usr/java/jdk1.7.0_45/bin/javac” 16001
   alternatives –config java
   alternatives –config javac
   c. 檢查並測試
   update-alternatives –display java
   java -version

4. 添加hduser用戶
   a. groupadd hdgroup
   b. useradd -g hdgroup hduser
   c. passwc hduser
   d. chown -R hduser .
   e. chmod 755 hduser
   f. chgrp -R hdgroup hduser

5. 將da-master上的公鑰拷貝到新機器上
   ssh-copy-id -i id_rsa.pub hadoop@cloud011

6. 修改主機名稱和hosts文件
   127.0.0.1 localhost
   172.21.36.4 da-slave05
   172.21.101.7 web2
   172.21.101.30 da-master
   172.21.101.31 da-slave01
   172.21.101.32 da-slave02
   172.21.101.33 da-slave03
   172.21.101.34 da-slave04
   ::1 localhost6.localdomain6 localhost6

7. 進入hduser用戶環境
   a. su - hduser
   b. tar -zxf hadoop.2.2.0.tar.gz
   c. ln -s hadoop-2.2.0 /

8. 編輯環境變量
   vim ~/.bahsrc

9. 修改系統參數
   a.關閉防火牆
   service iptables stop
   chkconfig iptables off
   vim /etc/selinux/config

   改成disabled

   setenforce 0
   service iptables status

   b.修改最大文件打開數
   1）vim /etc/security/limits.conf

   • • nofile 204800
       2）vim /etc/security/limits.d/90-nproc.conf
     • soft nproc new-max-user-processes-value

   c.調整交換分區大小
   vim /etc/sysctl.conf
   vm.swappiness = 0
   vm.dirty_ratio = 5

   1. 安裝LZO包
      cd lzo-2.06
      ./configure –enable-shared
      make
      make install
      #編輯/etc/ld.so.conf，加入/usr/local/lib/後，執行/sbin/ldconfig
      或者cp /usr/local/lib/liblzo2.* /usr/lib64/

   10.將master上的數據庫配置參數文件拷貝到該機器上來

   11.同步時間： http://www.cnblogs.com/panfeng412/p/drop-caches-under-linux-system.html
   http://blog.sina.com.cn/s/blog_79fded950101gfc7.html

介紹得比較簡略，待有時間貼上圖文演示，敬請期待！