安裝方式:tar包安裝
二、準備
1、安裝java環境
aptitude install java7-jdk( aptitude install openjdk-7-jdk)
2、添加帳號
groupadd hbase
useradd -m hbase -g hbase -s /bin/bash -d /home/hbase
3、創建目錄
mkdir -p /vobiledata/HadoopData
chown-R hbase:hbase /vobiledata/HadoopData
4、安裝配置免密碼登錄
ssh-keygen -t rsa -P '' -f~/.ssh/id_rsa
cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
獲取所有機器pub key放到authorized_keys文件後再分發至所有機器
5、編輯/etc/hosts,添加集羣主機(假設是三臺)
192.168.40.201 debian1
192.168.40.200 debian2
192.168.40.197 debian3
三、Hadoop安裝流程
1、安裝
wget http://mirror.bit.edu.cn/apache/hadoop/common/hadoop-2.4.0/hadoop-2.4.0.tar.gz
tar zxvf hadoop-2.4.0.tar.gz
mv hadoop-2.4.0 /usr/local/hadoop
2、配置
cd /usr/local/hadoop/etc/hadoop/
修改hadoop-env.sh,添加
# The java implementation to use. export JAVA_HOME=/usr/lib/jvm/java-7-openjdk-amd64 export HADOOP_CONF_DIR=/usr/local/hadoop/etc/hadoop export PATH=$PATH:/usr/local/hadoop/bin
修改hdfs-site.xml
dfs.replication 3 dfs.namenode.name.dir /vobiledata/HadoopData/dfs/name dfs.datanode.data.dir /vobiledata/HadoopData/dfs/data hadoop.tmp.dir /vobiledata/HadoopData
配置core-site.xml
fs.default.name hdfs://debian1:9000
如果需要mapreduce需要以下配置
配置mapred-site.xml,設置Hadoop的執行框架:mapreduce.framework.name
mapreduce.framework.name yarn
配置yarn-site.xml
yarn.nodemanager.aux-services mapreduce_shuffle yarn.resourcemanager.hostname debian1
配置 masters, 加上要啓動datenode的機器主機名,例如localhost或者debian1
配置slaves,加上要啓動namenode的機器主機名,例如僞分佈式就用localhost,完全分佈式加上deibian1,debian2,debian3
PS:配置好後將配置文件傳到各臺機器
3、格式化hdfs
cd /usr/local/hadoop
./bin/hadoop namenode -format
執行後看到sucessful才能說明格式化成功
4、啓動hadoop
cd /usr/local/hadoop/sbin
./start-dfs.sh(需要map-reduce的話還需要啓動start-yarn.sh)
三、安裝zookeeper流程
1、安裝
wget http://mirror.bit.edu.cn/apache//zookeeper/zookeeper-3.4.6/zookeeper-3.4.6.tar.gz
tar -xzvf zookeeper-3.4.6.tar.gz
mv zookeeper-3.4.6 /usr/local/zookeeper
2、配置
vi .bashrc 加入
export ZOOKEEPER_HOME=/usr/local/zookeeper
PATH=$PATH:$ZOOKEEPER_HOME/bin
cd/usr/local/zookeeper/conf
修改zoo.cfg
dataDir=/vobiledata/HadoopData/zookeeper
PS:配置好後將配置文件傳到各臺機器
3、啓動
cd /usr/local/zookeeper/bin
./zkServer.sh start
四、hbase 安裝流程
1、安裝
wget http://mirror.bit.edu.cn/apache/hbase/hbase-0.96.2/hbase-0.96.2-hadoop2-bin.tar.gz
tar -xzvf hbase-0.96.2-hadoop2-bin.tar.gz
mv hbase-0.96.2-hadoop2 /usr/local/hbase
2、配置
cd /usr/local/hbase/conf
配置hbase-env.sh,加入以下配置
export JAVA_HOME=/usr/lib/jvm/java-7-openjdk-amd64 export HBASE_HOME=/usr/local/hbase/ export HADOOP_HOME=/usr/local/hadoop/ export HBASE_MANAGES_ZK=false --(需要用到hbase複製的話,就必須獨立安裝zookeeper) export PATH=$!PATH:/usr/local/hadoop/bin
配置hbase-site.xml
hbase.rootdir hdfs://debian1:9000/hbase hbase.zookeeper.property.dataDir /vobiledata/HadoopData/zookeeper hbase.zookeeper.quorum debian1,debian2,debian3 hbase.cluster.distributed true hbase.master.port 60000 hbase.replication true --- hbase複製需要配置
配置regionservers,加上要啓動regionservers的機器主機名,例如僞分佈式就用localhost,完全分佈式加上deibian1,debian2,debian3
PS:配置好後將配置文件傳到各臺機器
3、啓動hbase
cd /usr/local/hbase
./bin/start-hbase.sh
查看所有進程是否啓動
jps
14095 HMaster
14215 HRegionServer
13836 SecondaryNameNode
5690 Jps
13687 DataNode
22618 QuorumPeerMain
13573 NameNode
五、Hbase複製配置(可選)
1、確保各機器hbase-site.xml中已配置了
hbase.replication true
2、執行add_peer '', "slave.zookeeper.quorum:zookeeper.clientport.:zookeeper.znode.parent"
例如 add_peer '1', "debian1,debian2,debian3:2181:/hbase"
where is the peer ID; it should not be more than two characters(longer IDs may work, but have not been tested).
slave.zookeeper.quorum 爲slave服務器上hbase配置文件中hbase-site.xml中hbase.zookeeper.quorum
zookeeper.clientport 默認爲2181
zookeeper.znode.parent 可直接用/hbase
執行list_peers可看是否加上
3、複製需要保證每張表的每個列簇都開啓複製選項
disable 'your_table'
alter 'your_table', {NAME => 'family_name', REPLICATION_SCOPE=> '1'}
enable 'your_table'
或者創建的時候直接指定REPLICATION_SCOPE => '1'
create 'test',{NAME => 't1', REPLICATION_SCOPE => '1'},{NAME=> 't2', REPLICATION_SCOPE => '1'}
4、插入數據,測試是否能夠複製
另,詳細的配置文檔也可參照:http://www.oschina.net/question/54100_24054