centos6.8部署cloudera-manager

 

環境

centos 6.8

cm6.2

cdh6.2

jdk1.8

機器 node1   node2   node3

1、下載安裝包

下載地址https://archive.cloudera.com/cm6/6.2.0/redhat6/yum/RPMS/x86_64/

cm6.2的包

cdh6.2的包https://archive.cloudera.com/cdh6/6.2.0/parcels/

2、安裝前準備

以下每臺都執行

2.1.0、jdk 需要安裝到/usr/java/下

我用的就是yum本地源安裝的，或者rpm安裝，自動安裝到/usr/java下

這一步可以等到後面配置好yum本地源之後執行下面命令

yum install oracle-j2sdk1.8

2.1.1、ssh免密互信登陸

#每臺機器都執行
ssh-keygen -t rsa -P '' -f ~/.ssh/id_rsa 直接生成
 
# 我的是3臺機器，每臺機器都執行
ssh-copy-id -i node1
ssh-copy-id -i node2
ssh-copy-id -i node3

2.1.2、hosts主機名映射

# 每臺都執行
vim /etc/hosts

192.168.88.151 node1
192.168.88.152 node2
192.168.88.153 node3

2.1.3、時間同步,這個需要安裝，因爲cm監控會檢查

https://blog.csdn.net/qq_35315256/article/details/88684154（我的另外一篇博客）

2.1.4、普通用戶的sudu免密（root不需要）

chmod +w /etc/sudoers
vim /etc/sudoers
wangyf ALL=(ALL) NOPASSWD: ALL
chmod -w /etc/sudoers

2.1.5、防火牆關閉

service iptables stop 
chkconfig iptables off

2.1.6、禁用透明頁

echo never > /sys/kernel/mm/transparent_hugepage/defrag
echo never > /sys/kernel/mm/transparent_hugepage/enabled

vim /etc/rc.local #這是centos 6的寫法
加入上面兩句話

2.1.7、關閉SELinux

vim /etc/selinux/config
disabled 就可以
setenforce 0 # 臨時生效

2.1.8、swap分區優化

echo "vm.swappiness = 10" >> /etc/sysctl.conf
sysctl -p

2.1.9、如果用的mysql庫，驅動放到這，每臺機器都放

/usr/share/java/ 
mv mysql-connector-java-5.1.47.jar mysql-connector-java.jar

2.1.10、查看這些依賴是否缺少

yum install -y chkconfig python bind-utils psmisc libxslt zlib sqlite cyrus-sasl-plain cyrus-sasl-gssapi fuse fuse-libs redhat-lsb

2.1.11、安裝python2.7，我是yum在線安裝的

 

2.2、配置yum內部存儲庫

2.2.1、安裝http服務，我是選擇的node3這臺機器

yum  install  httpd
service  httpd  start
chkconfig  httpd  on 

2.2.2、安裝createrepo服務，(node3這臺機器)

yum -y install createrepo

2.2.3、如下圖操作，createrepo生成xml文件

2.2.4、測試一下

2.2.5、配置yum源，在node1上

cd  /etc/yum.repos.d/

vim  cloudera-repo.repo

[cloudera-repo]
name=cloudera-repo,version=6.2
baseurl=http://node3/cm6.2/
gpgcheck=1

執行命令

yum clean all
yum makecache
yum repolist 

3、安裝cloudera-manager6.2（在node1）

yum install cloudera-manager-daemons cloudera-manager-agent cloudera-manager-server

如果報密鑰沒有被安裝，就rpm --import 一下密鑰，密匙在官網對應版本

sudo rpm --import https://archive.cloudera.com/cm6/6.2.1/redhat6/yum/RPM-GPG-KEY-cloudera

官網文檔：https://docs.cloudera.com/documentation/enterprise/6/6.2/topics/configure_cm_repo.html

4、配置mysql數據庫

    CREATE DATABASE scm DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
    GRANT ALL ON scm.* TO 'scm'@'%' IDENTIFIED BY 'scm@123';
    CREATE DATABASE amon DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
    GRANT ALL ON amon.* TO 'amon'@'%' IDENTIFIED BY 'amon@123';
    CREATE DATABASE rman DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
    GRANT ALL ON rman.* TO 'rman'@'%' IDENTIFIED BY 'rman@123';
    CREATE DATABASE hue DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
    GRANT ALL ON hue.* TO 'hue'@'%' IDENTIFIED BY 'hue@123';
    CREATE DATABASE metastore DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
    GRANT ALL ON metastore.* TO 'metastore'@'%' IDENTIFIED BY 'metastore@123';
    CREATE DATABASE sentry DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
    GRANT ALL ON sentry.* TO 'sentry'@'%' IDENTIFIED BY 'sentry@123';
    CREATE DATABASE nav DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
    GRANT ALL ON nav.* TO 'nav'@'%' IDENTIFIED BY 'nav@123';
    CREATE DATABASE navms DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
    GRANT ALL ON navms.* TO 'navms'@'%' IDENTIFIED BY 'navms@123';
    CREATE DATABASE oozie DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
    GRANT ALL ON oozie.* TO 'oozie'@'%' IDENTIFIED BY 'oozie@123';
    CREATE DATABASE hive DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
    GRANT ALL ON hive.* TO 'hive'@'%' IDENTIFIED BY 'hive@123';
    flush privileges;

執行腳本 ,生成cm元數據

/opt/cloudera/cm/schema/scm_prepare_database.sh -h 127.0.0.1 -P 3306 mysql scm scm scm@123

5、配置parcel包

文章最一開始的cdh6.2的截圖包全部放到/opt/cloudera/parcel-repo/下

把.sha文件的字符串替換爲.json文件最後面的那個hash值

"hash": "5dc82b17dbdbb8ffd9bcd05e98358c111f2c0f12"

6、啓動

service cloudera-scm-server start

查看日誌

tailf /var/log/cloudera-scm-server/cloudera-scm-server.log

訪問：192.168.88.151:7180

7、配置嚮導

 

 

我的最後一個警告沒解決，也沒啥影響，其他參考上面準備環節都可以解決。

跟着一步一步的來，安裝工作完成。