hadoop2.2.0的基礎上配置hive0.12.0（支持mysql）

前提：已經安裝好了hadoop2.2.0，當然也可以是其他版本的hadoop，作者使用的是這個版本，並在vmware上配了一個master和3個slaver。配置方式參考：

http://blog.csdn.net/glgl2424/article/details/24868273

 

一、下載hive-0.12.0，並解壓到/usr/hive中

二、將hive-default.xml.template拷貝一份並重命名爲hive-site.xml，同樣，將hive-env.sh.template拷貝一份並重命名爲hive-env.sh

三、修改2個配置文件，env,sh主要是修改環境變量，而-site.xml主要是修改hive的元數據庫配置（本文以配置mysql作爲元數據）

關鍵配置如下：(-site.xml)

<property>
  <name>javax.jdo.option.ConnectionURL</name>
  <value>jdbc:mysql://localhost:3306/hive?createDatabaseIfNotExist=true</value>
  <description>JDBC connect string for a JDBC metastore</description>
</property>


<property>
  <name>javax.jdo.option.ConnectionDriverName</name>
  <value>com.mysql.jdbc.Driver</value>
  <description>Driver class name for a JDBC metastore</description>
</property>

<property>
  <name>javax.jdo.option.ConnectionUserName</name>
  <value>hive</value>
  <description>username to use against metastore database</description>
</property>


<property>
  <name>javax.jdo.option.ConnectionPassword</name>
  <value>hive</value>
  <description>password to use against metastore database</description>
</property>

<property>
  <name>hive.stats.dbclass</name>
  <value>jdbc:mysql</value>
  <description>The default database that stores temporary hive statistics.</description>
</property>

<property>
  <name>hive.stats.jdbcdriver</name>
  <value>com.mysql.jdbc.Driver</value>
  <description>The JDBC driver for the database that stores temporary hive statistics.</description>
</property>


<property>
  <name>hive.stats.dbconnectionstring</name>
  <value>jdbc:mysql://localhost:3306/hive_stats?useUnicode=true&amp;characterEncoding=latin1&amp;user=mysql_username&amp;password=mysql_password&amp;createDatabaseIfNotExist=true</value>
  <description>The default connection string for the database that stores temporary hive statistics.</description>
</property>

<property>
  <name>hive.server2.thrift.sasl.qop</name>
  <value>auth</value>
  <description>Sasl QOP value; Set it to one of following values to enable higher levels of
     protection for hive server2 communication with clients.
      "auth" - authentication only (default)
      "auth-int" - authentication plus integrity protection
      "auth-conf" - authentication plus integrity and confidentiality protection
     This is applicable only hive server2 is configured to use kerberos authentication.
 </description>
</property>

<property>
  <name>hive.metastore.schema.verification</name>
  <value>false</value>
   <description>
   Enforce metastore schema version consistency.
   True: Verify that version information stored in metastore matches with one from Hive jars.  Also disable automatic
         schema migration attempt. Users are required to manully migrate schema after Hive upgrade which ensures
         proper metastore schema migration. (Default)
   False: Warn if the version information stored in metastore doesn't match with one from in Hive jars.
   </description>
</property>

ps:hive在hadoop中的路徑可以默認不修改，env中java-home類似的配置可以省略，因爲hadoop時已經配置了

     HADOOP_HOME=/usr/hadoop
    export HIVE_CONF_DIR=/usr/hive/conf

四：安裝mysql並添加hive賬號（賬號密碼都是hive）

1、執行yum install mysql-connector-java命令，自動安裝mysql

2、將/usr/share/java/mysql-connector-java-5.1.17.jar拷貝到hive的lib目錄既可

3、添加用戶：

•create database hive 

•創建hive用戶,並授權

•grant all on hive.* to hive@'%'  identified by 'hive';  

•flush privileges;  

五、啓動hive，第一次啓動會報錯，後續就正確了

六、驗證：

1、hive是否可以創建數據庫和表

hive> create table myhive(id int ,name string);
OK
Time taken: 3.32 seconds
hive> show tables;
OK
myhive
Time taken: 0.261 seconds, Fetched: 1 row(s)

2、加載hadoop中的文件

//待補充

3、select和insert

//待補充

安裝Hive參考地址：

http://blog.csdn.net/vah101/article/details/18653543

http://blog.chinaunix.net/uid-451-id-3143781.html

http://www.cnblogs.com/ggjucheng/archive/2012/04/26/2471341.html

http://www.throwexcept.com/5/32/2026.html

常見問題：

1、hive啓動報錯：mysql驅動jar沒有放入hive/lib中 http://bbs.csdn.net/topics/390787105?page=1#post-397387341

2、debug：hive --hiveconf  hive.root.logger=DEBUG,console

3、hadoop用戶無法啓動hive，啓動報錯如下：

14/05/22 07:10:34 INFO hive.metastore: Trying to connect to metastore with URI thrift://localhost:9083

14/05/22 07:10:34 WARN hive.metastore: Failed to connect to the MetaStore Server...

org.apache.thrift.transport.TTransportException: java.net.ConnectException: Connection refused

解決辦法：進入/usr/hive/bin目錄下，執行./hive

[hadoop@Master bin]$ pwd
/usr/hive/bin
[hadoop@Master bin]$ ./hive

 而不是直接在根目錄下執行hive

[hadoop@Master ~]$ pwd
/home/hadoop
[hadoop@Master ~]$ hive

學習文章：

http://www.open-open.com/doc/view/8e07f484a8db411cb7964f52263811f8

http://www.blogjava.net/hengheng123456789/archive/2011/01/05/342354.html

 http://www.iteblog.com/archives/tag/hive的那些事

http://www.iteblog.com/archives/886 cj96 

http://www.iteblog.com/archives/851

 

配置參考文章：

http://www.throwexcept.com/5/32/2026.html

http://www.cnblogs.com/ggjucheng/archive/2012/04/26/2471341.html

 http://www.ibm.com/developerworks/cn/data/library/bd-hivelibrary/

 

 

注意點：

1、目前是把hive的目錄放到跟hadoop同級，即/usr/hive

2、hive包中conf目錄下有個默認的文件名叫hive-default.xml.template，我們要做的是拷貝一份，且命名爲hive-site.xml，這是很坑爹的，同時裏面有個<value>auth</auth>的錯誤

3、hive.metastore.warehouse.dir hive的表其實是存在hdfs的一個數據文件，並不是真正的表，表自身的信息稱爲元數據，表裏面的數據其實就是hdfs中的數據，所以需要指定hive在hdfs中的目錄

4、hive啓動報錯：參見：http://www.micmiu.com/bigdata/hive/hive-default-setup/

 

實戰：

http://www.open-open.com/lib/view/open1386742941845.html

 

數據：

http://www.ibm.com/developerworks/cn/data/library/bd-hivelibrary/

常見的mysql命令：

http://www.360doc.com/content/10/1125/16/4506941_72353321.shtml

hive命令：

http://blog.csdn.net/xiaoping8411/article/details/7605039 

hive數據導入：

http://www.iteblog.com/archives/949