Hive基於PostgreSQL安裝與配置

Hive基於PostgreSQL安裝與配置

軟件依賴與版本號

安裝Hive的前提條件是已經部署了Hadoop和PostgreSQL。具體安裝Hadoop的方法見前面發的文章Hadoop僞分佈式安裝，安裝PostgreSQL的方法見前面發的文章PostgreSQL學習筆記（一）：安裝篇。部署軟件版本號如下所示：

軟件名稱	版本號
Hadoop	2.9.2
Hive	2.3.6
PostgreSQL	12.0.0

安裝步驟

• 第一步：下載安裝包apache-hive-2.3.6-bin.tar.gz並上傳到服務器；

• 第二步：解壓縮後，將安裝文件拷貝到/usr/local/目錄下。

  tar xzvf apache-hive-2.3.6-bin.tar.gz
  mv apache-hive-2.3.6-bin /usr/local/hive
  

• 第三步：打開/etc/profile，配置環境變量。

  vi /etc/profile
  

  輸入以下內容：

  export HIVE_HOME=/usr/local/hive
  export PATH=$HIVE_HOME/bin:$PATH
  

  保存後生效：

  source /etc/profile
  

• 第四步：配置PostgreSQL作爲元數據庫

  • 進入/usr/local/hive/conf/目錄，執行如下操作

    cp hive-default.xml.template hive-site.xml
    vi hive-site.xml
    

  • 找到如下內容的name並修改對應的value

    <property>
        <name>javax.jdo.option.ConnectionURL</name>
        <value>jdbc:postgresql://{hostname}:5432/{hivedatabase}?createDatabaseIfNotExist=true</value>
    </property>
    <property>
        <name>javax.jdo.option.ConnectionDriverName</name>
        <value>org.postgresql.Driver</value>
        <description>Driver class name for a JDBC metastore</description>
    </property>
    <property>
        <name>javax.jdo.option.ConnectionPassword</name>
        <value>{hivepassword}</value>
        <description>password to use against metastore database</description>
    </property>
    <property>
        <name>javax.jdo.option.ConnectionUserName</name>
        <value>{hiveusername}</value>
        <description>Username to use against metastore database</description>
    </property>
    <property>
        <name>hive.metastore.schema.verification</name>
        <value>false</value>
        <description>
          Enforce metastore schema version consistency.
          True: Verify that version information stored in is compatible with one from Hive jars.  Also disable automatic
                schema migration attempt. Users are required to manually migrate schema after Hive upgrade which ensures
                proper metastore schema migration. (Default)
          False: Warn if the version information stored in metastore doesn't match with one from in Hive jars.
        </description>
    </property>
    

  • 調整臨時目錄

    <property>
        <name>hive.exec.local.scratchdir</name>
        <value>/usr/local/hive/tmp</value>
        <description>Local scratch space for Hive jobs</description>
    </property>
    <property>
        <name>hive.downloaded.resources.dir</name>
        <value>/usr/local/hive/tmp/resources</value>
        <description>Temporary local directory for added resources in the remote file system.</description>
    </property>
    <property>
        <name>hive.querylog.location</name>
        <value>/usr/local/hive/tmp</value>
        <description>Location of Hive run time structured log file</description>
    </property>
    <property>
        <name>hive.server2.logging.operation.log.location</name>
        <value>/usr/local/hive/tmp/operation_logs</value>
        <description>Top level directory where operation logs are stored if logging functionality is enabled</description>
    </property>
    

• 第五步：使用schematool 初始化metastore的schema

  schematool -dbType postgres -initSchema
  

• 第六步：啓動hive

  # hive