hadoop eclipse plugin 編譯安裝,問題總結

插件名稱:hadoop2x-eclipse-plugin
插件地址:https://github.com/winghc/hadoop2x-eclipse-plugin

 
 1.下載並解壓hadoop2.x,下載地址http://hadoop.apache.org/releases.html#Download(我下載的是編譯好的包)
 2.下載並解壓eclipse(我的是4.4.1版本,其他的版本類似)
 3.下載hadoop2x-eclipse-plugin插件並解壓自己喜歡的目錄,爲了方便表達,我暫時叫他"H2EP_HOME"
 4.插件編譯需要ant工具,下載地址http://ant.apache.org/bindownload.cgi
    配置好ANT_HOME環境變量,指向ant解壓地址,配置PATH環境變量,增加%ANT_HOME%\bin(linux環境類似)
 5.打開命令行工具,進入“H2EP_HOME”目錄;
 6.執行ant jar -Dversion=2.x.x -Dhadoop.version=2.x.x -Declipse.home=/opt/eclipse -Dhadoop.home=/usr/share/hadoop
   eclipse.home配置成eclipse安裝目錄
   hadoop.home配置成hadoop的解壓目錄
   將2.x.x修改成對應的hadoop的版本號
  
 7.命令行在ivy-resolve-common處卡了
   原因是找不到幾個依賴包,那幾個依賴包可能是換路徑了,其實不需要這幾個依賴包也可以
   解決方案:
   修改"H2EP_HOME"\src\contrib\eclipse-plugin\build.xml
   找到:

<target name="compile" depends="init, ivy-retrieve-common" unless="skip.contrib">

    去掉depends修改爲

<target name="compile" unless="skip.contrib">

 8.再次執行第6步的編譯命令,會提示copy不到相關jar包的錯誤,
    解決方案:
    修改"H2EP_HOME"\ivy\libraries.properties文件,
    將報錯的jar包版本號跟換成與"HADOOP_HOME"\share\hadoop\common\lib下面jar對應的版本號
    此步可能會有多個jar包版本不匹配,需要多次修改
 9.再次執行第6步的編譯命令,執行成功
    在"H2EP_HOME"\build\contrib\eclipse-plugin下生成hadoop-eclipse-plugin-2.x.x.jar插件
 10.將hadoop-eclipse-plugin-2.x.x.jar放到eclipse的plugins目錄下,啓動eclipse
 11.打開window===>prefernces,找到Hadoop Map/Reduce選項卡
  配置hadoop installation directory目錄,指向hadoop的安裝目錄
 12.打開window====>show view====>other,找到Map/Reduce Locations,使其顯示
 13.在Map/Reduce Locations中右鍵=====>new hadoop locations,
  此時沒反應,查看eclipse日誌(工作空間\.metadata\.log),發現報錯:

java.lang.ClassNotFoundException: org.apache.commons.collections.map.UnmodifiableMap  

  解決方案:
  修改"H2EP_HOME"\src\contrib\eclipse-plugin\build.xml
  增加:

<copy file="${hadoop.home}/share/hadoop/common/lib/commons-collections-${commons-collections.version}.jar"  todir="${build.dir}/lib" verbose="true"/>

   <jar>標籤增加

lib/commons-collections-${commons-collections.version}.jar,

 14.執行 eclipse.exe -clean(清理一下緩存,不然有可能還是出現13步的問題)啓動eclipse


 完整的build.xml如下("H2EP_HOME"\src\contrib\eclipse-plugin\build.xml)

<?xml version="1.0" encoding="UTF-8" standalone="no"?>

<!--
   Licensed to the Apache Software Foundation (ASF) under one or more
   contributor license agreements.  See the NOTICE file distributed with
   this work for additional information regarding copyright ownership.
   The ASF licenses this file to You under the Apache License, Version 2.0
   (the "License"); you may not use this file except in compliance with
   the License.  You may obtain a copy of the License at

       http://www.apache.org/licenses/LICENSE-2.0

   Unless required by applicable law or agreed to in writing, software
   distributed under the License is distributed on an "AS IS" BASIS,
   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
   See the License for the specific language governing permissions and
   limitations under the License.
-->

<project default="jar" name="eclipse-plugin">

  <import file="../build-contrib.xml"/>

  <path id="eclipse-sdk-jars">
    <fileset dir="${eclipse.home}/plugins/">
      <include name="org.eclipse.ui*.jar"/>
      <include name="org.eclipse.jdt*.jar"/>
      <include name="org.eclipse.core*.jar"/>
      <include name="org.eclipse.equinox*.jar"/>
      <include name="org.eclipse.debug*.jar"/>
      <include name="org.eclipse.osgi*.jar"/>
      <include name="org.eclipse.swt*.jar"/>
      <include name="org.eclipse.jface*.jar"/>

      <include name="org.eclipse.team.cvs.ssh2*.jar"/>
      <include name="com.jcraft.jsch*.jar"/>
    </fileset> 
  </path>

  <path id="hadoop-sdk-jars">
    <fileset dir="${hadoop.home}/share/hadoop/mapreduce">
      <include name="hadoop*.jar"/>
    </fileset> 
    <fileset dir="${hadoop.home}/share/hadoop/hdfs">
      <include name="hadoop*.jar"/>
    </fileset> 
    <fileset dir="${hadoop.home}/share/hadoop/common">
      <include name="hadoop*.jar"/>
    </fileset> 
  </path>



  <!-- Override classpath to include Eclipse SDK jars -->
  <path id="classpath">
    <pathelement location="${build.classes}"/>
    <!--pathelement location="${hadoop.root}/build/classes"/-->
    <path refid="eclipse-sdk-jars"/>
    <path refid="hadoop-sdk-jars"/>
  </path>

  <!-- Skip building if eclipse.home is unset. -->
  <target name="check-contrib" unless="eclipse.home">
    <property name="skip.contrib" value="yes"/>
    <echo message="eclipse.home unset: skipping eclipse plugin"/>
  </target>

 <!--<target name="compile" depends="init, ivy-retrieve-common" unless="skip.contrib">-->
 <!-- 此處去掉 depends="init, ivy-retrieve-common" -->
 <target name="compile"  unless="skip.contrib">
    <echo message="contrib: ${name}"/>
    <javac
     encoding="${build.encoding}"
     srcdir="${src.dir}"
     includes="**/*.java"
     destdir="${build.classes}"
     debug="${javac.debug}"
     deprecation="${javac.deprecation}">
     <classpath refid="classpath"/>
    </javac>
  </target>

  <!-- Override jar target to specify manifest -->
  <target name="jar" depends="compile" unless="skip.contrib">
    <mkdir dir="${build.dir}/lib"/>
    <copy  todir="${build.dir}/lib/" verbose="true">
          <fileset dir="${hadoop.home}/share/hadoop/mapreduce">
           <include name="hadoop*.jar"/>
          </fileset>
    </copy>
    <copy  todir="${build.dir}/lib/" verbose="true">
          <fileset dir="${hadoop.home}/share/hadoop/common">
           <include name="hadoop*.jar"/>
          </fileset>
    </copy>
    <copy  todir="${build.dir}/lib/" verbose="true">
          <fileset dir="${hadoop.home}/share/hadoop/hdfs">
           <include name="hadoop*.jar"/>
          </fileset>
    </copy>
    <copy  todir="${build.dir}/lib/" verbose="true">
          <fileset dir="${hadoop.home}/share/hadoop/yarn">
           <include name="hadoop*.jar"/>
          </fileset>
    </copy>

    <copy  todir="${build.dir}/classes" verbose="true">
          <fileset dir="${root}/src/java">
           <include name="*.xml"/>
          </fileset>
    </copy>



    <copy file="${hadoop.home}/share/hadoop/common/lib/protobuf-java-${protobuf.version}.jar"  todir="${build.dir}/lib" verbose="true"/>
    <copy file="${hadoop.home}/share/hadoop/common/lib/log4j-${log4j.version}.jar"  todir="${build.dir}/lib" verbose="true"/>
    <copy file="${hadoop.home}/share/hadoop/common/lib/commons-cli-${commons-cli.version}.jar"  todir="${build.dir}/lib" verbose="true"/>
    <copy file="${hadoop.home}/share/hadoop/common/lib/commons-configuration-${commons-configuration.version}.jar"  todir="${build.dir}/lib" verbose="true"/>
    <copy file="${hadoop.home}/share/hadoop/common/lib/commons-lang-${commons-lang.version}.jar"  todir="${build.dir}/lib" verbose="true"/>
	<!-- 此處增加 commons-collections 依賴-->
	<copy file="${hadoop.home}/share/hadoop/common/lib/commons-collections-${commons-collections.version}.jar"  todir="${build.dir}/lib" verbose="true"/>
	<copy file="${hadoop.home}/share/hadoop/common/lib/jackson-core-asl-${jackson.version}.jar"  todir="${build.dir}/lib" verbose="true"/>
    <copy file="${hadoop.home}/share/hadoop/common/lib/jackson-mapper-asl-${jackson.version}.jar"  todir="${build.dir}/lib" verbose="true"/>
    <copy file="${hadoop.home}/share/hadoop/common/lib/slf4j-log4j12-${slf4j-log4j12.version}.jar"  todir="${build.dir}/lib" verbose="true"/>
    <copy file="${hadoop.home}/share/hadoop/common/lib/slf4j-api-${slf4j-api.version}.jar"  todir="${build.dir}/lib" verbose="true"/>
    <copy file="${hadoop.home}/share/hadoop/common/lib/guava-${guava.version}.jar"  todir="${build.dir}/lib" verbose="true"/>
    <copy file="${hadoop.home}/share/hadoop/common/lib/hadoop-auth-${hadoop.version}.jar"  todir="${build.dir}/lib" verbose="true"/>
    <copy file="${hadoop.home}/share/hadoop/common/lib/commons-cli-${commons-cli.version}.jar"  todir="${build.dir}/lib" verbose="true"/>
    <copy file="${hadoop.home}/share/hadoop/common/lib/netty-${netty.version}.jar"  todir="${build.dir}/lib" verbose="true"/>

    <jar
      jarfile="${build.dir}/hadoop-${name}-${version}.jar"
      manifest="${root}/META-INF/MANIFEST.MF">
      <manifest>
	 <attribute name="Bundle-ClassPath" 
		value="classes/, 
 lib/hadoop-mapreduce-client-core-${hadoop.version}.jar,
 lib/hadoop-mapreduce-client-common-${hadoop.version}.jar,
 lib/hadoop-mapreduce-client-jobclient-${hadoop.version}.jar,
 lib/hadoop-auth-${hadoop.version}.jar,
 lib/hadoop-common-${hadoop.version}.jar,
 lib/hadoop-hdfs-${hadoop.version}.jar,
 lib/protobuf-java-${protobuf.version}.jar,
 lib/log4j-${log4j.version}.jar,
 lib/commons-cli-1.2.jar,
 lib/commons-configuration-1.6.jar,
 lib/commons-httpclient-3.1.jar,
 <!-- 此處修改commons-lang版本-->
 lib/commons-lang-${commons-lang.version}.jar,
 <!-- 此處增加commons-collections依賴-->
 lib/commons-collections-${commons-collections.version}.jar,
 lib/jackson-core-asl-1.8.8.jar,
 lib/jackson-mapper-asl-1.8.8.jar,
 lib/slf4j-log4j12-1.7.5.jar,
 lib/slf4j-api-1.7.5.jar,
 lib/guava-${guava.version}.jar,
 lib/netty-${netty.version}.jar"/>
	 </manifest>
      <fileset dir="${build.dir}" includes="classes/ lib/"/>
      <!--fileset dir="${build.dir}" includes="*.xml"/-->
      <fileset dir="${root}" includes="resources/ plugin.xml"/>
    </jar>
  </target>

</project>

 
   
 


 

發佈了63 篇原創文章 · 獲贊 134 · 訪問量 13萬+
發表評論
所有評論
還沒有人評論,想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章