Flink之wordcount

小編初學flink,看到各大論壇上有各種不同版本的,有一些還運行不了,小編就參照官網寫了一個

pom文件是這樣的

<dependencies>
    <!--flink——scala的依賴-->
    <dependency>
        <groupId>org.apache.flink</groupId>
        <artifactId>flink-scala_2.11</artifactId>
        <version>1.6.1</version>
    </dependency>
    <dependency>
        <groupId>org.apache.flink</groupId>
        <artifactId>flink-streaming-scala_2.11</artifactId>
        <version>1.6.1</version>
    </dependency>
    <!--解決日誌報錯SLF4J-->
    <dependency>
        <groupId>org.slf4j</groupId>
        <artifactId>slf4j-nop</artifactId>
        <version>1.7.2</version>
     </dependency>
</dependencies>

scala代碼

object WordCount {
  def main(args: Array[String]): Unit = {

    //獲取運行環境
    val env= ExecutionEnvironment.getExecutionEnvironment

    //連接此socket獲取輸入數據
    val text = env.readTextFile("E:\\Learning\\log.txt")

    //必須寫,不寫下面代碼會報錯  
    import org.apache.flink.api.scala._

    val counts = text.flatMap(_.toLowerCase.split("\\W+")filter(_.nonEmpty))
      .map((_,1))
      .groupBy(0)
      .sum(1)
    counts.print()

  }
}

其中遇到了兩個錯誤

1、寫flatMap時代碼報錯

解決方案,加上

import org.apache.flink.api.scala._

官網解釋: https://ci.apache.org/projects/flink/flink-docs-release-1.4/dev/types_serialization.html#type-information-in-the-scala-api

2、程序能執行,但是控制檯報錯(強迫症)
slf4j報錯
官網的解釋

This error is reported when the org.slf4j.impl.StaticLoggerBinder class could not be loaded into memory. This happens when no appropriate SLF4J binding could be found on the class path. Placing one (and only one) of slf4j-nop.jar, slf4j-simple.jar, slf4j-log4j12.jar, slf4j-jdk14.jar or logback-classic.jar on the class path should solve the problem.

解決方案:在Maven工程的pom文件中隨便加上依賴就可以了

    <dependency>
        <groupId>org.slf4j</groupId>
        <artifactId>slf4j-nop</artifactId>
        <version>1.7.2</version>
     </dependency>
發表評論
所有評論
還沒有人評論,想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章