Flink窗口之滑動窗口scala版本

Flink窗口之滑動窗口scala版本

代碼

package windows

import org.apache.flink.api.java.tuple.Tuple
import org.apache.flink.streaming.api.scala._
import org.apache.flink.streaming.api.windowing.time.Time
import org.apache.flink.streaming.api.windowing.windows.TimeWindow


/**滑動窗口*/
object SlidingWindows {
  def main(args: Array[String]): Unit = {
    val env: StreamExecutionEnvironment = StreamExecutionEnvironment.getExecutionEnvironment

    val line: DataStream[String] = env.socketTextStream("t1",9999)

    val flatMaped: DataStream[String] = line.flatMap(_.split(","))

    val wordAndOne: DataStream[(String, Int)] = flatMaped.map((_,1))

    val keyByed: KeyedStream[(String, Int), Tuple] = wordAndOne.keyBy(0)

    //開啓一個滑動窗口 時間間隔是10秒鐘 滑動時間是5秒
    val windows: WindowedStream[(String, Int), Tuple, TimeWindow] = keyByed.timeWindow(Time.seconds(10),Time.seconds(5))
    val result: DataStream[(String, Int)] = windows.sum(1)
    result.print()

    env.execute("SlidingWindows")

  }
}

輸入數據

[root@t1 ~]# nc -lk 9999
a,a,a
b,b,b
c,c,c
d,d,d
e,e,e
f,f,f

運行結果

6> (a,3)
6> (a,3)
2> (b,3)
4> (c,3)
1> (e,3)
4> (c,3)
5> (d,3)
2> (b,3)
5> (d,3)
1> (e,3)
2> (f,3)
2> (f,3)

總結

• 滑動窗口數據會造成重複
• 運用場景： 每隔30分鐘統計前1個小時的數據