val dstream = this.fire.createRandomIntStream()
dstream. foreachRDDAtLeastOnce(_ => {})
异常如下:
24/03/27 15:44:38 ERROR JobScheduler: Error running job streaming job 1711524750000 ms.0
java.lang.IllegalArgumentException: DStream必须为最原始的source流,不能经过transformation算子做转换!
at com.zto.fire.spark.ext.core.DStreamExt.$anonfun$foreachRDDAtLeastOnce$1(DStreamExt.scala:135)
at com.zto.fire.spark.ext.core.DStreamExt.$anonfun$foreachRDDAtLeastOnce$1$adapted(DStreamExt.scala:111)
at org.apache.spark.streaming.dstream.ForEachDStream.$anonfun$generateJob$2(ForEachDStream.scala:51)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:429)
at org.apache.spark.streaming.dstream.ForEachDStream.$anonfun$generateJob$1(ForEachDStream.scala:51)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at scala.util.Try$.apply(Try.scala:213)
at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.$anonfun$run$1(JobScheduler.scala:260)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:260)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
登录 后才可以发表评论