我在本地开发了一个Spark应用程序并没有遇到任何问题 . 但是当我想在Docker Image的Yarn Cluster中推送它时,我收到了以下消息:

线程“main”org.apache.spark.SparkException中的异常:作业因阶段失败而中止:阶段0.0中的任务2失败4次,最近失败:阶段0.0中失去的任务2.3(TID 26,沙箱):ExecutorLostFailure(执行者1丢失)驱动程序堆栈跟踪:at org.apache.spark.scheduler.DAGScheduler.org $ apache $ spark $ scheduler $ DAGScheduler $$ failJobAndIndependentStages(DAGScheduler.scala:1203)at org.apache.spark.scheduler.DAGScheduler $$ anonfun $ abappStage $ 1.apply(DAGScheduler.scala:1192)at org.apache.spark.scheduler.DAGScheduler $$ anonfun $ abortStage $ 1.apply(DAGScheduler.scala:1191)at scala.collection.mutable.ResizableArray $ class.foreach( ResizableArray.scala:59)位于org.apache.spark.scheduler的org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:1191)的scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:47) .DAGScheduler $$ anonfun $ handleTaskSetFailed $ 1.apply(DAGScheduler.scala:693)at org.apache.spark.scheduler.DAGScheduler $$ anonfun $ handleTaskSetFailed $ 1.apply( DAGScheduler.scala:693)scala.Option.foreach(Option.scala:236)org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:693)org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive (DAGScheduler.scala:1393)org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1354)at org.apache.spark.util.EventLoop $$ anon $ 1.run(EventLoop.scala:48)uote

用于启动应用程序的命令是:

spark-submit --class myapp.myapp_spark.App --master yarn-client /opt/myapp/myapp_spark.jar

我的应用程序使用的是Mongo数据库 . 它是否与内存问题,与Mongo或其他东西的连接有关?提前致谢