当我运行我的火花作业来创建我的DSE图时,我遇到了问题:
My code :
import com.datastax.bdp.graph.spark.graphframe._
object TestGraph extends App with SparkConnection {
val graph = spark.dseGraph(cassandraConf.GRAPH_NAME)
}
Stack :
Exception in thread "main" java.lang.NoSuchMethodError: it.unimi.dsi.fastutil.objects.ObjectArraySet.<init>(Ljava/util/Collection;)V
at com.datastax.bdp.graph.impl.schema.ColumnContainer.<init>(ColumnContainer.java:25)
at com.datastax.bdp.graph.impl.schema.ColumnDefinitions.<clinit>(ColumnDefinitions.java:67)
at com.datastax.bdp.graph.spark.graphframe.DseGraphFrame$.syntheticAdjVertexIdColumn(DseGraphFrame.scala:795)
at com.datastax.bdp.graph.spark.graphframe.EdgeBuilder.df(DseGraphFrameBuilder.scala:222)
at com.datastax.bdp.graph.spark.graphframe.DseGraphFrameBuilder$$anonfun$oneDirectionEdges$1$1.apply(DseGraphFrameBuilder.scala:96)
at com.datastax.bdp.graph.spark.graphframe.DseGraphFrameBuilder$$anonfun$oneDirectionEdges$1$1.apply(DseGraphFrameBuilder.scala:96)
at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
at scala.collection.Iterator$class.foreach(Iterator.scala:893)
at scala.collection.AbstractIterator.foreach(Iterator.scala:1336)
at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
at scala.collection.AbstractIterable.foreach(Iterable.scala:54)
at scala.collection.TraversableLike$class.map(TraversableLike.scala:234)
at scala.collection.AbstractTraversable.map(Traversable.scala:104)
at com.datastax.bdp.graph.spark.graphframe.DseGraphFrameBuilder.oneDirectionEdges$1(DseGraphFrameBuilder.scala:96)
at com.datastax.bdp.graph.spark.graphframe.DseGraphFrameBuilder.edges(DseGraphFrameBuilder.scala:99)
at com.datastax.bdp.graph.spark.graphframe.DseGraphFrameBuilder.graphFrame(DseGraphFrameBuilder.scala:107)
at com.datastax.bdp.graph.spark.graphframe.SparkSessionFunctions.dseGraph(SparkSessionFunctions.scala:20)
我使用了这个依赖项:
<dependency>
<groupId>com.datastax.dse</groupId>
<artifactId>dse-byos_2.11</artifactId>
<version>6.7.0</version>
</dependency>
当我在本地模式下运行时,它工作,但当我在cluter中运行我的工作时,我有这个错误 . 是否有任何配置或依赖项来添加依赖项?你能帮帮我吗!!