我是新来的火花，我正在尝试从火花分布提供的示例中运行Kafka流示例 . 火花2.0.2彬hadoop2.7

线程“main”中的异常java.lang.NoClassDefFoundError：scala / collection / GenTraversableOnce $ class

Attached image shows the stack trace

我试图使用命令在Windows shell上的Windows上运行代码

spark-submit --master local [*] --class org.apache.spark.examples.streaming.JavaDirectKafkaWordCount ../target/spark-examples_2.11-2.2.0-SNAPSHOT-jar-with-dependencies.jar

jar scala-lang-2.11.8.jar有类--scala / collection / GenTraversableOnce $类，jar包含在类路径中

类路径包括所有依赖jar

我正在使用maven构建的所有依赖jar来运行kakfa字数计数示例 .

所有maven依赖项都复制到一个文件夹，所有jar都包含在类路径中 .

我想知道我是否还应该做其他事情？如果有人可以看一看并指导它会很棒 . 我过去2天都在苦苦挣扎 .

POM附在下面

http://maven.apache.org/xsd/maven-4.0.0.xsd“> 4.0.0 org.apache.spark spark-parent_2.11 2.2.0-SNAPSHOT ../pom.xml

<artifactId>spark-examples_2.11</artifactId>
<packaging>jar</packaging>
<name>Spark Project Examples</name>
<url>http://spark.apache.org/</url>

<properties>
    <sbt.project.name>examples</sbt.project.name>
    <build.testJarPhase>none</build.testJarPhase>
    <build.copyDependenciesPhase>package</build.copyDependenciesPhase>
    <flume.deps.scope>provided</flume.deps.scope>
    <hadoop.deps.scope>provided</hadoop.deps.scope>
    <hive.deps.scope>provided</hive.deps.scope>
    <parquet.deps.scope>provided</parquet.deps.scope>
</properties>

<dependencies>
    <!-- Prevent our dummy JAR from being included in Spark distributions or 
        uploaded to YARN -->
    <dependency>
        <groupId>org.spark-project.spark</groupId>
        <artifactId>unused</artifactId>
        <version>1.0.0</version>
        <scope>provided</scope>
    </dependency>
    <dependency>
        <groupId>org.apache.spark</groupId>
        <artifactId>spark-core_${scala.binary.version}</artifactId>
        <version>${project.version}</version>
        <scope>provided</scope>
    </dependency>
    <dependency>
        <groupId>org.apache.spark</groupId>
        <artifactId>spark-streaming_${scala.binary.version}</artifactId>
        <version>${project.version}</version>
        <scope>provided</scope>
    </dependency>
    <dependency>
        <groupId>org.apache.spark</groupId>
        <artifactId>spark-mllib_${scala.binary.version}</artifactId>
        <version>${project.version}</version>
        <scope>provided</scope>
    </dependency>
    <dependency>
        <groupId>org.apache.spark</groupId>
        <artifactId>spark-hive_${scala.binary.version}</artifactId>
        <version>${project.version}</version>
        <scope>provided</scope>
    </dependency>
    <dependency>
        <groupId>org.apache.spark</groupId>
        <artifactId>spark-graphx_${scala.binary.version}</artifactId>
        <version>${project.version}</version>
        <scope>provided</scope>
    </dependency>
    <dependency>
        <groupId>org.apache.spark</groupId>
        <artifactId>spark-streaming-flume_${scala.binary.version}</artifactId>
        <version>${project.version}</version>
        <scope>provided</scope>
    </dependency>
    <dependency>
        <groupId>org.apache.spark</groupId>
        <artifactId>spark-streaming-kafka-0-8_${scala.binary.version}</artifactId>
        <version>${project.version}</version>
        <scope>provided</scope>
    </dependency>
    <dependency>
        <groupId>org.apache.commons</groupId>
        <artifactId>commons-math3</artifactId>
        <scope>provided</scope>
    </dependency>
    <dependency>
        <groupId>org.scalacheck</groupId>
        <artifactId>scalacheck_${scala.binary.version}</artifactId>
        <scope>test</scope>
    </dependency>
    <dependency>
        <groupId>org.scala-lang</groupId>
        <artifactId>scala-library</artifactId>
        <scope>provided</scope>
    </dependency>
    <dependency>
        <groupId>com.github.scopt</groupId>
        <artifactId>scopt_${scala.binary.version}</artifactId>
        <version>3.3.0</version>
    </dependency>
    <dependency>
        <groupId>com.twitter</groupId>
        <artifactId>parquet-hadoop-bundle</artifactId>
        <scope>provided</scope>
    </dependency>
</dependencies>

<build>
    <outputDirectory>target/scala-${scala.binary.version}/classes</outputDirectory>
    <testOutputDirectory>target/scala-${scala.binary.version}/test-classes</testOutputDirectory>
    <plugins>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-deploy-plugin</artifactId>
            <configuration>
                <skip>true</skip>
            </configuration>
        </plugin>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-install-plugin</artifactId>
            <configuration>
                <skip>true</skip>
            </configuration>
        </plugin>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-dependency-plugin</artifactId>
            <executions>
                <execution>
                    <id>copy-dependencies</id>
                    <phase>prepare-package</phase>
                    <goals>
                        <goal>copy-dependencies</goal>
                    </goals>
                    <configuration>
                        <outputDirectory>${project.build.directory}/lib1</outputDirectory>
                        <overWriteReleases>false</overWriteReleases>
                        <overWriteSnapshots>false</overWriteSnapshots>
                        <overWriteIfNewer>true</overWriteIfNewer>
                    </configuration>
                </execution>
            </executions>
        </plugin>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-jar-plugin</artifactId>
            <configuration>
                <outputDirectory>${jars.target.dir}</outputDirectory>
            </configuration>
        </plugin>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-assembly-plugin</artifactId>
            <version>2.6</version>
            <configuration>
                <descriptorRefs>
                    <descriptorRef>jar-with-dependencies</descriptorRef>
                </descriptorRefs>
            </configuration>
            <executions>
                <execution>
                    <phase>package</phase>
                    <goals>
                        <goal>single</goal>
                    </goals>
                </execution>
            </executions>
        </plugin>
    </plugins>
</build>
<profiles>
    <profile>
        <id>kinesis-asl</id>
        <dependencies>
            <dependency>
                <groupId>org.apache.spark</groupId>
                <artifactId>spark-streaming-kinesis-asl_${scala.binary.version}</artifactId>
                <version>${project.version}</version>
                <scope>provided</scope>
            </dependency>
        </dependencies>
    </profile>
</profiles>
java scala spark-streaming