将Spark项目从VirtualBox转移到Linux ......有些问题

loading...


0

我一直在使用VirtualBox中的一个项目,现在我正在尝试在Ubuntu中使用 . 我安装了 Spark (新版本)和 Scala 来使用这个项目,但是我有以下错误:

线程“AWT-EventQueue-0”中的异常java.util.ServiceConfigurationError:org.apache.spark.sql.sources.DataSourceRegister:无法在java实例化提供程序org.apache.spark.ml.source.libsvm.LibSVMFileFormat . java.base / java.util.ServiceLoader上的java.base / java.util.ServiceLoader.access $ 100(ServiceLoader.java:390)中的base / java.util.ServiceLoader.fail(ServiceLoader.java:581)$ ProviderImpl.newInstance (ServiceLoader.java:803)java.base / java.util.ServiceLoader $ ProviderImpl.get(ServiceLoader.java:721)at java.base / java.util.ServiceLoader $ 3.next(ServiceLoader.java:1394)at scala .collection.convert.Wrappers $ JIteratorWrapper.next(Wrappers.scala:43)at scala.collection.Iterator $ class.foreach(Iterator.scala:891)at scala.collection.AbstractIterator.foreach(Iterator.scala:1334)at at scala.collection.IterableLike $ class.foreach(IterableLike.scala:72)at scala.collection.AbstractIterable.foreach(Iterable.scala:54)at scala.collection.TraversableLike $ class.filterImpl(TraversableLike.scala: 247)scala.collection.TraversableLike $ class.filter(TraversableLike.scala:259)at scala.collection.AbstractTraversable.filter(Traversable.scala:104)at org.apache.spark.sql.execution.datasources.DataSource $ . 在org.apache上的org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:190)中的lookupDataSource(DataSource.scala:618)org.apache.spark.sql.DataFrameReader.csv(DataFrameReader.scala:596) hspProject.hupaProjectSpark.CSVSpark.sparkProcess(CSVSpark.java:68)中的.spark.sql.DataFrameReader.csv(DataFrameReader.scala:473)位于java.desktop的hupaProject.ui.MainWindow $ 2.actionPerformed(MainWindow.java:244) /javax.swing.AbstractButton.fireActionPerformed(AbstractButton.java:1967)java.desktop / javax.swing.AbstractButton $ Handler.actionPerformed(AbstractButton.java:2308)at java.desktop / javax.swing.DefaultButtonModel.fireActionPerformed(DefaultButtonModel) .java:405)位于java.desktop / javax.swing.plaf的java.desktop / javax.swing.DefaultButtonModel.setPressed(DefaultButtonModel.java:262) . java.desktop / javax.swing.JComponent.processMouseEvent(JComponent.java:3342)中的java.desktop / java.awt.Component.processMouseEvent(Component.java:6589)中的basic.BasicButtonListener.mouseReleased(BasicButtonListener.java:270) at java.desktop / java.awt.Component.processEvent(Component.java:6354)at java.desktop / java.awt.Container.processEvent(Container.java:2261)at java.desktop / java.awt.Component.dispatchEventImpl (Component.java:4966)java.desktop上的java.desktop / java.awt.Container.dispatchEventImpl(Container.java:2319)java.desktop / java.awt.Component.dispatchEvent(Component.java:4798) java.desktop / java.awt.LightweightDispatcher.dispatchEvent上的java.desktop / java.awt.LightweightDispatcher.processMouseEvent(Container.java:4543)中的java.awt.LightweightDispatcher.retargetMouseEvent(Container.java:4914)(Container.java: 4484)at java.desktop / java.awt.Container.dispatchEventImpl(Container.java:2305)at java.desktop / java.awt.Window.dispatchEventImpl(Window.java:2772)at java.desktop / java.awt .Component.dispatchEvent(Component.java:4798)位于java.desktop / java.awt.EventQueue.dispatchEventImpl(EventQueue.java:772)java.desktop / java.awt.EventQueue.access $ 600(EventQueue.java:97) java.desktop / java.awt.EventQueue $ 4.run(EventQueue.java:721)at java.desktop / java.awt.EventQueue $ 4.run(EventQueue.java:715)at java.base / java.security.AccessController java.base上的.doPrivileged(Native Method)java.base / java.security.ProtectionDomain $ JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:87)java上的JavaAcurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:97) .desktop / java.awt.EventQueue $ 5.run(EventQueue.java:745)at java.desktop / java.awt.EventQueue $ 5.run(EventQueue.java:743)at java.base / java.security.AccessController.doPrivileged java.desktop / java.awt.EventQueue.dispatchEvent(EventQueue.java)中的java.base / java.security.ProtectionDomain $ JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:87)上的(本机方法) :742)at java.desktop / java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:203)at java.desktop / java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:124)at java.desktop / java.awt . 位于java.desktop / java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101)的java.desktop / java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:109)中的EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:113) .desktop / java.awt.EventDispatchThread.run(EventDispatchThread.java:90)引起:java.lang.NoSuchMethodError:org.apache.spark.internal.Logging . $ init $(Lorg / apache / spark / internal / Logging; )V atorg.apache.spark.ml.source.libsvm.LibSVMFileFormat . (LibSVMRelation.scala:71)位于java.base / jdk.internal.reflect的java.base / jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)中 . java.base / java.lang.reflect.Constructor.newInstance(Constructor.java:)中java.base / jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)中的NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) 488)在java.base / java.util.ServiceLoader $ ProviderImpl.newInstance(ServiceLoader.java:779)... 52更多<

这是我的pom.xml:

http://maven.apache.org/xsd/maven-4.0.0.xsd“> 4.0.0

<groupId>projectx</groupId>
<artifactId>ProjectSpark</artifactId>
<version>0.0.1-SNAPSHOT</version>
<packaging>jar</packaging>

<name>ProjectSpark</name>
<url>http://maven.apache.org</url>

<properties>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
</properties>

<dependencies>

    <dependency>
        <groupId>org.apache.spark</groupId>
        <artifactId>spark-sql_2.11</artifactId>
        <version>2.3.2</version>
    </dependency>

    <!-- <dependency> <groupId>org.apache.spark</groupId> <artifactId>spark-mllib_2.11</artifactId> 
        <version>2.0.0</version> </dependency> -->

    <!-- https://mvnrepository.com/artifact/org.apache.spark/spark-mllib -->
    <dependency>
        <groupId>org.apache.spark</groupId>
        <artifactId>spark-mllib_2.12</artifactId>
        <version>2.4.0</version>
        <scope>runtime</scope>
    </dependency>

    <!-- quitado por problemas, haciendo pruebas -->
    <dependency>
        <groupId>org.mongodb</groupId>
        <artifactId>mongo-java-driver</artifactId>
        <version>3.8.2</version>
    </dependency>

    <dependency>
        <groupId>org.apache.directory.server</groupId>
        <artifactId>kerberos-client</artifactId>
        <version>2.0.0-M16</version>
    </dependency>

    <dependency>
        <groupId>org.apache.hadoop</groupId>
        <artifactId>hadoop-yarn-server-web-proxy</artifactId>
        <version>3.1.1</version>
    </dependency>


    <!-- https://mvnrepository.com/artifact/com.stratio.datasource/spark-mongodb -->
    <!--<dependency> <groupId>com.stratio.datasource</groupId> <artifactId>spark-mongodb_2.11</artifactId> 
        <version>0.12.0</version> </dependency> -->


    <dependency>
        <groupId>org.mongodb.spark</groupId>
        <artifactId>mongo-spark-connector_2.11</artifactId>
        <version>2.3.1</version>
    </dependency>

    <dependency>
        <groupId>junit</groupId>
        <artifactId>junit</artifactId>
        <version>3.8.1</version>
        <scope>test</scope>
    </dependency>

</dependencies>

<build>
    <sourceDirectory>src</sourceDirectory>
    <plugins>
        <plugin>
            <artifactId>maven-compiler-plugin</artifactId>
            <version>3.7.0</version>
            <configuration>
                <source>1.8</source>
                <target>1.8</target>
            </configuration>
        </plugin>
    </plugins>

</build>

可能是什么问题呢?

感谢你们!问候 .

0回答

No Data

评论

loading...

暂时没有评论!