我正在尝试使用命令行编译和运行Scala的WordCOunt程序,而不需要任何Maven和sbt支持 . 我用来编译scala程序的命令是
scalac -classpath /spark-2.3.0-bin-hadoop2.7/jars/ Wordcount.scala
import org.apache.spark._
import org.apache.spark.SparkConf
/** Create a RDD of lines from a text file, and keep count of
* how often each word appears.
*/
object wordcount {
def main(args: Array[String]) {
// Set up a SparkContext named WordCount that runs locally using
// all available cores.
val conf = new SparkConf().setAppName("WordCount")
conf.setMaster("local[*]")
val sc = new SparkContext(conf)
MY RESEARCH: 我已经参考了源代码,发现import语句在他们所需的jar中 .
For example SparkConf存在于程序中提到的org.apache.spark包中 .
https://github.com/apache/spark/blob/v2.3.1/core/src/main/scala/org/apache/spark/SparkConf.scala
ERRORS I AM FACING :
Wordcount.scala:3:错误:**对象apache不是包org import的成员org.apache.spark._ ^ ** Wordcount.scala:4:错误:**对象apache不是包org的成员import org.apache.spark.SparkConf ** ^ Wordcount.scala:14:错误:找不到:类型SparkConf val conf = new SparkConf() . setAppName(“WordCount”) ^ Wordcount.scala:16:error :找不到:类型SparkContext val sc = new SparkContext(conf) ^
发现了四个错误
1 回答
试试这个:
您的问题中提到的 scalac 命令存在问题 . 如果要从某个目录中选择所有jar并将其放在classpath中,则需要使用 *** wildcard** 字符并将路径包装在双引号内 .
详情请参阅:Including all the jars in a directory within the Java classpath