我试图将我的couchBase服务器连接到EMR Spark 1.4.1,同时遇到了
val airlines = sqlContext.read.couchbase(schemaFilter = org.apache.spark.sql.sources.EqualTo("type", "airline"))
<console>:24: error: value couchbase is not a member of org.apache.spark.sql.DataFrameReader
这些是在错误命令之前成功执行的所有命令:
-
spark-shell --packages com.couchbase.client:spark-connector_2.10:1.0.0
-
import org.apache.spark . {SparkContext,SparkConf}
-
val sc = new SparkContext(new SparkConf() . setAppName("test") . set("com.couchbase.bucket.travel-sample",“”))
-
val cfg = new SparkConf() . setAppName("keyValueExample") . setMaster("local[*]") . set("com.couchbase.bucket.travel-sample",“”)
-
import org.apache.spark.sql.SQLContext
-
val sql = new SQLContext(sc)
-
import com.couchbase.spark._
我还需要配置更多吗?由于我使用的是AWS EMR,我认为我不必修改.sbt文件?我想我已经在指定何时连接到spark-shell或在行(命令)7中导入了包?
1 回答
文档说您必须导入以下内容:
完整文档可在此处获取:http://developer.couchbase.com/documentation/server/current/connectors/spark-1.0/spark-shell.html