首页 文章

Datastax Spark Zeppelin身份验证

提问于
浏览
1

我有一个数据存储集群4.8(Cassandra Spark)激活了身份验证 . 我希望能够在我的集群上使用带有Spark master和我的数据库Cassandra的notebook Zeppelin .

我下载Zeppelin的二进制包0.5.6 . 我把它放在我的服务器上 . 如果我用默认配置启动它(./bin/zeppelin-daemon.sh start)它的工作正常http://ServerName:8080/#/ .

但是当我想使用我的DSE spark master时,结果输出是:

java.net.ConnectException:java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)中java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:344)的java.net.PlainSocketImpl.socketConnect(Native Method)连接被拒绝)java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)的java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)at java..Socket.connect(Socket.java:579)at org.apache位于org.apache.zeppelin.interpreter.remote.ClientFactory.create的org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:51)中的.thrift.transport.TSocket.open(TSocket.java:182) (ClientFactory.java:37)org.apache.commons.pool2.BasePooledObjectFactory.makeObject(BasePooledObjectFactory.java:60)org.apache.commons.pool2.impl.GenericObjectPool.create(GenericObjectPool.java:861)atg . org.apache.commons.pool2.impl.Gen上的apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:435) ericObjectPool.borrowObject(GenericObjectPool.java:363)org.apache.zeppelin.interpreter.remote.RemoteInterpreterProcess.getClient(RemoteInterpreterProcess.java:139)org.apache.zeppelin.interpreter.remote.RemoteInterpreter.init(RemoteInterpreter.java: 129)atg.apache.zeppelin.interpreter.remote.RemoteInterpreter.getFormType(RemoteInterpreter.java:257)org.apache.zeppelin.interpreter.LazyOpenInterpreter.getFormType(LazyOpenInterpreter.java:104)at org.apache.zeppelin.notebook .Paragraph.jobRun(Paragraph.java:198)org.apache.zeppelin.scheduler.Job.run(Job.java:169)org.apache.zeppelin.scheduler.RemoteScheduler $ JobRunner.run(RemoteScheduler.java:322) )java.util.concurrent.Executors $ java的RunnableAdapter.call(Executors.java:471)java.util.concurrent.FutureTask.run(FutureTask.java:262)java.util.concurrent.ScheduledThreadPoolExecutor $ ScheduledFutureTask.access $ 201 (ScheduledThreadPoolExecutor.java:178)java.util.concurrent.ScheduledThreadPoolExecutor $ ScheduledFutureTask.r un(ScheduledThreadPoolExecutor.java:292)java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)at java.util.concurrent.ThreadPoolExecutor $ Worker.run(ThreadPoolExecutor.java:615)at java.lang.Thread .RUN(Thread.java:745)

看起来笔记本电脑无法连接到我通常使用的spark-master

$> dse -u“username”-p“password”spark

我不确定这是问题,但我无法弄清楚在哪里设置这些参数xD .

有关信息,我将/zeppelin-0.5.6-incubating-bin-all/conf/zeppelin-env.sh设置为:

  • export MASTER=spark://ip_of_my_server:7077

  • export ZEPPELIN_MEM=-Xmx5g 正如它在Hello world in zeppelin failed中提到的拉动请求ZEPPELIN-305中所建议的那样(但我没有't think it'这个问题,因为这个请求在0.5.6中被关闭)

  • export SPARK_HOME=/usr/share/dse/spark 包含:

bin data lib python RELEASE sbin spark-jobserver

我虽然在解释器接口的“master”字段中输入了“spark:// ip_of_my_server:7077” .

那么你有什么想法解决我的问题并连接DSE spark和zeppelin :)?

1 回答

  • 0

    与Datastax专家进行一些交流之后Duy Hai Doan

    我得到一个解决方案,我建议你去他的博客http://www.doanduyhai.com/blog/?p=2325

    对于验证细节,请转到解释器参数并添加:

    • For Cassandra

    cassandra.hosts:“YourNodeIP”cassandra.credentials.username:“YourUserName”cassandra.credentials.password:“YourPassword”


    • For Spark

    spark.cassandra.auth.password:“YourPassword”spark.cassandra.auth.username:“YourUserName”spark.cassandra.connection.host:“YourSparkMasterIP”

    非常感谢Datastax和Duy

相关问题