我有一个数据中心,其版本是4.8.7 . 数据存储集群有6个节点(1个节点用于solr数据中心,5个节点用于分析数据中心),每个节点配置32个内核和64G内存 .

spark应用程序在分析数据中心运行,其错误日志为

错误[SparkWorker-0错误 Logger ] 2016-09-23 16:40:10,690 SparkWorker-0 ExternalLogger.java:96 - 将流写入文件/ var / lib / spark / worker / worker-0 / app-20160923163007-时出错0136/3 / stderr错误[SparkWorker-0错误 Logger ] 2016-09-23 16:40:10,690 SparkWorker-0 ExternalLogger.java:96 - java.io.IOException:流关闭ERROR [SparkWorker-0错误 Logger ] 2016- 09-23 16:40:10,690 SparkWorker-0 ExternalLogger.java:96 - 在java.io.BufferedInputStream.getBufIfOpen(BufferedInputStream.java:170)〜[na:1.8.0_92] ERROR [SparkWorker-0错误 Logger ] 2016- 09-23 16:40:10,690 SparkWorker-0 ExternalLogger.java:96 - 在java.io.BufferedInputStream.read1(BufferedInputStream.java:283)〜[na:1.8.0_92] ERROR [SparkWorker-0错误 Logger ] 2016- 09-23 16:40:10,690 SparkWorker-0 ExternalLogger.java:96 - 在java.io.BufferedInputStream.read(BufferedInputStream.java:345)〜[na:1.8.0_92] ERROR [SparkWorker-0错误 Logger ] 2016- 09-23 16:40:10,690 SparkWorker-0 ExternalLogger.java:96 - 在java.io.FilterInputStream.r ead(FilterInputStream.java:107)〜[na:1.8.0_92]错误[SparkWorker-0错误 Logger ] 2016-09-23 16:40:10,690 SparkWorker-0 ExternalLogger.java:96 - atg.apache.spark . util.logging.FileAppender.appendStreamToFile(FileAppender.scala:70)〜[spark-core_2.10-1.4.2.5.jar:1.4.2.5] ERROR [SparkWorker-0错误 Logger ] 2016-09-23 16:40:10,690 SparkWorker-0 ExternalLogger.java:96 - atg.apache.spark.util.logging.FileAppender $$ anon $ 1 $$ anonfun $ run $ 1.apply $ mcV $ sp(FileAppender.scala:39)[spark-core_2.10 -1.4.2.5.jar:1.4.2.5]错误[SparkWorker-0错误 Logger ] 2016-09-23 16:40:10,690 SparkWorker-0 ExternalLogger.java:96 - at org.apache.spark.util.logging.FileAppender $$ anon $ 1 $$ anonfun $ run $ 1.apply(FileAppender.scala:39)[spark-core_2.10-1.4.2.5.jar:1.4.2.5] ERROR [SparkWorker-0错误 Logger ] 2016-09-23 16 :40:10,690 SparkWorker-0 ExternalLogger.java:96 - atg.apache.spark.util.logging.FileAppender $$ anon $ 1 $$ anonfun $ run $ 1.apply(FileAppender.scala:39)[spark-core_2.10 -1.4.2.5.jar:1.4.2.5]错误[SparkWorker- 0错误 Logger ] 2016-09-23 16:40:10,691 SparkWorker-0 ExternalLogger.java:96 - at org.apache.spark.util.Utils $ .logUncaughtExceptions(Utils.scala:1652)[spark-core_2.10- 1.4.2.5.jar:1.4.2.5]错误[SparkWorker-0错误 Logger ] 2016-09-23 16:40:10,691 SparkWorker-0 ExternalLogger.java:96 - at org.apache.spark.util.logging.FileAppender $ $ anon $ 1.run(FileAppender.scala:38)[spark-core_2.10-1.4.2.5.jar:1.4.2.5] WARN [sparkMaster-akka.actor.default-dispatcher-2] 2016-09-23 16: 40:10,691 Slf4jLogger.scala:71 - 尝试与无法访问的远程地址关联[akka.tcp:// sparkWorker @ localhost:53714] . 地址现在被封闭了5000毫秒,所有发送到此地址的消息都将被发送到死信 . 原因:连接被拒绝:/ localhost:53714 INFO [sparkMaster-akka.actor.default-dispatcher-13] 2016-09-23 16:40:10,691 Logging.scala:59 - akka.tcp:// sparkWorker @ localhost:53714得到了解散,删除它 . INFO [SparkWorker-0 logger] 2016-09-23 16:40:10,693 SparkWorker-0 ExternalLogger.java:88 - 启动命令:“/ usr / java / jdk1.8.0_92 / jre / bin / java”“-cp”“/etc/dse/spark/:/usr/share/dse/dse-core-4.8.7.jar:/usr/share/dse/dse-hadoop-4.8.7.jar:/usr/share/dse/ DSE蜂箱-4.8.7.jar:/usr/share/dse/dse-search-4.8.7.jar:/usr/share/dse/dse-spark-4.8.7.jar:在/ usr /共享/ DSE /dse-sqoop-4.8.7.jar:/etc/dse/spark/:/usr/share/dse/spark/lib/activation-1.1.jar:/usr/share/dse/spark/lib/akka-actor_2 .10-2.3.4-spark.jar:/usr/share/dse/spark/lib/akka-remote_2.10-2.3.4-spark.jar:在/ usr /共享/ DSE /火花/ LIB /阿卡-slf4j_2 .10-2.3.4-spark.jar:/usr/share/dse/spark/lib/antlr-2.7.7.jar:/usr/share/dse/spark/lib/antlr-runtime-3.4.jar:/ USR /共享/ DSE /火花/ LIB / arpack_combined_all-0.1.jar:/usr/share/dse/spark/lib/asm-3.1.jar:/usr/share/dse/spark/lib/asm-4.0.jar: /usr/share/dse/spark/lib/asm-commons-3.1.jar:/usr/share/dse/spark/lib/asm-tree-3.1.jar:/usr/share/dse/spark/lib/avro -1.7.7.jar:/usr/share/dse/spark/lib/avro-ipc-1.7.7.jar:/usr/share/dse/spark/lib/avro-mapred-1.7.7-hadoop1.jar :/usr/share/dse/spark/lib/avro-mapred-1.7.7.jar:/usr/share/dse/spark/lib/bonecp-0.8.0.RELEASE.jar:/我们R /共享/ DSE /火花/ LIB / breeze_2.10-0.11.2.jar:/usr/share/dse/spark/lib/breeze-macros_2.10-0.11.2.jar:在/ usr /共享/ DSE /火花/ lib目录/ chill_2.10-0.5.0.jar:/usr/share/dse/spark/lib/chill-java-0.5.0.jar:在/ usr /共享/ DSE /火花/ lib目录/公地CLI- 1.2.jar:/usr/share/dse/spark/lib/commons-codec-1.9.jar:/usr/share/dse/spark/lib/commons-collections-3.2.2.jar:在/ usr /共享/ DSE /spark/lib/commons-compress-1.4.1.jar:/usr/share/dse/spark/lib/commons-httpclient-3.1.jar:/usr/share/dse/spark/lib/commons-io-2.4的.jar:/usr/share/dse/spark/lib/commons-lang-2.4.jar:/usr/share/dse/spark/lib/commons-lang3-3.3.2.jar:在/ usr /共享/ DSE /火花/ lib目录/共享记录-1.2.jar:/usr/share/dse/spark/lib/commons-math3-3.4.1.jar:/usr/share/dse/spark/lib/commons-net-2.2 . jar :/usr/share/dse/spark/lib/compress-lzf-1.0.3.jar:/usr/share/dse/spark/lib/config-1.2.1.jar:在/ usr /共享/ DSE /火花/lib/core-1.1.2.jar:/usr/share/dse/spark/lib/guava-16.0.1.jar:/usr/share/dse/spark/lib/httpclient-4.4.1.jar:/ USR /共享/ DSE /火花/ LIB /的HttpCore-4.4.1.jar:在/ usr /共享/ DS E /火花/ LIB /常 Spring 藤2.4.0.jar:/usr/share/dse/spark/lib/jackson-annotations-2.3.5.jar:在/ usr /共享/ DSE /火花/ LIB / Jackson - 芯 - 2.3.5.jar:/usr/share/dse/spark/lib/jackson-core-asl-1.9.13.jar:/usr/share/dse/spark/lib/jackson-databind-2.3.5.jar: /usr/share/dse/spark/lib/jackson-mapper-asl-1.9.13.jar:/usr/share/dse/spark/lib/jackson-module-scala_2.10-2.3.5.jar:/usr /share/dse/spark/lib/jansi-1.4.jar:/usr/share/dse/spark/lib/JavaEWAH-0.3.2.jar:/usr/share/dse/spark/lib/javax.servlet-3.0 .0.v201112011016.jar:/usr/share/dse/spark/lib/javolution-5.5.1.jar:/usr/share/dse/spark/lib/jaxb-api-2.2.7.jar:在/ usr /共享/ DSE /火花/ LIB / JAXB核-2.2.7.jar:/usr/share/dse/spark/lib/jaxb-impl-2.2.7.jar:在/ usr /共享/ DSE /火花/ LIB / JDO-API-3.0.1.jar:/usr/share/dse/spark/lib/jersey-core-1.9.jar:/usr/share/dse/spark/lib/jersey-server-1.9.jar:在/ usr /share/dse/spark/lib/jets3t-0.7.1.jar:/usr/share/dse/spark/lib/jetty-all-7.6.0.v20120127.jar:/usr/share/dse/spark/lib /jetty-continuation-8.1.14.v20131031.jar:/usr/share/dse/spark/lib/je TTY-HTTP-8.1.14.v20131031.jar:/usr/share/dse/spark/lib/jetty-io-8.1.14.v20131031.jar:在/ usr /共享/ DSE /火花/ LIB /码头-安全 - 8.1.14.v20131031.jar:/usr/share/dse/spark/lib/jetty-server-8.1.14.v20131031.jar:/usr/share/dse/spark/lib/jetty-servlet-8.1.14 . v20131031.jar:/usr/share/dse/spark/lib/jetty-util-6.1.26.jar:/usr/share/dse/spark/lib/jetty-util-8.1.14.v20131031.jar:在/ usr /share/dse/spark/lib/jline-0.9.94.jar:/usr/share/dse/spark/lib/jline-2.10.5.jar:/usr/share/dse/spark/lib/joda-convert -1.2.jar:/usr/share/dse/spark/lib/joda-time-2.3.jar:/usr/share/dse/spark/lib/jodd-core-3.6.3.jar:在/ usr /共享/ DSE /火花/ lib目录/ jpam-1.1.jar:/usr/share/dse/spark/lib/json-20090211.jar:/usr/share/dse/spark/lib/json4s-ast_2.10-3.2.10 . jar :/usr/share/dse/spark/lib/json4s-core_2.10-3.2.10.jar:/usr/share/dse/spark/lib/json4s-jackson_2.10-3.2.10.jar:在/ usr /share/dse/spark/lib/jsr166e-1.1.0.jar:/usr/share/dse/spark/lib/jsr305-2.0.1.jar:/usr/share/dse/spark/lib/jta-1.1的.jar:/usr/share/dse/spark/lib/jtransforms-2.4.0.jar:/ U SR /共享/ DSE /火花/ LIB / KRYO-2.21.jar:/usr/share/dse/spark/lib/libfb303-0.9.3.jar:/usr/share/dse/spark/lib/libthrift-0.9 . 3.jar:/usr/share/dse/spark/lib/lz4-1.2.0.jar:/usr/share/dse/spark/lib/mail-1.4.1.jar:在/ usr /共享/ DSE /火花/lib/mesos-0.21.1-shaded-protobuf.jar:/usr/share/dse/spark/lib/metrics-core-3.1.2.jar:/usr/share/dse/spark/lib/metrics-graphite -3.1.2.jar:/usr/share/dse/spark/lib/metrics-json-3.1.2.jar:/usr/share/dse/spark/lib/metrics-jvm-3.1.2.jar:/ USR /股/ DSE /火花/ lib目录/ minlog-1.2.jar:/ UR /共享/ DSE /火花/ LIB / objenesis-1.2.jar:/usr/share/dse/spark/lib/opencsv-2.3.jar:/usr/share/dse/spark/lib/oro-2.0.8 . jar :/usr/share/dse/spark/lib/paranamer-2.6.jar:/usr/share/dse/spark/lib/parquet-column-1.6.0rc3.jar:在/ usr /共享/ DSE /火花/ lib目录/parquet-common-1.6.0rc3.jar:/usr/share/dse/spark/lib/parquet-encoding-1.6.0rc3.jar:/usr/share/dse/spark/lib/parquet-format-2.2.0 -rc1.jar:/usr/share/dse/spark/lib/parquet-generator-1.6.0rc3.jar:/usr/share/dse/spark/lib/parquet-hadoop-1.6.0rc3.jar:在/ usr /共享/ DSE /火花/ LIB /拼花的hadoop束-1.3.2.jar:/usr/share/dse/spark/lib/parquet-jackson-1.6.0rc3.jar:在/ usr /共享/ DSE /火花/ LIB / PMML剂-1.1.15.jar:/usr/share/dse/spark/lib/pmml-model-1.1.15.jar:/usr/share/dse/spark/lib/pmml-schema-1.1 . 15.jar:/usr/share/dse/spark/lib/protobuf-java-2.5.0-spark.jar:/usr/share/dse/spark/lib/py4j-0.8.1.jar:在/ usr /股/dse/spark/lib/pyrolite-4.4.jar:/usr/share/dse/spark/lib/quasiquotes_2.10-2.0.1.jar:/usr/share/dse/spark/lib/reflectasm-1.07-shaded的.jar:在/ usr /共享/ DSE /梁K / LIB / RoaringBitmap-0.4.5.jar:/usr/share/dse/spark/lib/scala-compiler-2.10.5.jar:/usr/share/dse/spark/lib/scala-library-2.10 . 5.jar:/usr/share/dse/spark/lib/scalap-2.10.5.jar:/usr/share/dse/spark/lib/scala-reflect-2.10.5.jar:在/ usr /共享/ DSE /spark/lib/servlet-api-2.5.jar:/usr/share/dse/spark/lib/slf4j-api-1.7.12.jar:/usr/share/dse/spark/lib/snappy-0.2.jar :/usr/share/dse/spark/lib/snappy-java-1.0.5.3.jar:/usr/share/dse/spark/lib/spark-bagel_2.10-1.4.2.5.jar:在/ usr /共享/ DSE /火花/ LIB /火花 Cassandra -connector_2.10-1.4.3.jar:/usr/share/dse/spark/lib/spark-cassandra-connector-java_2.10-1.4.3.jar:在/ usr /共享/ DSE /火花/ LIB /火花catalyst_2.10-1.4.2.5.jar:/usr/share/dse/spark/lib/spark-core_2.10-1.4.2.5.jar:在/ usr /共享/ DSE /火花/ LIB /火花graphx_2.10-1.4.2.5.jar:/usr/share/dse/spark/lib/spark-hive_2.10-1.4.2.5.jar:在/ usr /共享/ DSE /火花/ LIB /火花launcher_2.10-1.4.2.5.jar:/usr/share/dse/spark/lib/spark-mllib_2.10-1.4.2.5.jar:在/ usr /共享/ DSE /火花/ LIB /火花网络 - common_2.10-1.4.2.5.jar:在/ usr /共享/ DS E /火花/ LIB /火花网络shuffle_2.10-1.4.2.5.jar:/usr/share/dse/spark/lib/spark-repl_2.10-1.4.2.5.jar:在/ usr /共享/ DSE /火花/ LIB /火花sql_2.10-1.4.2.5.jar:/usr/share/dse/spark/lib/spark-streaming_2.10-1.4.2.5.jar:在/ usr /共享/ DSE /火花/ LIB /火花unsafe_2.10-1.4.2.5.jar:/usr/share/dse/spark/lib/spire_2.10-0.7.4.jar:/usr/share/dse/spark/lib/spire-macros_2.10- 0.7.4.jar:/usr/share/dse/spark/lib/ST4-4.0.4.jar:/usr/share/dse/spark/lib/stax-api-1.0.1.jar:在/ usr /股/dse/spark/lib/stream-2.7.0.jar:/usr/share/dse/spark/lib/stringtemplate-3.2.1.jar:/usr/share/dse/spark/lib/tachyon-0.6.4的.jar:/usr/share/dse/spark/lib/tachyon-client-0.6.4.jar:/usr/share/dse/spark/lib/uncommons-maths-1.2.2a.jar:在/ usr /共享/ DSE /火花/ lib中/未使用-1.0.0.jar:/usr/share/dse/spark/lib/velocity-1.7.jar:/usr/share/dse/spark/lib/xz-1.0.jar:在/ usr /share/dse/spark/lib/zookeeper-3.4.5.jar:/usr/share/dse/spark/lib/datanucleus-api-jdo-3.2.6.jar:/usr/share/dse/spark/lib /datanucleus-core-3.2.10.jar:/usr/share/dse/spark/lib/datanucleus-rd BMS-3.2.9.jar:/usr/share/dse/spark/lib/hive-0.13.1-cassandra-connector-0.2.14.jar:/usr/share/dse/spark/lib/hive-0.13- metastore- Cassandra 连接器-0.2.14.jar:/usr/share/dse/spark/lib/hive-beeline-0.13.1a.jar:/usr/share/dse/spark/lib/hive-cli-0.13 . 1a.jar:/usr/share/dse/spark/lib/hive-common-0.13.1a.jar:/usr/share/dse/spark/lib/hive-exec-0.13.1a.jar:在/ usr /股/dse/spark/lib/hive-hwi-0.13.1a.jar:/usr/share/dse/spark/lib/hive-jdbc-0.13.1a.jar:/usr/share/dse/spark/lib/hive -metastore-0.13.1a.jar:/usr/share/dse/spark/lib/hive-serde-0.13.1a.jar:/usr/share/dse/spark/lib/hive-service-0.13.1a.jar :/usr/share/dse/spark/lib/hive-shims-0.13.1a.jar:/usr/share/dse/spark/lib/hive-shims-0.20-0.13.1a.jar:在/ usr /共享/ DSE /火花/ LIB /蜂房垫片-0.20S-0.13.1a.jar:/usr/share/dse/spark/lib/hive-shims-0.23-0.13.1a.jar:在/ usr /共享/ DSE /火花/lib/hive-shims-common-0.13.1a.jar:/usr/share/dse/spark/lib/hive-shims-common-secure-0.13.1a.jar:/usr/share/dse/spark/lib /spark-hive-thriftserver_2.10-1.4.2.5.jar:/usr/share/dse/c ommon / ANTLR-2.7.7.jar:/usr/share/dse/common/antlr-3.2.jar:/usr/share/dse/common/antlr-runtime-3.2.jar:在/ usr /共享/ DSE /公共/aopalliance-1.0.jar:/usr/share/dse/common/api-asn1-api-1.0.0-M24.jar:/usr/share/dse/common/api-asn1-ber-1.0.0-M24的.jar:/usr/share/dse/common/api-i18n-1.0.0-M24.jar:/usr/share/dse/common/api-ldap-client-api-1.0.0-M24.jar:/ USR /共享/ DSE /普通/ API-LDAP的编解码器核-1.0.0-M24.jar:/usr/share/dse/common/api-ldap-codec-standalone-1.0.0-M24.jar:/ USR /股/ DSE /普通/ API-LDAP的额外-C癸1.0.0-M24.jar:/usr/share/dse/common/api-ldap-extras-codec-api-1.0.0-M24.jar:在/ usr /共享/ DSE /普通/ API-LDAP的模型-1.0.0-M24.jar:/usr/share/dse/common/api-ldap-net-mina-1.0.0-M24.jar:/usr/share/dse/common/api-util-1.0 . 0-M24.jar:/usr/share/dse/common/asm-5.0.3.jar:/usr/share/dse/common/commons-beanutils-1.7.0.jar:在/ usr /共享/ DSE /公共/commons-beanutils-core-1.8.0.jar:/usr/share/dse/common/commons-codec-1.9.jar:/usr/share/dse/common/commons-collections-3.2.2.jar:/ USR /股/ DSE /普通/公共编译-2.6.1.jar:/usr/share/dse/common/commons-configuration-1.6.jar:/usr/share/dse/common/commons-digester-1.8 . jar :/usr/share/dse/common/commons-io-2.4.jar:/usr/share/dse/common/commons-lang-2.6.jar:/usr/share/dse/common/commons-logging-1.1 .1.jar:/usr/share/dse/common/commons-pool-1.6.jar:/usr/share/dse/common/guava-16.0.1.jar:在/ usr /共享/ DSE /普通/ guice- 3.0.jar:/usr/share/dse/common/guice-multibindings-3.0.jar:/usr/share/dse/common/HdrHistogram-1.2.1.1.jar:在/ usr /共享/ DSE /普通/ Jackson - annotati ONS-2.2.2.jar:/usr/share/dse/common/jackson-core-2.2.2.jar:/usr/share/dse/common/jackson-databind-2.2.2.jar:在/ usr /股/dse/common/janino-2.6.1.jar:/usr/share/dse/common/javassist-3.18.2-GA.jar:/usr/share/dse/common/java-uuid-generator-3.1.3的.jar:/usr/share/dse/common/javax.inject-1.jar:/usr/share/dse/common/jbcrypt-0.4d.jar:在/ usr /共享/ DSE /普通/ JCL-过SLF4J -1.7.10.jar:/usr/share/dse/common/jline-1.0.jar:/usr/share/dse/common/journalio-1.4.2.jar:在/ usr /共享/ DSE /普通/ jsr305- 2.0.1.jar:/usr/share/dse/common/kmip-1.7.1e.jar:/usr/share/dse/common/log4j-1.2.13.jar:在/ usr /共享/ DSE /普通/米娜-core-2.0.7.jar:/usr/share/dse/common/reflections-0.9.10.jar:/usr/share/dse/common/slf4j-api-1.7.10.jar:在/ usr /共享/ DSE /普通/ StringTemplate的-3.2.jar:/usr/share/dse/common/validation-api-1.1.0.Final.jar:在/ etc / DSE /:在/ usr /共享/ DSE / hadoop的/中:/ etc / DSE / hadoop的/:在/ usr /共享/ DSE / hadoop的/ LIB /航空公司-0.6.jar:/usr/share/dse/hadoop/lib/ant-1.6.5.jar:在/ usr /共享/ DSE / hadoop的/ LIB /自动机-1.11-8.jar:在/ usr /共享/ DSE / hadoop的/ LIB /公地的BeanUtils-1.7.0.jar:/usr/share/dse/hadoop/lib/commons-beanutils-core-1.8.0.jar:/usr/share/dse/hadoop/lib/commons-cli-1.2 . jar :/usr/share/dse/hadoop/lib/commons-codec-1.4.jar:/usr/share/dse/hadoop/lib/commons-collections-3.2.2.jar:在/ usr /共享/ DSE / Hadoop的/lib/commons-configuration-1.6.jar:/usr/share/dse/hadoop/lib/commons-digester-1.8.jar:/usr/share/dse/hadoop/lib/commons-el-1.0.jar:/ USR /股/ DSE / Hadoop的/ lib目录/公地httpclient的-3.0.1.jar:/usr/share/dse/hadoop/lib/commons-lang-2.4.jar:在/ usr /共享/ DSE / Hadoop的/ lib目录/共享记录-1.1.1.jar:/usr/share/dse/hadoop/lib/commons-math-2.1.jar:/usr/share/dse/hadoop/lib/commons-net-1.4.1.jar: /usr/share/dse/hadoop/lib/core-3.1.1.jar:/usr/share/dse/hadoop/lib/elephant-bird-hadoop-compat-4.3.jar:/usr/share/dse/hadoop /lib/ftplet-api-1.0.0.jar:/usr/share/dse/hadoop/lib/ftpserver-core-1.0.0.jar:/usr/share/dse/hadoop/lib/ftpserver-deprecated-1.0的3.0 M2.jar:/usr/share/dse/hadoop/lib/hadoop-core-1.0.4.18.jar:/usr/share/dse/hadoop/lib/hadoop-examples-1.0.4.18的.jar:/usr/share/dse/hadoop/lib/hadoop-fairscheduler-1.0.4.18.jar:/usr/share/dse/hadoop/lib/hadoop-streaming-1.0.4.18.jar:在/ usr /共享/ DSE / hadoop的/ LIB / Hadoop的测试1.0.4.18.jar:/usr/share/dse/hadoop/lib/hadoop-tools-1.0.4.18.jar:在/ usr /共享/ DSE / hadoop的/ LIB / hsqldb- 1.8.0.10.jar:/usr/share/dse/hadoop/lib/jackson-core-asl-1.8.8.jar:/usr/share/dse/hadoop/lib/jackson-mapper-asl-1.8.8 . jar :/usr/share/dse/hadoop/lib/javax.inject-1.jar:/usr/share/dse/hadoop/lib/jets3t-0.7.1.jar:在/ usr /共享/ DSE / Hadoop的/ lib目录/jetty-6.1.26.jar:/usr/share/dse/hadoop/lib/jetty-util-6.1.26.jar:/usr/share/dse/hadoop/lib/jsp-2.1-6.1.14.jar :/usr/share/dse/hadoop/lib/jsp-api-2.1-6.1.14.jar:/usr/share/dse/hadoop/lib/kfs-0.3.jar:在/ usr /共享/ DSE / Hadoop的/ LIB / Mina的核心2.0.0-M5.jar:/usr/share/dse/hadoop/lib/netty-3.9.8.Final.jar:/usr/share/dse/hadoop/lib/oro-2.0 . 8.jar:/usr/share/dse/hadoop/lib/servlet-api-2.5-20081211.jar:/usr/share/dse/hadoop/lib/servlet-api-2.5-6.1.14.jar:在/ usr /share/dse/hadoop/lib/snappy-java-1.0.5.3.jar:/usr/share/dse/h adoop / LIB / xmlenc-0.52.jar:在/ usr /共享/ DSE / Cassandra /../驱动器/ LIB / Cassandra 驱动器芯-2.1.7.1.jar:在/ usr /共享/ DSE / Cassandra /../驱动器/ LIB / Cassandra -驾驶员DSE-2.1.7.1.jar:在/ usr /共享/ DSE / Cassandra /../驱动器/ LIB /度量核-3.0.2.jar:在/ usr /共享/ DSE / Cassandra /../driver/lib/slf4j-api-1.7.5.jar:/usr/share/dse/cassandra/lib/antlr-runtime-3.5.2.jar:/etc/dse/cassandra/:/usr/分享/ DSE / Cassandra / tools / lib目录/ stress.jar:/usr/share/dse/cassandra/lib/antlr-3.5.2.jar:在/ usr /共享/ DSE / Cassandra / lib目录/ Cassandra -ALL-21.14.1272.jar:/usr/share/dse/cassandra/lib/cassandra-clientutil-2.1.14.1272.jar:/usr/share/dse/cassandra/lib/cassandra-thrift-2.1.14.1272.jar:在/ usr /share/dse/cassandra/lib/commons-cli-1.1.jar:/usr/share/dse/cassandra/lib/commons-codec-1.9.jar:/usr/share/dse/cassandra/lib/commons-lang -2.6.jar:/usr/share/dse/cassandra/lib/commons-lang3-3.1.jar:/usr/share/dse/cassandra/lib/commons-logging-1.2.jar:在/ usr /共享/ DSE / Cassandra / lib目录/公地math3-3.2.jar:/usr/share/dse/cassandra/lib/compress-lzf-0.8.4.jar:/usr/share/dse/cassandra/lib/concurrentlinkedhashmap-lru-1.3 . jar :/usr/share/dse/cassandra/lib/disruptor-3.0.1.jar:/usr/share/dse/cassandra/lib/elephant-bird-hadoop-compat-4.3.jar:在/ usr /共享/ DSE /cassandra/lib/fastutil-6.5.7.jar:/usr/share/dse/cassandra/lib/guava-16.0.1.jar:/usr/share/dse/cassandra/lib/high-scale-lib-1.0 .6.jar:/usr/share/dse/cassandra/lib/httpclient-4.4.1.jar:/usr/share/dse/cassandra/lib/httpcore-4.4.1.jar:在/ usr /共享/ DSE / Cassandra / LIB / Jackson 核-ASL-1.9.2.jar:在/ usr /共享/ DSE / CASSA国家药品管理机构/ lib目录/ Jackson 映射器-ASL-1.9.2.jar:/usr/share/dse/cassandra/lib/jamm-0.3.0.jar:/usr/share/dse/cassandra/lib/jbcrypt-0.4d的.jar:/usr/share/dse/cassandra/lib/jna-4.0.0.jar:/usr/share/dse/cassandra/lib/joda-time-1.6.2.jar:在/ usr /共享/ DSE / Cassandra / lib目录/ JSON-简单1.1.jar:/usr/share/dse/cassandra/lib/libthrift-0.9.3.jar:/usr/share/dse/cassandra/lib/logback-classic-1.1.2 . jar :/usr/share/dse/cassandra/lib/logback-core-1.1.2.jar:/usr/share/dse/cassandra/lib/lz4-1.2.0.jar:在/ usr /共享/ DSE / Cassandra /lib/metrics-core-2.2.0.jar:/usr/share/dse/cassandra/lib/netty-all-4.0.34.Final.jar:/usr/share/dse/cassandra/lib/reporter-config -2.1.0.jar:/usr/share/dse/cassandra/lib/slf4j-api-1.7.12.jar:/usr/share/dse/cassandra/lib/snakeyaml-1.12.jar:在/ usr /共享/ DSE / Cassandra / lib目录/活泼-java的1.0.5.3.jar:/usr/share/dse/cassandra/lib/ST4-4.0.8.jar:/usr/share/dse/cassandra/lib/stream-2.5 . 2.jar:/usr/share/dse/cassandra/lib/super-csv-2.1.0.jar:/usr/share/dse/cassandra/lib/thrift-server-0.3.7.jar:在/ usr /股/ DSE /火花/ LIB / DA tanucleus-API JDO-3.2.6.jar:/usr/share/dse/spark/lib/datanucleus-rdbms-3.2.9.jar:/usr/share/dse/spark/lib/datanucleus-core-3.2 . 10.jar:/ etc / dse / hadoop /“” - Xms2048M“” - Xmx2048M“” - Ddse.system_memory_in_mb = 64264“” - Dcassandra.config.loader = com.datastax.bdp.config.DseConfigurationLoader“” - Dcassandra . logdir = / var / log / cassandra“” - ddse.client.configuration.impl = com.datastax.bdp.transport.client.HadoopBasedClientConfiguration“” - Dlogback.configurationFile = / etc / dse / spark / logback-spark-executor . xml“”org.apache.spark.DseSecureRunner“”org.apache.spark.executor.CoarseGrainedExecutorBackend“” - driver-url“”akka.tcp:// sparkDriver @ localhost:57972 / user / CoarseGrainedScheduler“” - executor -id“”4“” - hostname“”localhost“”--cores“”11“” - app-id“”app-20160923163206-0137“” - worker-url“”akka.tcp:// sparkWorker @ localhost:53730 / user / Worker“ERROR [SparkWorker-0错误 Logger ] 2016-09-23 16:40:10,696 SparkWorker-0 ExternalLogger.java:96 - 将流写入文件/ var / lib / spark / worker时出错/ Worker -0 / APP-2 0160923163007-0136 / 3 / stdout错误[SparkWorker-0错误 Logger ] 2016-09-23 16:40:10,696 SparkWorker-0 ExternalLogger.java:96 - java.io.IOException:流已关闭ERROR [SparkWorker-0错误 Logger ] 2016-09-23 16:40:10,696 SparkWorker-0 ExternalLogger.java:96 - 在java.io.BufferedInputStream.getBufIfOpen(BufferedInputStream.java:170)〜[na:1.8.0_92] ERROR [SparkWorker-0错误 Logger ] 2016-09-23 16:40:10,696 SparkWorker-0 ExternalLogger.java:96 - 在java.io.BufferedInputStream.read1(BufferedInputStream.java:283)〜[na:1.8.0_92] ERROR [SparkWorker-0错误 Logger ] 2016-09-23 16:40:10,696 SparkWorker-0 ExternalLogger.java:96 - 在java.io.BufferedInputStream.read(BufferedInputStream.java:345)〜[na:1.8.0_92] ERROR [SparkWorker-0错误 Logger ] 2016-09-23 16:40:10,696 SparkWorker-0 ExternalLogger.java:96 - 在java.io.FilterInputStream.read(FilterInputStream.java:107)〜[na:1.8.0_92] ERROR [SparkWorker-0错误 Logger ] 2016-09-23 16:40:10,696 SparkWorker-0 ExternalLogger.java:96 - 在org.apache.spark . util.logging.FileAppender.appendStreamToFile(FileAppender.scala:70)〜[spark-core_2.10-1.4.2.5.jar:1.4.2.5] ERROR [SparkWorker-0错误 Logger ] 2016-09-23 16:40:10,696 SparkWorker-0 ExternalLogger.java:96 - atg.apache.spark.util.logging.FileAppender $$ anon $ 1 $$ anonfun $ run $ 1.apply $ mcV $ sp(FileAppender.scala:39)[spark-core_2.10 -1.4.2.5.jar:1.4.2.5]错误[SparkWorker-0错误 Logger ] 2016-09-23 16:40:10,696 SparkWorker-0 ExternalLogger.java:96 - atorg.apache.spark.util.logging.FileAppender $$ anon $ 1 $$ anonfun $ run $ 1.apply(FileAppender.scala:39)[spark-core_2.10-1.4.2.5.jar:1.4.2.5] ERROR [SparkWorker -0 error logger] 2016-09-23 16:40:10,697 SparkWorker-0 ExternalLogger.java:96 - atg.apache.spark.util.logging.FileAppender $$ anon $ 1 $$ anonfun $ run $ 1.apply(FileAppender .scala:39)[spark-core_2.10-1.4.2.5.jar:1.4.2.5]错误[SparkWorker-0错误 Logger ] 2016-09-23 16:40:10,697 SparkWorker-0 ExternalLogger.java:96 - at org.apache.spark.util.Utils $ .logUncaughtExceptions(Utils.scala:1652)[spark-core_2.10-1.4.2.5.jar:1.4.2.5]错误[SparkWorker-0错误 Logger ] 2016-09-23 16 :40:10,697 SparkWorker-0 ExternalLogger.java:96 - at org.apache.spark.util.logging.FileAppender $$ anon $ 1.run(FileAppender.scala:38)[spark-core_2.10-1.4.2.5.jar :1.4.2.5]

此外,将自动生成一个以java_1464320280.hprof命名的大文件 . 这个文件的大小太大,以至于系统磁盘空间不足,最终会让cassandra节点失效 .