Flume Version 1.8.0 Elasticsearch 5.0.0 Java Version 1.8.0_101
我在/ flume / lib中放了elasticserach-5.0.0.jar和lucene-core-5.0.0.jar
这是我的flume配置脚本
a1.channels = ch1 a1.sources = r1 a1.sinks = s1 a1.channels.ch1.type = memory a1.channels.ch1.capacity = 1000 a1.channels.ch1.transactionCapacity = 1000 a1.channels.ch1.keep -alive = 30 a1.sources.r1.type = exec a1.sources.r1.shell = / bin / bash -c a1.sources.r1.command = tail -n 0 -F / mnt / echat-log / info / echat_old / echat_third / echat.log.2018-02-07 a1.sources.r1.channels = ch1 a1.sources.r1.threads = 5 a1.sources.r1.restartThrottle = 100000 a1.sources.r1.restart = true a1 .sources.r1.logStdErr = true a1.sinks.s1.channel = ch1 a1.sinks.s1.type = elasticsearch a1.sinks.s1.hostNames = 192.168.1.4:9200 a1.sinks.s1.indexName = foo_index a1 . sinks.s1.indexType = bar_type a1.sinks.s1.batchSize = 500 a1.sinks.s1.serializer = org.apache.flume.sink.elasticsearch.ElasticSearchDynamicSerializer
这是错误消息：
信息：包括通过（/ usr / local / hadoop / bin / hadoop）找到的Hadoop库用于HDFS访问信息：包括通过（）为Hive访问找到的Hive库exec /usr/local/jdk1.8.0_101/bin/java - Xmx20m -Dflume.root.logger = INFO，console -cp'/mnt/flume_Outer_1.8/conf:/mnt/flume_Outer_1.8/lib/:/usr/local/hadoop-2.6.5/etc/hadoop:/usr /local/hadoop-2.6.5/share/hadoop/common/lib/:/usr/local/hadoop-2.6.5/share/hadoop/common/:/usr/local/hadoop-2.6.5/share/hadoop /hdfs:/usr/local/hadoop-2.6.5/share/hadoop/hdfs/lib/:/usr/local/hadoop-2.6.5/share/hadoop/hdfs/:/usr/local/hadoop-2.6 . 5 /共享/ hadoop的/纱线/ LIB /：在/ usr /本地/ Hadoop的2.6.5 /共享/ hadoop的/纱线/：在/ usr /本地/ Hadoop的2.6.5 /共享/ hadoop的/映射精简/ LIB /：/ usr / local / hadoop-2.6.5 / share / hadoop / mapreduce /：/ usr / local / hadoop / contrib / capacity-scheduler / .jar：/ lib / *' - Djava.library.path =：/ usr / java / packages / lib / amd64：/ usr / lib64：/ lib64：/ lib：/ usr / lib org.apache.flume.node.Application -f conf / infotest -n a1 SLF4J：类路径包含多个SLF4J绑定 . SLF4J：在[jar：file：/mnt/flume_Outer_1.8/lib/slf4j-log4j12-1.6.1.jar！/org/slf4j/impl/StaticLoggerBinder.class]中找到绑定SLF4J：在[jar：file：中找到绑定： /usr/local/hadoop-2.6.5/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J：见http：//www.slf4j .org / codes.html＃multiple_bindings的解释 . 18/02/08 18:07:49 INFO node.PollingPropertiesFileConfigurationProvider：配置提供程序开始于18/02/08 18:07:49 INFO node.PollingPropertiesFileConfigurationProvider：重新加载配置文件：conf / infotest 18/02/08 18:07:49 INFO conf.FlumeConfiguration：Processing：s1 18/02/08 18:07:49 INFO conf.FlumeConfiguration：Processing：s1 18/02/08 18:07:49 INFO conf.FlumeConfiguration：Processing：s1 18/02/08 18 ：07：49 INFO conf.FlumeConfiguration：处理：s1 18/02/08 18:07:49 INFO conf.FlumeConfiguration：添加了sinks：s1代理：a1 18/02/08 18:07:49 INFO conf.FlumeConfiguration：Processing ：s1 18/02/08 18:07:49 INFO conf.FlumeConfiguration：Processing：s1 18/02/08 18:07:49 INFO conf.FlumeConfiguration：Processing：s1 18/02/08 18:07:49 INFO conf .FlumeConfiguration：验证后的水槽配置包含代理的配置：[a1] 18/02/08 18:07:49 INFO node.AbstractConfigurationProvider：创建 Channels 18/02/08 18:07:49 INFO channel.DefaultChannelFactory：创建实例通道ch1类型内存18/02/08 18:07:49 INFO node.AbstractConfigurationProvider：创建通道ch1 18/02/08 18:07:49 INFO source.DefaultSourceFactory：创建源r1的实例，输入exec 18/02 / 08 18:07:49 INFO sink.DefaultSinkFactory：创建接收器的实例：s1，类型：elasticsearch 18/02/08 18:07:49 INFO node.AbstractConfigurationProvider：通道ch1连接到[r1，s1] 18/02/08 18:07:49 INFO node.Application：开始新配置：{sourceRunners：{r1 = EventDrivenSourceRunner：{source：org.apache.flume.source.ExecSource {name：r1，state：IDLE}}} sinkRunners：{s1 = SinkRunner：{policy：org.apache.flume.sink.DefaultSinkProcessor@6add9eb counterGroup：{name：null counters：{}}}} channels：{ch1 = org.apache.flume.channel.MemoryChannel {name：ch1}}} 18/02/08 18:07:49 INFO node.Application：Starting Channel ch1 18/02/08 18:07:49 INFO instrumentation.MonitoredCounterGroup：监控的计数器组类型：CHANNEL，name：ch1：已成功注册新MBean . 18/02/08 18:07:49 INFO instrumentation.MonitoredCounterGroup：组件类型：CHANNEL，name：ch1 started 18/02/08 18:07:49 INFO node.Application：Starting Sink s118/02/08 18:07:49 INFO node.Application：Starting Source r1 18/02/08 18:07:49 INFO source.ExecSource：exec source from command：tail -n 0 -F / mnt / echat- log / info / echat_old / echat_third / echat.log.2018-02-07 18/02/08 18:07:49 INFO instrumentation.MonitoredCounterGroup：监视的类型的计数器组：SOURCE，name：r1：成功注册了新的MBean . 18/02/08 18:07:49 INFO instrumentation.MonitoredCounterGroup：组件类型：SOURCE，名称：r1开始于18/02/08 18:07:49 INFO elasticsearch.ElasticSearchSink：ElasticSearch sink {}开始于18/02/08 18 ：07：49 INFO instrumentation.MonitoredCounterGroup：类型的受监视计数器组：SINK，name：s1：已成功注册新MBean . 18/02/08 18:07:49 INFO instrumentation.MonitoredCounterGroup：组件类型：SINK，名称：s1开始于18/02/08 18:07:49 WARN client.ElasticSearchTransportClient：[192.168.1.4：9200] 18/02 / 08 18:07:49 ERROR lifecycle.LifecycleSupervisor：无法启动SinkRunner：{policy：org.apache.flume.sink.DefaultSinkProcessor@6add9eb counterGroup：{name：null counters：{}}} - 以下是例外 . . java.lang.NoSuchMethodError：org.elasticsearch.common.transport.InetSocketTransportAddress（Ljava /郎/字符串; I）在org.apache.flume.sink.elasticsearch.client.ElasticSearchTransportClient.configureHostnames（ElasticSearchTransportClient.java:141）在V org.apache.flume.sink.elasticsearch.client.ElasticSearchTransportClient . （ElasticSearchTransportClient.java:77）在org.apache.flume.sink.elasticsearch.client.ElasticSearchClientFactory.getClient（ElasticSearchClientFactory.java:48）在org.apache.flume .sink.elasticsearch.ElasticSearchSink.start（ElasticSearchSink.java:358）在org.apache.flume.sink.DefaultSinkProcessor.start（DefaultSinkProcessor.java:45）在org.apache.flume.SinkRunner.start（SinkRunner.java:79 ）在org.apache.flume.lifecycle.LifecycleSupervisor $ MonitorRunnable.run（LifecycleSupervisor.java:249）在java.util.concurrent.Executors $ RunnableAdapter.call（Executors.java:511）在java.util.concurrent.FutureTask中 . runAndReset（FutureTask.java:308）at java.util.concurrent.ScheduledThreadPoolExec尤特$ ScheduledFutureTask.access $ 301（ScheduledThreadPoolExecutor.java:180）在java.util.concurrent.ScheduledThreadPoolExecutor中$ ScheduledFutureTask.run（ScheduledThreadPoolExecutor.java:294）在java.util.concurrent.ThreadPoolExecutor.runWorker（ThreadPoolExecutor.java:1142）在java.util.concurrent.ThreadPoolExecutor中的$ Worker.run（ThreadPoolExecutor.java:617）在java.lang.Thread.run（Thread.java:745）18/02/08 18时07分49秒INFO elasticsearch.ElasticSearchSink：ElasticSearch片{}停止18/02/08 18:07:49 INFO instrumentation.MonitoredCounterGroup：组件类型：SINK，名称：s1已停止18/02/08 18:07:49 INFO instrumentation.MonitoredCounterGroup：Shutdown指标类型：SINK，name ：s1 . sink.start.time == 1518084469218 18/02/08 18:07:49 INFO instrumentation.MonitoredCounterGroup：Shutdown Metric for type：SINK，name：s1 . sink.stop.time == 1518084469230 18/02/08 18:07:49 INFO instrumentation.MonitoredCounterGroup：Shutdown Metric for type：SINK，name：s1 . sink.batch.complete == 0 18/02/08 18:07:49 INFO instrumentation.MonitoredCounterGroup：Shutdown Metric for type：SINK，name：s1 . sink.batch.empty == 0 18/02/08 18:07:49 INFO instrumentation.MonitoredCounterGroup：Shutdown Metric for type：SINK，name：s1 . sink.batch.underflow == 0 18/02/08 18:07:49 INFO instrumentation.MonitoredCounterGroup：Shutdown Metric for type：SINK，name：s1 . sink.connection.closed.count == 1 18/02/08 18:07:49 INFO instrumentation.MonitoredCounterGroup：Shutdown Metric for type：SINK，name：s1 . sink.connection.creation.count == 0 18/02/08 18:07:49 INFO instrumentation.MonitoredCounterGroup：Shutdown Metric for type：SINK，name：s1 . sink.connection.failed.count == 0 18/02/08 18:07:49 INFO instrumentation.MonitoredCounterGroup：Shutdown Metric for type：SINK，name：s1 . sink.event.drain.attempt == 0 18/02/08 18:07:49 INFO instrumentation.MonitoredCounterGroup：Shutdown Metric for type：SINK，name：s1 . sink.event.drain.sucess == 0 18/02/08 18:07:49 WARN lifecycle.LifecycleSupervisor：Component SinkRunner：{policy：org.apache.flume.sink.DefaultSinkProcessor@6add9eb counterGroup：{name：null counters： {}}}已停止，因为由于缺少依赖项而无法成功启动18/02/08 18:08:19错误source.ExecSource：运行命令时失败：tail -n 0 -F / mnt / echat-log / info /echat_old/echat_third/echat.log.2018-02-07 org.apache.flume.ChannelFullException：提交队列的空间不能是收购 . 在org.apache.flume.channel.BasicTransactionSemantics.commit（org.apache.flume.channel.BasicTransactionSemantics.commit）上，org.apache.flume.channel.MemoryChannel $ MemoryTransaction.doCommit（MemoryChannel.java:128）中的接收器可能无法跟上源，或者缓冲区大小太紧 . BasicTransactionSemantics.java:151）atg.apache.flume.channel.ChannelProcessor.processEventBatch（ChannelProcessor.java:194）org.apache.flume.source.ExecSource $ ExecRunnable.flushEventBatch（ExecSource.java:378）org.apache .flume.source.ExecSource $ ExecRunnable.run（ExecSource.java:338）java.util.concurrent.Executors $ RunnableAdapter.call（Executors.java:511）at java.util.concurrent.FutureTask.run（FutureTask.java） ：266）java.util.concurrent.ThreadPoolExecutor.runWorker（ThreadPoolExecutor.java:1142）at java.util.concurrent.ThreadPoolExecutor $ Worker.run（ThreadPoolExecutor.java:617）at java.lang.Thread.run（Thread . java：745）18/02/08 18:08:19错误source.ExecSource：处理事件批处理时发生异常org.apache.flume.ChannelException：java.lang.I Org.apache.flume.channel.BasicTransactionSemantics.commit（BasicTransactionSemantics.java:154）中的orter.apache.flume.channel.ChannelProcessor.processEventBatch（ChannelProcessor.java:194）中的orter.apache.flume.source.ExecSource $的nterruptedException ExecRunnable.flushEventBatch（ExecSource.java:378）org.apache.flume.source.ExecSource $ ExecRunnable.access $ 100（ExecSource.java:251）org.apache.flume.source.ExecSource $ ExecRunnable $ 1.run（ExecSource . java：320）at java.util.concurrent.Executors $ RunnableAdapter.call（Executors.java:511）at java.util.concurrent.FutureTask.runAndReset（FutureTask.java:308）at java.util.concurrent.ScheduledThreadPoolExecutor $ ScheduledFutureTask .access $ 301（ScheduledThreadPoolExecutor.java:180）java.util.concurrent.ScheduledThreadPoolExecutor $ ScheduledFutureTask.run（ScheduledThreadPoolExecutor.java:294）at java.util.concurrent.ThreadPoolExecutor.runWorker（ThreadPoolExecutor.java:1142）at java.util .concurrent.ThreadPoolExecutor $ Worker.run（ThreadPoolExecutor.java:617）在java.lang.Thread.run（Thread.java:745）引起：java.util.concurrent.locks中的java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos（AbstractQueuedSynchronizer.java:1039）中的java.lang.InterruptedException .AbstractQueuedSynchronizer.tryAcquireSharedNanos（AbstractQueuedSynchronizer.java:1328）位于org.apache.flume.channel.MemoryChannel的java.util.concurrent.Semaphore.tryAcquire（Semaphore.java:582）$ MemoryTransaction.doCommit（MemoryChannel.java:126）at at org.apache.flume.channel.BasicTransactionSemantics.commit（BasicTransactionSemantics.java:151）... 11更多
我是否下载了错误的jar文件？或者配置文件出了什么问题？这对我来说有点困难因为我不太熟悉java的东西 . 更糟糕的是，我是非母语英语的人，我们的社区还不够积极地找到一些可以解决我问题的博客 .