首页 文章

如何检查kafka机器上的损坏/坏文件

提问于
浏览
0

我们想检查或验证 /var/kafka/kafka-logs/ 下的日志,以了解哪些文件已损坏或缺少信息......等 .

kafka version - 10.0.2

是否可以通过CLI或其他方法验证日志?

这样做的原因,我的问题是因为我们从 /var/log/kafka/server.log 与索引和timeindexes问题(损坏指标)的通知,我们担心,如果这些文件也是问题的一部分 .

ls -ltr /var/kafka-kafka-logs

drwxr-xr-x. 2 kafka hadoop 16384 May 15 14:22 kuapi.jurg.pri.processed-1
drwxr-xr-x. 2 kafka hadoop 20480 May 15 14:22 kuapi.jurg.pri.decoded-95
drwxr-xr-x. 2 kafka hadoop 16384 May 15 14:22 kuapi.jurg.pri.decoded-36
drwxr-xr-x. 2 kafka hadoop 16384 May 15 14:22 kuapi.jurg.pri.processed-53
drwxr-xr-x. 2 kafka hadoop 20480 May 15 14:22 kuapi.jurg.pri.decoded-44
drwxr-xr-x. 2 kafka hadoop 20480 May 15 14:22 kuapi.jurg.pri.processed-17
drwxr-xr-x. 2 kafka hadoop 20480 May 15 14:22 kuapi.jurg.pri.decoded-83
drwxr-xr-x. 2 kafka hadoop 20480 May 15 14:22 kuapi.jurg.pri.decoded-15
drwxr-xr-x. 2 kafka hadoop 16384 May 15 14:22 kuapi.jurg.pri.processed-66
drwxr-xr-x. 2 kafka hadoop  4096 May 15 14:22 kuapi.jurg.tc.ppe_retry.gad-0
drwxr-xr-x. 2 kafka hadoop  4096 May 15 14:22 kuapi.jurg.tc.ppe_retry.wce-2
drwxr-xr-x. 2 kafka hadoop  4096 May 15 14:22 kuapi.jurg.tc.ppe_retry.wce-3
drwxr-xr-x. 2 kafka hadoop  4096 May 15 14:22 kuapi.jurg.tc.ppe_mission.imei-4
drwxr-xr-x. 2 kafka hadoop  4096 May 15 14:22 kuapi.jurg.pri.tc.initial.nce-4
drwxr-xr-x. 2 kafka hadoop  4096 May 15 14:22 kuapi.jurg.tc.ppe_update.wce-5
drwxr-xr-x. 2 kafka hadoop  4096 May 15 14:22 __consumer_offsets-14
drwxr-xr-x. 2 kafka hadoop 16384 May 15 14:22 kuapi.jurg.pri.decoded-55
drwxr-xr-x. 2 kafka hadoop 28672 May 15 14:22 kuapi.jurg.mdr.initial-1
drwxr-xr-x. 2 kafka hadoop 16384 May 15 14:22 kuapi.jurg.pri.processed-79
drwxr-xr-x. 2 kafka hadoop 16384 May 15 14:22 kuapi.jurg.pri.decoded-8

主题下的示例 - kuapi.jurg.pri.decoded-8 我们有:

ls kuapi.jurg.pri.decoded-8
00000000000000000000.index  00000000000000000000.log  00000000000000000000.timeindex  00000000000000000008.index  00000000000000000008.log  00000000000000000008.timeindex

要么

-rw-r--r--  1 kafka hadoop 1073738298 May  1 10:37 00000000000134399406.log
-rw-r--r--  1 kafka hadoop 1073738600 May  1 20:36 00000000000135316397.log
-rw-r--r--  1 kafka hadoop 1073739285 May  2 11:04 00000000000136250016.log
-rw-r--r--  1 kafka hadoop 1073735132 May  2 21:53 00000000000137161280.log
-rw-r--r--  1 kafka hadoop 1073740561 May  3 11:44 00000000000138080865.log
-rw-r--r--  1 kafka hadoop 1073739395 May  4 02:53 00000000000138994373.log
-rw-r--r--  1 kafka hadoop 1073741803 May  4 09:33 00000000000139934457.log
-rw-r--r--  1 kafka hadoop 1073737793 May  4 20:57 00000000000140876452.log
-rw-r--r--  1 kafka hadoop 1073727455 May  5 12:16 00000000000141818227.log
-rw-r--r--  1 kafka hadoop 1073741783 May  8 16:42 00000000000142754084.log
-rw-r--r--  1 kafka hadoop  471017673 May  8 19:32 00000000000143708830.log

1 回答

  • 0

    我'm not sure how much you'将要离开它,但你使用DumpLogSegments util检查日志段和索引

    例如,这会吐出一些输出

    kafka-run-class kafka.tools.DumpLogSegments --files 00000000000000000000.log
    

相关问题