我有一个Java应用程序输出格式的日志
时间戳UUID1的一些信息
时间戳UUID1更多信息
时间戳UUID1 x = 1
时间戳UUID2的一些信息
时间戳UUID2更多信息
时间戳UUID2 x = 2
时间戳UUID3的一些信息
时间戳UUID3更多信息
时间戳UUID3 x = 1
我想使用Elsatic Search,LogStash和Kibana实现日志分析框架 . 是否可以仅根据X值获取日志?
例如:-
如果我查询X = 1,我应该只得到以下日志 .
时间戳UUID1的一些信息
时间戳UUID1更多信息
时间戳UUID1 x = 1
时间戳UUID3的一些信息
时间戳UUID3更多信息
时间戳UUID3 x = 1
如果我查询X = 2,我应该只得到以下日志 .
时间戳UUID2的一些信息
时间戳UUID2更多信息
时间戳UUID2 x = 2
我控制着日志消息格式 . 如果不能直接进行此查询,我也可以更改消息格式 .
更新1:
我会更具体一点 .
以下是我的日志声明 .
MDC.put("uuid", UUID.randomUUID().toString());
logger.info("Assigning value to the variable : {}", name);
this.setVal(value.getVal());
logger.info("{} = {}", name, value.getVal());
logger.info("Assigned value {} to the variable : {}", value.getVal(),
name);
MDC.clear();
我使用UDP在Logstash中收到了日志语句 . 我收到的消息就像 .
{
"@timestamp" => "2015-04-01T10:23:37.846+05:30",
"@version" => 1,
"message" => "Assigning value to the variable : X",
"logger_name" => "com.example.logstash.Variable",
"thread_name" => "pool-1-thread-1",
"level" => "INFO",
"level_value" => 20000,
"HOSTNAME" => "pnibinkj-W7-1",
"uuid" => "ab17b842-8348-4474-98e4-8bc2b8dd6781",
"host" => "127.0.0.1"
}
{
"@timestamp" => "2015-04-01T10:23:37.846+05:30",
"@version" => 1,
"message" => "Assigning value to the variable : Y",
"logger_name" => "com.example.logstash.Variable",
"thread_name" => "pool-1-thread-2",
"level" => "INFO",
"level_value" => 20000,
"HOSTNAME" => "pnibinkj-W7-1",
"uuid" => "d5513e4c-de3b-4144-87e4-87b077ac8056",
"host" => "127.0.0.1"
}
{
"@timestamp" => "2015-04-01T10:23:37.862+05:30",
"@version" => 1,
"message" => "Y = 1",
"logger_name" => "com.example.logstash.Variable",
"thread_name" => "pool-1-thread-2",
"level" => "INFO",
"level_value" => 20000,
"HOSTNAME" => "pnibinkj-W7-1",
"uuid" => "d5513e4c-de3b-4144-87e4-87b077ac8056",
"host" => "127.0.0.1"
}
{
"@timestamp" => "2015-04-01T10:23:37.863+05:30",
"@version" => 1,
"message" => "X = 1",
"logger_name" => "com.example.logstash.Variable",
"thread_name" => "pool-1-thread-1",
"level" => "INFO",
"level_value" => 20000,
"HOSTNAME" => "pnibinkj-W7-1",
"uuid" => "ab17b842-8348-4474-98e4-8bc2b8dd6781",
"host" => "127.0.0.1"
}
{
"@timestamp" => "2015-04-01T10:23:37.863+05:30",
"@version" => 1,
"message" => "Assigned value 1 to the variable : X",
"logger_name" => "com.example.logstash.Variable",
"thread_name" => "pool-1-thread-1",
"level" => "INFO",
"level_value" => 20000,
"HOSTNAME" => "pnibinkj-W7-1",
"uuid" => "ab17b842-8348-4474-98e4-8bc2b8dd6781",
"host" => "127.0.0.1"
}
{
"@timestamp" => "2015-04-01T10:23:37.863+05:30",
"@version" => 1,
"message" => "Assigned value 1 to the variable : Y",
"logger_name" => "com.example.logstash.Variable",
"thread_name" => "pool-1-thread-2",
"level" => "INFO",
"level_value" => 20000,
"HOSTNAME" => "pnibinkj-W7-1",
"uuid" => "d5513e4c-de3b-4144-87e4-87b077ac8056",
"host" => "127.0.0.1"
}
有2个UUID
"d5513e4c-de3b-4144-87e4-87b077ac8056" for "Y = 1"
"ab17b842-8348-4474-98e4-8bc2b8dd6781" for "X = 1"
每个UUID还有两条消息 . 我想把它们组合成一个单一的事件 .
我不确定,如何为这种情况编写多行过滤器 .
filter {
multiline {
pattern => "."
what => "previous"
stream_identity => "%{uuid}"
}
}
看来,“模式”和“什么”是必填字段 . 我应该为这些领域提供什么 . 我如何使用Stream Identity?
请指出我正确的方向 .
谢谢,保罗
2 回答
您需要组合您的消息(请参阅multiline {}过滤器,它支持stream_identity),然后常规查询将返回相应的消息 .
如果X是一些唯一值,那么使用kibana过滤器应该是可能的,但是如果显示所示格式的日志,则需要使用多行过滤器将条目连接在一起 .
有了它,您可能会使用类似的查询