I'm writing a KQL to build Kibana Visualize. I've build a query to find my expected result but it's not perfect.
Points to be noted -
- Data is logger messages, not json
- I searched a lot but most of answers and stackoverflow suggestions were for json data
- My queries are on "message" field
Expected Result - All log messages which have noOfFlexJobs>2
Here is my Query-1 -
message:"thread=getEmployeeNumbersForFlexJob-2b80ac18-d97c-4af0-a14b-53b14f1bbc32" and message:"noOfFlexJobs=" Query-1 Result -
Time message Jan 28, 2021 @ 09:20:14.503 2021-01-28T09:20:14.503-0600 level=INFO, thread=getEmployeeNumbersForFlexJob-2b80ac18-d97c-4af0-a14b-53b14f1bbc32, cat=EmployeeJobsScript, [] msg=Employee jobs data for scriptName=getEmployeeNumbersForFlexJob scriptStatus=RUNNING executionId=2b80ac18-d97c-4af0-a14b-53b14f1bbc32 EmployeeNumber=A1480876, noOfJobPrefs=0, noOfFlexJobs=0 Jan 28, 2021 @ 09:20:14.486 2021-01-28T09:20:14.486-0600 level=INFO, thread=getEmployeeNumbersForFlexJob-2b80ac18-d97c-4af0-a14b-53b14f1bbc32, cat=EmployeeJobsScript, [] msg=Employee jobs data for scriptName=getEmployeeNumbersForFlexJob scriptStatus=RUNNING executionId=2b80ac18-d97c-4af0-a14b-53b14f1bbc32 EmployeeNumber=a787754, noOfJobPrefs=0, noOfFlexJobs=1 Jan 28, 2021 @ 09:20:14.470 2021-01-28T09:20:14.470-0600 level=INFO, thread=getEmployeeNumbersForFlexJob-2b80ac18-d97c-4af0-a14b-53b14f1bbc32, cat=EmployeeJobsScript, [] msg=Employee jobs data for scriptName=getEmployeeNumbersForFlexJob scriptStatus=RUNNING executionId=2b80ac18-d97c-4af0-a14b-53b14f1bbc32 EmployeeNumber=A1478669, noOfJobPrefs=0, noOfFlexJobs=1 Jan 28, 2021 @ 09:20:14.454 2021-01-28T09:20:14.454-0600 level=INFO, thread=getEmployeeNumbersForFlexJob-2b80ac18-d97c-4af0-a14b-53b14f1bbc32, cat=EmployeeJobsScript, [] msg=Employee jobs data for scriptName=getEmployeeNumbersForFlexJob scriptStatus=RUNNING executionId=2b80ac18-d97c-4af0-a14b-53b14f1bbc32 EmployeeNumber=A1478668, noOfJobPrefs=0, noOfFlexJobs=0 Jan 28, 2021 @ 09:20:14.443 2021-01-28T09:20:14.443-0600 level=INFO, thread=getEmployeeNumbersForFlexJob-2b80ac18-d97c-4af0-a14b-53b14f1bbc32, cat=EmployeeJobsScript, [] msg=Employee jobs data for scriptName=getEmployeeNumbersForFlexJob scriptStatus=RUNNING executionId=2b80ac18-d97c-4af0-a14b-53b14f1bbc32 EmployeeNumber=A1278828, noOfJobPrefs=0, noOfFlexJobs=3 Jan 28, 2021 @ 09:20:14.418 2021-01-28T09:20:14.418-0600 level=INFO, thread=getEmployeeNumbersForFlexJob-2b80ac18-d97c-4af0-a14b-53b14f1bbc32, cat=EmployeeJobsScript, [] msg=Employee jobs data for scriptName=getEmployeeNumbersForFlexJob scriptStatus=RUNNING executionId=2b80ac18-d97c-4af0-a14b-53b14f1bbc32 EmployeeNumber=A1472766, noOfJobPrefs=0, noOfFlexJobs=4 Jan 28, 2021 @ 09:20:14.391 2021-01-28T09:20:14.391-0600 level=INFO, thread=getEmployeeNumbersForFlexJob-2b80ac18-d97c-4af0-a14b-53b14f1bbc32, cat=EmployeeJobsScript, [] msg=Employee jobs data for scriptName=getEmployeeNumbersForFlexJob scriptStatus=RUNNING executionId=2b80ac18-d97c-4af0-a14b-53b14f1bbc32 EmployeeNumber=A1478985, noOfJobPrefs=0, noOfFlexJobs=5 Jan 28, 2021 @ 09:20:14.380 2021-01-28T09:20:14.379-0600 level=INFO, thread=getEmployeeNumbersForFlexJob-2b80ac18-d97c-4af0-a14b-53b14f1bbc32, cat=EmployeeJobsScript, [] msg=Employee jobs data for scriptName=getEmployeeNumbersForFlexJob scriptStatus=RUNNING executionId=2b80ac18-d97c-4af0-a14b-53b14f1bbc32 EmployeeNumber=A1472442, noOfJobPrefs=0, noOfFlexJobs=11 Jan 28, 2021 @ 09:20:14.357 2021-01-28T09:20:14.357-0600 level=INFO, thread=getEmployeeNumbersForFlexJob-2b80ac18-d97c-4af0-a14b-53b14f1bbc32, cat=EmployeeJobsScript, [] msg=Employee jobs data for scriptName=getEmployeeNumbersForFlexJob scriptStatus=RUNNING executionId=2b80ac18-d97c-4af0-a14b-53b14f1bbc32 EmployeeNumber=A1502372, noOfJobPrefs=0, noOfFlexJobs=0 Jan 28, 2021 @ 09:20:14.352 2021-01-28T09:20:14.352-0600 level=INFO, thread=getEmployeeNumbersForFlexJob-2b80ac18-d97c-4af0-a14b-53b14f1bbc32, cat=EmployeeJobsScript, [] msg=Employee jobs data for scriptName=getEmployeeNumbersForFlexJob scriptStatus=RUNNING executionId=2b80ac18-d97c-4af0-a14b-53b14f1bbc32 EmployeeNumber=A1477010, noOfJobPrefs=0, noOfFlexJobs=0 Jan 28, 2021 @ 09:20:14.342 2021-01-28T09:20:14.342-0600 level=INFO, thread=getEmployeeNumbersForFlexJob-2b80ac18-d97c-4af0-a14b-53b14f1bbc32, cat=EmployeeJobsScript, [] msg=Employee jobs data for scriptName=getEmployeeNumbersForFlexJob scriptStatus=RUNNING executionId=2b80ac18-d97c-4af0-a14b-53b14f1bbc32 EmployeeNumber=A1467206, noOfJobPrefs=0, noOfFlexJobs=16 To get desired result I have updated my query- Query-2
message:"thread=getEmployeeNumbersForFlexJob-2b80ac18-d97c-4af0-a14b-53b14f1bbc32" and (message:"noOfFlexJobs=3" or message:"noOfFlexJobs=4" or message:"noOfFlexJobs=5") Query-2 Result
Time message Jan 28, 2021 @ 09:20:14.443 2021-01-28T09:20:14.443-0600 level=INFO, thread=getEmployeeNumbersForFlexJob-2b80ac18-d97c-4af0-a14b-53b14f1bbc32, cat=EmployeeJobsScript, [] msg=Employee jobs data for scriptName=getEmployeeNumbersForFlexJob scriptStatus=RUNNING executionId=2b80ac18-d97c-4af0-a14b-53b14f1bbc32 EmployeeNumber=A1278828, noOfJobPrefs=0, noOfFlexJobs=3 Jan 28, 2021 @ 09:20:14.418 2021-01-28T09:20:14.418-0600 level=INFO, thread=getEmployeeNumbersForFlexJob-2b80ac18-d97c-4af0-a14b-53b14f1bbc32, cat=EmployeeJobsScript, [] msg=Employee jobs data for scriptName=getEmployeeNumbersForFlexJob scriptStatus=RUNNING executionId=2b80ac18-d97c-4af0-a14b-53b14f1bbc32 EmployeeNumber=A1472766, noOfJobPrefs=0, noOfFlexJobs=4 Jan 28, 2021 @ 09:20:14.391 2021-01-28T09:20:14.391-0600 level=INFO, thread=getEmployeeNumbersForFlexJob-2b80ac18-d97c-4af0-a14b-53b14f1bbc32, cat=EmployeeJobsScript, [] msg=Employee jobs data for scriptName=getEmployeeNumbersForFlexJob scriptStatus=RUNNING executionId=2b80ac18-d97c-4af0-a14b-53b14f1bbc32 EmployeeNumber=A1478985, noOfJobPrefs=0, noOfFlexJobs=5 I understand why I'm getting only 3 rows, If I'll add remaining query parameters for 6,7,....etc I'll get my desired output. But I'm not sure what will be the max value for noOfFlexJobs
I tried message:"noOfFlexJobs=">2 but it didn't work.
Is it possible to query on message statements? Is there a way to find all statements which has noOfFlexJobs>2?
Thanks! in advance.
https://stackoverflow.com/questions/65947390/kibana-kql-finding-all-log-statements-when-parameter-value-is-greater-than-2 January 29, 2021 at 09:00AM
没有评论:
发表评论