Kibana KQL - finding all log statements when parameter value is greater than 2

Question

I'm writing a KQL to build Kibana Visualize. I've build a query to find my expected result but it's not perfect.

Points to be noted -

1. Data is logger messages, not json
2. I searched a lot but most of answers and stackoverflow suggestions were for json data
3. My queries are on "message" field

Expected Result - All log messages which have noOfFlexJobs>2

Here is my Query-1 -

message:"thread=getEmployeeNumbersForFlexJob-2b80ac18-d97c-4af0-a14b-53b14f1bbc32" and message:"noOfFlexJobs="

Query-1 Result -

    Time    message
Jan 28, 2021 @ 09:20:14.503 2021-01-28T09:20:14.503-0600 level=INFO, thread=getEmployeeNumbersForFlexJob-2b80ac18-d97c-4af0-a14b-53b14f1bbc32, cat=EmployeeJobsScript, [] msg=Employee jobs data for scriptName=getEmployeeNumbersForFlexJob scriptStatus=RUNNING executionId=2b80ac18-d97c-4af0-a14b-53b14f1bbc32 EmployeeNumber=A1480876, noOfJobPrefs=0, noOfFlexJobs=0
Jan 28, 2021 @ 09:20:14.486 2021-01-28T09:20:14.486-0600 level=INFO, thread=getEmployeeNumbersForFlexJob-2b80ac18-d97c-4af0-a14b-53b14f1bbc32, cat=EmployeeJobsScript, [] msg=Employee jobs data for scriptName=getEmployeeNumbersForFlexJob scriptStatus=RUNNING executionId=2b80ac18-d97c-4af0-a14b-53b14f1bbc32 EmployeeNumber=a787754, noOfJobPrefs=0, noOfFlexJobs=1
Jan 28, 2021 @ 09:20:14.470 2021-01-28T09:20:14.470-0600 level=INFO, thread=getEmployeeNumbersForFlexJob-2b80ac18-d97c-4af0-a14b-53b14f1bbc32, cat=EmployeeJobsScript, [] msg=Employee jobs data for scriptName=getEmployeeNumbersForFlexJob scriptStatus=RUNNING executionId=2b80ac18-d97c-4af0-a14b-53b14f1bbc32 EmployeeNumber=A1478669, noOfJobPrefs=0, noOfFlexJobs=1
Jan 28, 2021 @ 09:20:14.454 2021-01-28T09:20:14.454-0600 level=INFO, thread=getEmployeeNumbersForFlexJob-2b80ac18-d97c-4af0-a14b-53b14f1bbc32, cat=EmployeeJobsScript, [] msg=Employee jobs data for scriptName=getEmployeeNumbersForFlexJob scriptStatus=RUNNING executionId=2b80ac18-d97c-4af0-a14b-53b14f1bbc32 EmployeeNumber=A1478668, noOfJobPrefs=0, noOfFlexJobs=0
Jan 28, 2021 @ 09:20:14.443 2021-01-28T09:20:14.443-0600 level=INFO, thread=getEmployeeNumbersForFlexJob-2b80ac18-d97c-4af0-a14b-53b14f1bbc32, cat=EmployeeJobsScript, [] msg=Employee jobs data for scriptName=getEmployeeNumbersForFlexJob scriptStatus=RUNNING executionId=2b80ac18-d97c-4af0-a14b-53b14f1bbc32 EmployeeNumber=A1278828, noOfJobPrefs=0, noOfFlexJobs=3
Jan 28, 2021 @ 09:20:14.418 2021-01-28T09:20:14.418-0600 level=INFO, thread=getEmployeeNumbersForFlexJob-2b80ac18-d97c-4af0-a14b-53b14f1bbc32, cat=EmployeeJobsScript, [] msg=Employee jobs data for scriptName=getEmployeeNumbersForFlexJob scriptStatus=RUNNING executionId=2b80ac18-d97c-4af0-a14b-53b14f1bbc32 EmployeeNumber=A1472766, noOfJobPrefs=0, noOfFlexJobs=4
Jan 28, 2021 @ 09:20:14.391 2021-01-28T09:20:14.391-0600 level=INFO, thread=getEmployeeNumbersForFlexJob-2b80ac18-d97c-4af0-a14b-53b14f1bbc32, cat=EmployeeJobsScript, [] msg=Employee jobs data for scriptName=getEmployeeNumbersForFlexJob scriptStatus=RUNNING executionId=2b80ac18-d97c-4af0-a14b-53b14f1bbc32 EmployeeNumber=A1478985, noOfJobPrefs=0, noOfFlexJobs=5
Jan 28, 2021 @ 09:20:14.380 2021-01-28T09:20:14.379-0600 level=INFO, thread=getEmployeeNumbersForFlexJob-2b80ac18-d97c-4af0-a14b-53b14f1bbc32, cat=EmployeeJobsScript, [] msg=Employee jobs data for scriptName=getEmployeeNumbersForFlexJob scriptStatus=RUNNING executionId=2b80ac18-d97c-4af0-a14b-53b14f1bbc32 EmployeeNumber=A1472442, noOfJobPrefs=0, noOfFlexJobs=11
Jan 28, 2021 @ 09:20:14.357 2021-01-28T09:20:14.357-0600 level=INFO, thread=getEmployeeNumbersForFlexJob-2b80ac18-d97c-4af0-a14b-53b14f1bbc32, cat=EmployeeJobsScript, [] msg=Employee jobs data for scriptName=getEmployeeNumbersForFlexJob scriptStatus=RUNNING executionId=2b80ac18-d97c-4af0-a14b-53b14f1bbc32 EmployeeNumber=A1502372, noOfJobPrefs=0, noOfFlexJobs=0
Jan 28, 2021 @ 09:20:14.352 2021-01-28T09:20:14.352-0600 level=INFO, thread=getEmployeeNumbersForFlexJob-2b80ac18-d97c-4af0-a14b-53b14f1bbc32, cat=EmployeeJobsScript, [] msg=Employee jobs data for scriptName=getEmployeeNumbersForFlexJob scriptStatus=RUNNING executionId=2b80ac18-d97c-4af0-a14b-53b14f1bbc32 EmployeeNumber=A1477010, noOfJobPrefs=0, noOfFlexJobs=0
Jan 28, 2021 @ 09:20:14.342 2021-01-28T09:20:14.342-0600 level=INFO, thread=getEmployeeNumbersForFlexJob-2b80ac18-d97c-4af0-a14b-53b14f1bbc32, cat=EmployeeJobsScript, [] msg=Employee jobs data for scriptName=getEmployeeNumbersForFlexJob scriptStatus=RUNNING executionId=2b80ac18-d97c-4af0-a14b-53b14f1bbc32 EmployeeNumber=A1467206, noOfJobPrefs=0, noOfFlexJobs=16

To get desired result I have updated my query- Query-2

message:"thread=getEmployeeNumbersForFlexJob-2b80ac18-d97c-4af0-a14b-53b14f1bbc32" and (message:"noOfFlexJobs=3" or message:"noOfFlexJobs=4" or message:"noOfFlexJobs=5")

Query-2 Result

Time    message
Jan 28, 2021 @ 09:20:14.443 2021-01-28T09:20:14.443-0600 level=INFO, thread=getEmployeeNumbersForFlexJob-2b80ac18-d97c-4af0-a14b-53b14f1bbc32, cat=EmployeeJobsScript, [] msg=Employee jobs data for scriptName=getEmployeeNumbersForFlexJob scriptStatus=RUNNING executionId=2b80ac18-d97c-4af0-a14b-53b14f1bbc32 EmployeeNumber=A1278828, noOfJobPrefs=0, noOfFlexJobs=3
Jan 28, 2021 @ 09:20:14.418 2021-01-28T09:20:14.418-0600 level=INFO, thread=getEmployeeNumbersForFlexJob-2b80ac18-d97c-4af0-a14b-53b14f1bbc32, cat=EmployeeJobsScript, [] msg=Employee jobs data for scriptName=getEmployeeNumbersForFlexJob scriptStatus=RUNNING executionId=2b80ac18-d97c-4af0-a14b-53b14f1bbc32 EmployeeNumber=A1472766, noOfJobPrefs=0, noOfFlexJobs=4
Jan 28, 2021 @ 09:20:14.391 2021-01-28T09:20:14.391-0600 level=INFO, thread=getEmployeeNumbersForFlexJob-2b80ac18-d97c-4af0-a14b-53b14f1bbc32, cat=EmployeeJobsScript, [] msg=Employee jobs data for scriptName=getEmployeeNumbersForFlexJob scriptStatus=RUNNING executionId=2b80ac18-d97c-4af0-a14b-53b14f1bbc32 EmployeeNumber=A1478985, noOfJobPrefs=0, noOfFlexJobs=5

I understand why I'm getting only 3 rows, If I'll add remaining query parameters for 6,7,....etc I'll get my desired output. But I'm not sure what will be the max value for noOfFlexJobs

I tried message:"noOfFlexJobs=">2 but it didn't work.

Is it possible to query on message statements? Is there a way to find all statements which has noOfFlexJobs>2?

Thanks! in advance.

question from:https://stackoverflow.com/questions/65947390/kibana-kql-finding-all-log-statements-when-parameter-value-is-greater-than-2

Answer 1

I have figured it out. It could be done using not keyword in the KQL.

So answers is:

message:"thread=getEmployeeNumbersForFlexJob-2b80ac18-d97c-4af0-a14b-53b14f1bbc32" and not message:"noOfFlexJobs=0" and not message:"noOfFlexJobs=1"