Splunk Commands Append, Chart and Dedup. I wouldn't worry about the number of records scanned, if they both got identical results, but I'd make sure the time frames and output results were identical before assuming the code was working apples-to-apples. The where command returns only the results for which the eval expression returns true. Check the results against each other and make sure they came out identical. (50k?)įootnote 2 - use at the end of your earliest and latest to make sure the two timelines are exactly the same. It is a transforming command which has a natural limit on how many results it will allow. Then do whatever makes sense.įootnote: Be careful of table. For overall throughput, slightly more CPU time but all of it on the indexers is far better than slightly less CPU time all on the search head. They are close enough in overall performance that you can go either way and no one will say "Boo" bout it.Ĭheck the details of the run and see how much of that time is on the indexers and how much on the search head. Index=main sourcetype=access_combined_wcookie action=purchase status=200 file=success.So, given your results, it looks like the results are in alignment with my expectations - dedup is slightly less efficient, as expected, but only slightly so. The results contain as many rows as there are. This example takes the incoming result set and calculates the sum of the bytes field and groups the sums by the values in the host field. (index=main sourcetype=access_combined_wcookie action=purchase status=200 file=success.do | dedup JSESSIONID | table JSESSIONID, action, status | rename JSESSIONID as UserSessions The name of the column is the name of the aggregation. /tutorial/splunk/labs/fundamental/Splunk_f1_Data.zip Sample Data - Download sample data for lab. | dedup 3 sourceįor events that have the same 'source' AND 'host' values, keep the first 3 that occur and remove all subsequent events. The Splunk dedup command, short for deduplication, is an SPL command that eliminates duplicate values in fields, thereby reducing the number change it to. | dedup source sortby -_sizeįor events that have the same 'source' value, keep the first 3 that occur and remove all subsequent events. Remove duplicates of results with the same 'source' value and sort the events by the '_size' field in descending order. Remove duplicates of results with the same 'source' value and sort the events by the '_time' field in ascending order. Remove duplicates of results with the same 'host' value. If you search the _raw field, the text of every event in memory is retained which impacts your search performance. For real-time searches, the first events that are received are search, which are not necessarily the most recent events.Īvoid using the dedup command on the _raw field if you are searching over a large volume of data. For historical searches, the most recent events are searched first. Events returned by dedup are based on search order. With the dedup command, you can specify the number of duplicate events to keep for each value of a single field, or for each combination of values among several fields. Dedup: Splunk Commands Tutorials & Reference Commands Category: Filtering Commands: dedup Use: Removes the events that contain an identical combination of values for the fields that you specify.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |