Getting Started Searching Splunk

1 Enabling Cheat Codes
2 Your First Search / Using Splunk Processing Language (SPL)

Enabling Cheat Codes

Across the top of the window you will see your name with a small down carrot. This is your personal preferences menu.

Click your name.
Select Preferences
Click “SPL Editor”

Below is a Window similar to what you should see. If you are new to Splunk we would strongly recommend that you start with “Full” as your search assistant with “Line Numbers” and “Search auto-format” turned on.

Now click on the Themes section. You should choose “Light Theme” or “Dark Theme” bases on your personal preferences but not “Black on White”. The “Black on White” does not offer syntax highlighting and it not recommended for starting out.

Your First Search / Using Splunk Processing Language (SPL)

What do you have access to?

Before we search for something, you are going to want to know where to look. So the question becomes, what do you have access to?

Copy the code for the following search:

| eventcount summarize=f index=* | stats count by index

And paste it in the search editor as shown below, then click on the magnifying glass.

After a short search period, Splunk will return a list of indexes that you have access to. Below is an example of what you will see.

Note: The results will likely be different from person-to-person as not everyone has the same access to data. If you do not have access to an index that you believe you should, please have the data owner of that service submit a ticket requesting access on your behalf.

Performing a search on an index

Scenario: Someone is unable to reach a server. The server is known to be active but there is no response on the network. In this case we know the server itself is operating properly and we believe there is a block at the firewall that is preventing access.

We know that we have access to the firewall indexes. The data center firewall is a Fortinet so we are going to want to search the “firewall_fortinet” index.

Important: When you are searching indexes and you are not sure what fields you should use to limit the search to a more accurate set a results, make sure you set the time of the search to “Last 15 minutes” so that you are not waiting for hundreds or thousands or even millions of results to be returned.

We can use the following search, set the time and click the magnifying glass.

index=firewall_fortinet

So we have over 40K results to sift through. This is way to much to be helpful so lets take a close look at what has been returned to us:

You will notice that we can see field names such as “srcip”, “dstip”, and “dstport”. These fields have been indexed by Splunk so you can search by them.

Let us assume that the server’s IP address is 137.99.218.133

Let us assume that our workstation IP is 10.99.4.40 and that we are trying to use remote desktop which is on port 3389.

We can use those fields and values to make a key pair and use them in the search

Tip: You can use shift+enter to move to a new line to get exactly what is shown below. It has been formatted to make it more legible but everything can be entered on a single line.

Use the following search, set the time and click the magnifying glass.

index=firewall_fortinet
                    srcip="10.99.4.40"
                    dstip="137.99.218.133"
                    dstport=3389 
                    action=blocked

dstip is where the traffic is going to.

srcip is where the traffic coming from.

dstport if the destination port of the service. 3389 is RDP/Remote Desktop

action is either blocked or allowed. (It is true that there are other possible values but that is for more advanced troubleshooting not covered in this intro guide.)

Now we only have 39 results.

Making the results easier to read

This is way better but the information is still not really all that readable. We can now apply some formatting to make everything easier to read.

This puts everything into a pretty table

You can now tell the server team that you need a firewall rule that will allow inbound traffic to the IP that is being blocked on the SDC firewall in the ITS VDOM.