You’re in a dark room, and your screen is crawling with alerts from your IDS/IPS about an IP address trying an exploit on one of your internet-facing devices. You scramble to investigate the event, checking VirusTotal, IPinfo, Whois, and even Google to try to figure out how dangerous the IP might be, only to figure out after an hour that it's actually just Shodan...or Censys…or Pingdom…or a brain-dead bot script….
GreyNoise recently met with Robert*, Manager of Cybersecurity Incident Response & Operations at a large food service & hospitality company. Robert lives this reality every day as he and his Incident Response team respond to network events from their MSSP and internal detections. Recently Robert found a way to leverage GreyNoise Intelligence data integrated into Cortex XSOAR incident response to help his team answer these questions much more quickly and effectively. We caught up with Robert recently to learn how he had done it.
* Customer names have been changed to maintain anonymity
Robert: Sure — I lead the incident response and security operations team at my company. I've been here for about six years and was previously an individual contributor on a joint team that combined security engineering, incident response, and security operations.
A few years ago, we figured it probably doesn't make sense to keep this one giant team, and it would be nice to start developing some specialization. So we chunked the team into two groups — a security engineering team, and an incident response and security operations team. I was fortunate to be selected to run the new IR and Ops team, and I’ve been doing it for about two years now.
Robert: We work with several external partners that triage alerts for us, as well as conduct our own custom detections and investigations. We actually have four different alert classifications:
So basically, we use our MSSP partner as Tier 1 analysts, with my teams handling Tier 2 and above. We're not a 24x7 shop, so we push 24x7 monitoring of our assets to our partners, basically saying “Here's your escalation plan during business hours, here's your escalation plan outside of business hours. Go get them.”
Once we get an alert through any of these sources, we'll do triage, and beyond the initial triage, we’ll do remediation, and then resilience. Any kind of gaps that we have in those tools, we supplement with our own detections, or if it's a use case that’s specific to the company, we'll do it internally as well.
Robert: I remember seeing Andrew [Morris, founder and CEO of GreyNoise] talking about it on Twitter, and I checked it out and thought, wow this is really awesome. I think the biggest sort of “aha” for me was looking at the amount of time it could save me during an investigation.
What was happening quite often to us was, we would get these IDS/IPS alerts about an exploitation attempt. And we’d have to run the alerts to the ground, and it was taking forever.
I realized, looking at GreyNoise, that this tool would tell me if this is a targeted attack, or if it’s a piece of automation that some attacker is using to try to build their botnet. Because if it's an automated attack, and the script fails even slightly, then the whole attack is probably a wash at that point. There's not an operator with hands-on-keyboard to make the adjustments needed, so the automation just fails and moves on to the next IP.
Having that kind of insight was a game-changer for us, in how we analyzed those alerts and how we looked at that type of attack surface. So we started using the Community edition of GreyNoise with our old homegrown incident response tool to query things, and it was tremendously helpful.
For example, one of the things we were looking at was vulnerability scans. If we’re getting vulnerability scanned by somebody who’s scanning everybody else too, okay, welcome to the Internet. But if somebody’s scanning us that GreyNoise has never heard of, that's interesting — why are they just scanning us? Are we being targeted? GreyNoise majorly short-cycled a lot of investigations on these kinds of events.
Another example, we would see somebody slinging an Apache Struts exploit. Okay, are they doing this just to us, or are they doing this to everybody? And again, that kind of changed the dynamic of how we responded to that event.
Robert: A while back, we started to look at various SOAR platforms, and I ended up doing evaluations of what was then Demisto (it’s Cortex XSOAR now), Splunk Phantom, Swimlane, and Siemplify. I engaged with the vendors, downloaded the software, ran an evaluation environment, and actually kicked the tires on each of them. Ultimately we landed on XSOAR, and that started the clock of bringing things over from our existing incident response queue.
As we started to bring events into XSOAR, particularly the ones where we had been using GreyNoise, I realized there was a great opportunity to integrate GreyNoise into XSOAR to automate the process. I went to my director and made the case that we've been using this tool called GreyNoise, and it's super helpful. Now that we've got these events coming from our MSSP into XSOAR, we could start leveraging automation to query GreyNoise, instead of having to manually copy the IP, open up a new tab, paste it in and run a search.
I had been talking to Andrew a fair amount on Twitter, and he sent me over an integration between GreyNoise and XSOAR (there was no marketplace at the time), and it worked really well. So we signed up for a paid plan with GreyNoise to get more IP lookups and more context information. We now use GreyNoise on most of our network detection-based alerts, to give context and alleviate a lot of strain from a research perspective on those events. So it's been great.
The way we’ve integrated GreyNoise into XSOAR is that any time we see an IP address, XSOAR invokes the IP enrichment command for every integration that supports it. This includes GreyNoise, Virus Total, IPinfo.io, whatever. It'll run them all, bring all that data back, and show it to the analyst. One of the areas where GreyNoise is really useful with this approach is inside of our playbook for the IDS/IPS alerts from our MSSP.
In addition to alerts coming from our MSSP, we do custom detections in our SIEM, Splunk Enterprise Security. So while we will write content in Splunk, and raise these as notable events in ES, all of the results get mirrored into XSOAR. We try to make XSOAR the “one-stop-shop” for every event that we’re working.
Robert: Just being able to tell the background noise of the Internet, that was huge for us and made us go “wow”. To see that this activity is just a consequence of being online, but that activity over there GreyNoise hadn't seen, so they're only paying attention to us, and it actually merits us looking into more deeply because they're not just slinging exploits or vulnerability scans at every IP address on the Internet. And we've also seen the opposite, where we see yes, they are a benign scanner, and it's Shodan, cool, we can ignore it.
Because GreyNoise is good at providing context around what IPs are doing in the broader Internet, it typically plays a lot with the IDS/IPS stuff that our MSSP handles — the primary data that we pivot off of for these events is the IP address. We also use GreyNoise on an ad hoc basis — for example, if I see a weird IP on a login to our identity provider, I’ll look it up in GreyNoise and see if it does anything funky on the Internet. And so it's often just helpful context.
Another use case that’s been really helpful is the new “Rule It OuT” tagging, which identifies known legitimate traffic. For example, if I'm doing an investigation on a host that's been compromised and I've got a list of IPs it's talked to in the last 24 hours, I just want to rip out all the stuff that is legitimate, like Office 365 or Slack or cloud solutions that our users should be talking to. I can go to the analysis page on GreyNoise and just drop this big block of IPs in there, and it will tell me if any of these are ones that I should really be caring about, and help me eliminate the ones that I don’t. It gives me a good way to filter for a first pass — I can rule this stuff out for now, and focus on what's left. Then if I still can't find anything, I can go back to the Office 365 IP and look for anomalies there. Because If there's an Eastern European IP address buried in amongst the Office 365 ones, I know which one I want to look at first.
Robert: Every time we find an IP that GreyNoise knows about we ask ourselves, “Hey, is this IP trying to exploit CVE fill-in-the-blank here? Do we run that software? No? OK, done.” Because it's an opportunistic scanner, and GreyNoise is saying the IP is focused on this particular CVE, or it loves these six CVEs for this product suite. This allows us to conclude that, because we don't use that product suite, we're good, and we can ignore it.
Alternatively, the answer might be “yes, we do use that product suite”. In that case, I’ll go read about the CVE and see what versions are vulnerable, and then look at our system and see if it’s something we need to dig into, or whether we already patched for it.
As part of our triage, we also check in with our vulnerability management team. If we see a scanner out on the Internet scanning through our IPs, we’ll go dig up our internal scan results for the same IP and see what we found. Because we can pretty much know that anything we found, the bad guys just did too.
There’s some nice visibility we get with GreyNoise around vulnerability scans. For example, for events that we get, we can look at our firewall and easily see that this particular IP scans all of these public IPs. Then we can check in GreyNoise to see, what ports does this scanner like to scan? We obviously know they scan port 80 because that's why we're here. But do they scan 443 (SSL)? Recently Andrew tweeted about some new behavior GreyNoise tracks looking at whether scanners follow 301 or 302 redirects.
Say we determine that this IP only scans port 80 — cool. Now we can go into XSOAR and CURL all of the IPs that they scanned on port 80 and check to see if there’s a common vulnerability they might be looking for. Or do we get our load balancer telling them they need to go to 443? Because if we get that, and this IP doesn’t follow redirects, and this IP doesn't scan 443, then he's hitting a load balancer appliance, and I don't care. In XSOAR, that playbook will close the event if this is the condition that's met. And no analyst ever has to interact with this event.
Robert: I remember first seeing Andrew talking about GreyNoise on Twitter, and when I checked it out I thought, wow this is really awesome. I think the biggest sort of “aha” for me was looking at the amount of time it could save me during an investigation
We started using the GreyNoise Community edition with our old homegrown incident response tool, and it was tremendously helpful. That experience allowed me to go to my director and make the case for leveraging automation to query GreyNoise, instead of having to manually copy the IP, open up a new tab, paste it in and run a search. Today we use GreyNoise on most of our network detection-based alerts, and to enrich every event we get from our MSSP. I would estimate that GreyNoise is saving us probably an hour of work every time we get one of these vulnerability scan events — we’ve automated our approach completely, so an analyst doesn’t even have to touch it (and we see around 8-10 of these events every week). So long story short, GreyNoise really does shorten the cycle of some of these vulnerability scan investigations and is saving us about 40 hours per month that we can apply to other high-priority security problems.
I’m also getting some good value out of using GreyNoise for ad hoc investigation context help — for example, when I've got a bunch of IPs and I just want to filter out the ones that I don't need to care about. We use GreyNoise to give context and alleviate a lot of strain from a research perspective on those events. So it's been great.
Curious how GreyNoise can save your SOC time? Try the product for free.