Tag Archives: security

Splunk Funk

Recently, I was asked to evaluate an organization’s Splunk deployment. This request flummoxed me, because while I’ve always been a fan of the tool’s capabilities, I’ve never actually designed an implementation or administered it. I love the empowerment of people building their own dashboards and alerts, but this only works when there’s a dedicated Splunk-Whisperer carefully overseeing the deployment and socializing the idea of using it as self-service, cross-functional tool.  As I started my assessment, I entered what can only be called a “dark night of the IT soul” because my findings have led me to question the viability of most enterprise monitoring systems.

The original implementer recently moved on to greener pastures and (typically) left only skeletal documentation. As I started my investigation, I discovered  a painfully confusing distributed deployment built with little to no understanding of  “best practices” for the product. With no data normalization and almost non-existent data input management, the previous admin had created the equivalent of a Splunk Wild West, allowing most data to flow in with little oversight or control. With an obscenely large number of sourcetypes and sources, the situation horrified Splunk support and they told me my only option was to rebuild, a scenario that filled me with nerd-angst.

In the past, I’ve written about the importance of using machine data for infrastructure visibility. It’s critical for security, but also performance monitoring and troubleshooting. Log correlation and analysis is a key component of any healthy infrastructure and without it, you’re like a mariner lost at sea. So imagine my horror when confronted by a heaping pile of garbage data thrown into a very expensive application like Splunk.

Most organizations struggle with a monitoring strategy because it simply isn’t sexy. It’s hard to get business leadership excited about dashboards, pie charts and graphs without contextualizing them in a report. “Yeah baby, let me show you those LOOOOW latency times in our web app.” It’s a hard sell, especially when you see the TCO for on-premise log correlation and monitoring tools. Why not focus on improving a product that could bring in more customer dollars or a new service to make your users happier?  Most shops are so focused on product delivery and firefighting, they simply don’t have cycles left for thinking about proactive service management. So you end up with infrastructure train wrecks, with little to no useful monitoring.

While a part of me still believes in using the best tools to gain intelligence and visibility from an infrastructure, I’m tired of struggling. I’m beginning to think I’d be happy with anything, even a Perl script, that works consistently with a low LOE. I need that data now and no longer have the luxury of waiting until there’s a budget for qualified staff and the right application. Lately, I’m finding it pretty hard to resist the siren song of SaaS log management tools that promise onboarding and insight into machine data within minutes, not hours. Just picture it: no more agents or on-premise systems to manage, just immediate visibility into data.  Most other infrastructure components have moved to the cloud, maybe it’s inevitable for log management and monitoring. Will I miss the flexibility and power of tools like Splunk and ELK? Probably, but I no longer have the luxury of nostalgia.

 

 

Tagged , , , , ,

Danger: Stunt Hacking Ahead

On 4/18, Ars Technica reported on a recent 60 Minutes stunt-hacking episode by some telecom security researchers. During the episode, US representative Ted Lieu had a cell phone intercepted via vulnerabilities in the SS7 network. I’m no voice expert*, but it’s clear that both the 60 Minutes story and the Ars Technica article are pretty muddled attempts to dissect the source of these vulnerabilities. This is probably because trying to understand legacy telephony protocols such as SS7 is only slightly less challenging than reading ancient Sumerian.

Since I was dubious regarding the “findings” from these reports,  I  reached out to my VoIP bestie, @Unregistered436.

Screenshot 2016-04-20 11.33.19

 

While we can argue over how useful media FUD is in getting security issues the attention they deserve, I have other problems with this story:

  • This isn’t new information. Researchers (including the ones appearing in 60 Minutes) have been reporting on problems with SS7 for years. A cursory Google search found the following articles and presentations, including some by the Washington Post.

German Researchers Discover a Flaw That Could Let Anyone Listen to Your Cell Calls 12/18/14

Locating Mobile Phones Using Signalling System #7 1/26/13

For Sale: Systems that Can Secretly Track Where Cellphone Users Go Around the Globe 8/24/14

Toward the HLR, Attacking the SS7 & SIGTRAN Applications H2HC, Sao Paulo, Brazil, December 2009

  • If you’re going to reference critical infrastructure such as the SS7 network, why not discuss how migration efforts with IP convergence in the telco industry relate to this issue and could yield improvements? There are also regulatory concerns which impact the current state of the telecommunications infrastructure as well. Maybe Ted Lieu should start reading all those FCC documents and reports.
  • Legacy protocols don’t get ripped out or fixed overnight (IPv4 anyone?), so the congressman’s call to have someone “fired” is spurious. If security “researchers”  really want things to change, they should contribute to ITU, IEEE and IETF working groups or standards committees and help build better protocols. Or *shudder* take a job with a telecom vendor. We all need to take some ownership to help address these problems.

*If you want to learn more about telecom regulation, you should definitely follow Sherry Lichtenberg. For VoIP and SS7 security, try Patrick McNeil and Philippe Langlois.

Tagged , , , , , ,

The Endpoint Is Dead To Me

Another day, another vulnerability. This month the big non-event was Badlock, following the recent trend of using a splashy name to catch the media’s attention so they could whip the management meat puppets into a paranoid frenzy. Many in the security community seem unperturbed by this latest bug, because let’s face it, nothing can surprise us after the last couple of really grim years.

But then more Java and Adobe Flash bugs were announced, some new iOS attack using Wi-Fi and NTP that can brick a device, followed by an announcement from Apple that they’ve decided to kill QuickTime for Windows instead of fixing a couple of critical flaws. All of this is forcing  me to admit the undeniable fact that trying to secure the endpoint is a waste of time. Even if you’re using a myriad of management tools, patching systems, vulnerability scanners, and DLP agents; you will fail, because you can never stay ahead of the game.

The endpoint must be seen for what it truly is: a limb to be isolated and severed from the healthy body of the enterprise at the first sign of gangrene. It’s a cootie-ridden device that while necessary for our users, must be isolated as much as possible from anything of value. A smelly turd to be flushed without remorse or hesitation. No matter what the user says, it is not valuable and nothing of value to the organization should ever be stored on it.

This doesn’t mean I’m advocating removing all endpoint protection or patching agents. But it’s time to get real and accept the fact that most of this corporate malware is incapable of delivering what the vendors promise. Moreover, most often these applications negatively impact system performance, which infuriates our users. But instead of addressing this issue, IT departments layer on more of this crapware on the endpoint, which only encourages users to find ways to disable it. One more security vulnerability for the organization. A dedicated attacker will figure out a way to bypass most of it anyway, so why do we bother to trust software that is often more harmful to business productivity than the malware we’re trying to block?  We might as well give out “Etch A Sketches” instead of laptops.

google_on_my_etch_a_sketch_by_pikajane-d3846dy

Tagged , , , , , ,

Fear and Loathing in Vulnerability Management

Vulnerability management – the security program that everyone loves to hate, especially those on the receiving end of a bunch of arcane reports. Increasingly, vulnerability management programs seem to be monopolizing more and more of a security team’s time, mostly because of the energy involved with scheduling, validating and interpreting scans. But does this effort actually lead anywhere besides a massive time suck and interdepartmental stalemates? I would argue that if the core of your program is built on scanning, then you’re doing it wrong.

The term vulnerability management is a misnomer, because “management” implies that you’re solving problems. Maybe in the beginning scanning and vulnerability tracking helped organizations, but now it’s just another method used by security leadership to justify their every increasing black-hole budgets. “See? I told you it was bad. Now I need more $$$$ for this super-terrific tool!”

Vulnerability management programs shouldn’t be based on scanning, they should be focused on the hard stuff: policies, standards and procedures with scans used for validation. If you’re stuck in an endless cycle of scanning, patching, more scanning and more patching; you’ve failed.

You should be focused on building processes that bake build standards and vulnerability remediation into a deployment procedure. Work with your Infrastructure team to support a DevOps model that eliminates those “pet” systems. Focus on “cattle” or immutable systems that can and should be continuously replaced when an application or operating system needs to be upgraded. Better yet, use containers. Have a glibc vulnerability? The infrastructure team should be creating new images that can be rolled out across the organization instead of trying to “patch and pray” after finally getting a maintenance window. You should have a resilient enough environment that can tolerate change; an infrastructure that’s self-healing because it’s in a state of continuous deployment.

I recognize that most organizations aren’t there, but THIS should be your goal, not scanning and patching. Because it’s a treadmill to nowhere. Or maybe you’re happy doing the same thing over and over again with no difference in the result. If so, let me find you a very large hamster wheel.

hamster_meme

 

Tagged , , , , , ,

Introducing: Security SOC Puppets

Gert and Bernie

Please join Gert, Bernie and friends in their wild adventures through cyberspace! In episode one, our woolen friends explore the frustrating topic of email encryption.

Tagged , , , ,

Security Threat Levels with a Side of FUD

Today the SANS Internet Storm Center raised it’s Infocon Threat Level to “yellow” due to the recently announced backdoor in Juniper devices. I wouldn’t have even known this if someone hadn’t pointed it out to me and then I felt like I was in an episode of Star Trek. I kept waiting for the ship’s computer to make an announcement so I could strap myself into my chair.

While the level names are different, the colors seem to mirror the old Homeland Security color-coded advisory system, which was eliminated in 2011 due to questions over it’s usefulness.

2000px-Hsas-chart_with_header.svg

According to a story on CNN.com:

“The old color coded system taught Americans to be scared, not prepared,” said ranking member Rep. Bennie Thompson, D-Mississippi. “Each and every time the threat level was raised, very rarely did the public know the reason, how to proceed, or for how long to be on alert. I have raised concerns for years about the effectiveness of the system and have cited the need for improvements and transparency. Many in Congress felt the system was being used as a political scare tactic — raising and lowering the threat levels when it best suited the Bush administration.”

I have a similar experience with SANS’ Infocon and the reactions from management.

Pointy-haired Fearless Leader: OMG, the SANS Infocon is at YELLOW!!! The end of the Internet is nigh!

Much Put-Upon Security Architect: Please calm down and take a Xanax. It’s just a color.

I’d like to propose a simpler and more useful set of threat levels with recommended actions. Let’s call it the Postmodern Security Threat Action Matrix:

Level Description Action
Tin Foil Hat Normal levels of healthy paranoia You can still check your email and watch Netflix. But remember they’re always watching….
Adult Diaper It’s damn scary out there. Trust no one. Remember to update your Tor browser. Have your “go bag” ready.
Fetal Position Holy underwear Batman, it’s the end. Destroy all electronic devices and move into a bomb shelter. The Zombie Apocalypse is imminent.
Tagged , , , , ,

Don’t Let the Grinch Ruin Your Credit

Believe it or not, I actually like to educate my friends and acquaintances about technology. It makes my skeptical, shriveled, infosec heart grow a few sizes larger when I solve even the simplest problems, making someone’s life a little easier. So I was ecstatic to create and teach a free online-safety webinar for one of my favorite programs, AARP Tek Academy. While not as exciting as chasing down hackers or fighting a DDoS attack, it was a very rewarding experience.  And I didn’t have to argue with anyone about budgets or risk. So please share it with your Luddite friends this holiday season.

You can access the webinar here.

grinch_heart

Tagged , , , , ,

Security Training for Cheapskates

During a recent webinar I gave, someone asked how soon I would be doing another one. I was flattered, but responded that because of a full-time job as an architect, my time was limited. “Besides,” I said, “you don’t need to wait for me, there’s plenty of free or inexpensive security training available online.”

Security professionals love to share and show off what they’ve learned. Some of us crave the warm fuzzy of helping our colleagues, while others do it to demonstrate their wicked skills or build their resume. Regardless of the motivation, that means there’s always abundant content to help you learn and grow.

Here’s a list of useful sites that I’ll try to keep updated. If you know of others and would like to contribute or if you think the training is outdated or bad, please let me know and I’ll adjust the list accordingly.

Securitytube.net – a project of security researcher, Vivek Ramachandran.

Hak5.org – Online security show produced by Darren Kitchen (of Pineapple WiFi router fame) and a collection of nerds who demo security tools and hacks. Includes Metasploit Minute with the awesome @Mubix.

OWASP – The Open Web Application Security Project has lots of “how to” guides and videos.

Offensive Security’s Vimeo Channel

Metasploit Unleased, Made for Hackers for Charity, an ethical hacking course provided free of charge to the InfoSec community in an effort to raise funds and awareness for underprivileged children in East Africa.

Georgia Weidman:Bulb Security – creator of the Smartphone Pentest Framework, researcher and author of Penetration Testing: A Hands-on Introduction to Hacking. She offers inexpensive online training in pentesting.

Adrian Crenshaw’s site, Irongeek, with conference and training videos.

Official BlackHat Conference Youtube Channel 

Defcon Youtube Channel 

Chaos Communication Congress videos

OpenSecurityTraining.info – CreativeCommons licensed security training site

Cyber Kung Fu for the Eight (8) Domains of CISSP – Training videos from Larry Greenblatt, a CISSP training guru.

Pentester Academy – video training site available for monthly or yearly subscription fee. Some free content.

Pentester Lab – Free online pentesting courses with practice images.

Penetration Testing Practice Lab – A mindmap of available vulnerable applications and systems practicing pentesting.

ENISA(European Union Agency for Network and Information Security) incident handling training

Carnegie Mellon University Software Engineering Institute (SEI) training – low-cost security training from a research, development and training center involved in computer software and network security.

Cybrary – free online IT and security training that grew out of a Kickstarter project.

Udemy, Coursera, edX and many universities offer MOOCs in computer science and information security. You can get a list from MOOC-Online.

Tagged , , , ,

Security Vs. Virtualization

Recently, I wrote an article for Network Computing about the challenges of achieving visibility in the virtualized data center.

Security professionals crave data. Application logs, packet captures, audit trails; we’re vampiric in our need for information about what’s occurring in our organizations. Seeking omniscience, we believe that more data will help us detect intrusions, feeding the inner control freak that still believes prevention is within our grasp. We want to see it all, but the ugly reality is that most of us fight the feeling that we’re flying blind.

In the past, we begged for SPAN ports from the network team, frustrated with packet loss. Then we bought expensive security appliances that used prevention techniques and promised line-rate performance, but were often disappointed when they didn’t “fail open,” creating additional points of failure and impacting our credibility with infrastructure teams.

So we upgraded to taps and network packet brokers, hoping this would offer increased flexibility and insight for our security tools while easing fears of unplanned outages. We even created full network visibility layers in the infrastructure, thinking we were finally ahead of the game.

Then we came face-to-face with the nemesis of visibility: virtualization. It became clear that security architecture would need to evolve in order to keep up.

You can read and comment on the rest of the article here.

Tagged , , , , , , ,

Is Your Security Architecture Default-Open or Default-Closed?

One of the most significant failures I see in organizations is an essential misalignment between Operations and Security over the default network state. Is it default-open or default-closed? And I’m talking about more than the configuration of fail-open or fail-closed on your security controls.

Every organization must make a philosophical choice regarding its default security state and the risk it’s willing to accept. For example, you may want to take a draconian approach, i.e. shooting first, asking questions later. This means you generally validate an event as benign before resuming normal operations after receiving notification of an incident.

But what if the security control detecting the incident negatively impacts operations through enforcement? If your business uptime is too critical to risk unnecessary outages, you may decide to continue operating until a determination is made that an event is actually malicious.

Both choices can be valid, depending upon your risk appetite. But you must make a choice, socializing that decision within your organization. Otherwise, you’re left with confusion and conflict over how to proceed during an incident.

baby_meme

Tagged , , , , ,
<span>%d</span> bloggers like this: