Last week, we discussed how perfect network security is impossible. A determined and well-resourced attacker can always find a way in. Your goal is therefore to cover as many of the risk areas as you can with the budget you have.

In this post, we take a look at ways at effective ways to block most inbound and outbound network attacks.

Defending a network against inbound attacks

By inbound attacks, I mean traditional hacking attempts against a network’s front-door elements like web servers, web applications, email systems, and remote access (VPN) systems.

Firewalls

The first and most obvious security element against inbound attack is a firewall. Pretty much any commercial firewall is suitable, but I generally recommend using a more sophisticated next-generation or unified threat management (UTM) firewall.

A firewall is a network device more than it is a security device. It facilitates connection between your internal network and the public Internet. Since everybody these days uses private IP addressing, a firewall is the right place to do address translation between internal and external address spaces.

A firewall also has the ability to filter incoming or outgoing connections based on simple Layer 3 and Layer 4 elements in the packet header. These include internal and external IP addresses, as well as TCP and UDP port numbers.

A firewall also has to be stateful, which means it keeps track of every individual session passing through it. You shouldn’t be able to get packets past a firewall simply by constructing them to look like part of a pre-existing session.

The problem with basic firewalls is that they assume everything that looks like a duck must be a duck. If it’s a TCP session on port 443, then it must be HTTPS. But this assumption completely neglects the possibility that somebody might shift an illegal application to a legal port. Ports are just numbers—they’re easy to change.

firewall brick wall network infrastructure security device

Photo: Daniel King on Flickr

Intrusion detection and prevention systems

To get around a firewall’s essential flaw, it’s useful to couple a firewall with an intrusion detection system or intrusion prevention system (IDS/IPS). An IDS/IPS is a device that watches every packet and session to look for signs of malicious activity. Generally, more serious problems cause the IDS/IPS to use prevention mode, where it drops the session. Less serious problems and things that are suspicious but not necessarily bad are merely detected, resulting in an alert.

Most IDS/IPS devices use a combination of factors to detect malicious behavior. They monitor for what might be called application orthodoxy, which means they reject sessions that don’t appear to be following the established rules for the protocol they’re using. And they use signature-based detection to look for known patterns of malicious behavior. To be really effective, you need to make sure those signatures are kept up to date, which generally means some sort of subscription model.

The reason I like next-generation or UTM firewalls is because they include an IDS/IPS in the same box, and display IDS alerts on the same management interface. Such a two-in-one device is cost-effective and simplifies management. Some UTM firewalls also look for things like virus signatures during file transfers.

Reverse proxies and web application firewalls

The next important inbound protection you can deploy at the network edge is a reverse proxy, a device that masquerades as a web server or similar Internet-accessible server. The real server sits somewhere inside the network, and the reverse proxy passes the data to and from that real server. (Obviously, you only need this type of protection if you have some sort of server that’s accessible from the Internet.)

In many cases, a reverse proxy is used to translate between an insecure protocol like HTTP, which might be all that a legacy application server supports, and a more secure one like HTTPS, which is more appropriate for anything on the public Internet.

The problem with reverse proxies is that they don’t inspect the contents of the traffic they’re relaying. That’s fine for protocol-based attacks, but not for application-based attacks. For example, two of the most dangerous and common types of attacks against web servers are SQL injection and cross-site scripting. In both cases, the HTTP traffic looks perfectly fine, but is specially constructed to trigger a bug in the implementation of the web server that gives the attacker access they shouldn’t have.

For this reason, I’m not a huge fan of reverse proxies unless they’re also web application firewalls (WAF). A WAF is also a device that sits between the Internet and a web server, but it explicitly sanitizes every request coming from the remote user. At a minimum, it will remove quotation marks and other special characters that are typical of SQL injection attacks.

Some WAF devices can also be configured to know exactly what types of data are allowed in specific fields on a web page that accepts input. Some WAFs will even monitor the connection between a web server and a database to ensure that an otherwise innocent-looking request didn’t somehow result in a huge table dump.

inspecting cloud network monitoring security

Photo: Kate Ter Haar on Flickr

Email scanners

Another device frequently exposed to the public Internet is an email server. I’ve seen attackers directly trying to do malicious things to an email server, but more typically, they’re trying to deliver malware or spam. So I like to deploy an email scanner in front of an email server. (Note that you can completely avoid the need for this type of defense if you use an outsourced email service.)

Email scanners are concerned with two problems: They want to eliminate or at least reduce spam, and they want to reduce malware attacks by scanning for viruses. An emerging use of these devices is a reduction in phishing attacks as well. However, a well-constructed phishing attack looks so much like a legitimate email message that it’s really only feasible to eliminate the bad ones.

Imagine that an incoming email message has a virus in an attachment. If the email scanner catches it, it quarantines the message and the end user doesn’t see it. If the email scanner doesn’t catch it, then you have to hope whatever anti-malware system you have on that user’s workstation will catch it. For this reason, it makes sense to use completely different malware scanning systems on emails scanners and workstations. I’ll discuss malware scanning systems in more detail later in this post

Defending a network against outbound attacks

Outbound attacks include malicious traffic that originates inside a network and goes out to the Internet. It probably sounds like a pretty low priority. Why should you spend any special effort looking for evidence that your staff or corporate clients are hacking somebody else?

While that’s one of the things we’re looking for, it’s far from the most important. There are two other things we really care about here:

  • Attempts by malicious software in the infrastructure to reach out to its controllers
  • The leaking of sensitive data to external parties
network data leak security infrastructure

Photo: Korona Lacasse on Flickr

DNS

The simplest and least expensive thing you can do to protect against outbound attacks is to use a really robust domain name system (DNS). Malware has three ways to call home for instructions. It can use:

  • Hard-coded IP addresses—but they’re very easy to block once discovered
  • Domain names the malware rotates through on a programmed schedule
  • Compromised legitimate sites or advertising services

DNS-based protection can be a useful first line of defense against the second type of attack, but not the others.

Interestingly, though, there do appear to be some malware systems that use DNS queries for communicating with other command-and-control traffic. That is, the results of a DNS query might be interpreted by the malware to mean something other than an IP address. This is another way a good DNS filtering system can be helpful in combatting malware.

However, the problem with relying on DNS for malware is that the malware is still sitting there inside your environment. It hasn’t been eliminated, just temporarily silenced.

IDS/IPS

Another useful line of defense against outbound attacks is an IDS/IPS, perhaps the same one deployed for inbound protection. But this time we’re interested in different types of attacks with a completely different set of signatures. Now we’re looking for malware inside a network as it tries to connect to command-and-control (C&C) servers on the Internet. We’re also looking for indications of unauthorized VPNs or remote access Trojans (RATs).

Logs

If you use a central authentication system like LDAP or Active Directory, (and I believe you should) then you get another excellent defensive tool essentially for free. Simply monitor LDAP or Active Directory logs. Look for repeated failed logins, which might be an indication of somebody attempting a brute force attack. Look for people logging in when they shouldn’t be or logging into systems they shouldn’t be on, particularly users with special privileges, like system administrators.

Probably the most effective type of advanced persistent threat (APT) attack is one where the attackers steal legitimate user credentials. Then they don’t need to devise any exceptionally clever hack—they simply log in and poke around until they find something interesting.

network log files wood pile network security

Photo: Wonderlane on Flickr

Web proxy servers

The next thing to look at for protection against outbound attack is a web proxy server, a system that intercepts all outbound web requests and tries to serve them locally. If somebody just loaded a particular web page, the proxy can take the page from its cache and send it back to the user immediately.

Proxy servers provide better performance, as well as an opportunity to centrally scan all web content, including encrypted SSL content, and reject things that look malicious.

I don’t like to make a proxy my first line of defense. It can obscure and limit the effectiveness of some of the other tools. For example, if you see a DNS request or an outbound connection to a known malware domain, it’s a multi-step process to track it back to a particular workstation.

And I haven’t found that the scanning capabilities of most proxies are any better than those found on the best UTM firewalls anyway. The important thing a web proxy buys you is the ability to decrypt and inspect HTTPS (SSL) content.

Forensic packet capture

Finally, if I had all of the security tools we’ve already discussed and I really needed something more to help me deal with incident response following an attack, I’d look at forensic packet capture tools. These are carefully optimized and targeted network protocol analyzers that record interesting looking sessions. They then allow you to search through a massive historical database of these sessions.

Forensic tools are mostly useful during the incident response process when you’re trying to figure out what systems might have been affected by an attack, what credentials might have been compromised, and what data stolen or altered. That’s useful information, but it’s obviously not the starting point.

network evidence forensics security analysis

Photo: IAEA Imagebank on Flickr

Endpoint security

Malware sneaks onto a system in many different ways. The typical malware attack starts with some sort of dropper download. The dropper is a small piece of code whose main function is to get itself installed—somehow, somewhere—then to reach out to the C&C network for further instructions. Usually, the instructions will involve downloading additional malware modules, installing them, running them, and possibly reaching back again for more instructions.

The drop-and-reach-back methodology of most malware gives us many opportunities to catch attacks. First, we try to detect and block the suspicious domains. Then we try to detect and block the dropper download. Then we try to detect and quarantine the dropper software. Then we try to detect and block the C&C traffic. Then we try to detect and quarantine the other malware modules.

The first couple of items on that list are already covered by the UTM firewall and other tools we’ve already mentioned. So the most critical malware-specific tool in our defensive toolbox is good endpoint security.

An endpoint security tool is a mix of traditional anti-virus and behavior-based malware detection. Don’t bother with a straight anti-virus system anymore. Traditional anti-virus programs scan files and use signatures to spot anything that looks like a known virus. But modern malware isn’t always file-based, and it’s often able to change its appearance by repeatedly encrypting and re-encrypting itself. So on their own, signatures aren’t very useful.

However, signature-based detection can be supplemented with behavior-based indicators of compromise (IOC). Malware does malicious things like encrypting files, making strange entries in Windows Registry, or modifying system files. A good endpoint security program will monitor for these types of actions, as well as traditional file signatures.

Sandboxes

Another extremely useful modern anti-malware defense is a sandbox, which grabs a copy of anything being downloaded by any device. This could be email attachments, JavaScript code, a Windows executable file, a Flash animation—really anything.

The sandbox tries to unpack the files and run them in a special, isolated virtual machine. As it does so, it watches carefully for any signs of malicious behavior. The hope is to prevent malware from ever reaching a workstation, which is particularly important for those malware strains like crypto-viruses, which deploy as soon as they’re downloaded and immediately start destroying files.

Sandbox detection can usually be done so fast that the end user isn’t even aware their download was intercepted. However, sometimes malware can evade detection in a sandbox. Sandboxes keep getting better at detecting malware. Malware keeps getting better at detecting sandboxes. So a sandbox should never be considered a replacement for good endpoint security.

sandbox network infrastructure security isolation

Photo: F Delventhal on Flickr

Bringing network defenses together

Once you have several essential elements of network security up and running, you’ll quickly find you don’t have the resources to look at all of them. Firewall and IDS logs alone accumulate at rates of kilobits per second, even after filtering. Active Directory and DNS logs are often just as bad. There’s no way a human can monitor these systems in real time. Instead, you need a way of storing, filtering, and correlating messages into something meaningful.

Initially, it might be enough to use the management tools that come with your equipment. A good UTM firewall generally has a management GUI. You can always use the GUI on your domain controller to look at Active Directory logs. And most endpoint security systems also have a central console. But at a certain point, there are just too many different consoles, and they aren’t sharing information with one another.

This is where a security incident event management (SIEM) system becomes useful. Often the SIEM doubles as a searchable long-term storage system for log messages. Some organizations deploy these functions separately, but ideally, I like them to be together.

The SIEM is a single pane of glass that correlates information it receives from the various security systems you’ve deployed and presents you with all the relevant information about each event.

For example, if somebody is attacking a web server, you might receive relevant data about the attack from the firewall, the IDS/IPS, the WAF and perhaps also the Active Directory server. The SIEM rolls all of that information together so you don’t have to manually search through each of the different data sources to understand exactly what’s going on.

Effective security requires many tools working together. You’ll often hear the expression “defense in depth”, and this is what it means. One of the most important tasks is to identify your risk areas. Then allocate your budget and your time carefully to cover the highest risks first.

The other point that I really want to stress is that tools alone are not security. You can’t lock the door and expect that will keep out all the burglars. Somebody has to be looking at the tools, investigating every single anomaly and eliminating the threats as they appear. Even the risks will change over time, so you need to continually re-evaluate your highest risk areas to make sure they’re appropriately covered.