Limiting Network Risks

Understanding the Importance of System Security

Before we can reasonably discuss ways to make your Mac OS X machine more secure, it's important to understand why you should care and what you're facing.

You might be wondering, "Why bother with security?" You've never cared about securing your Mac before. In fact, prior to Mac OS X, security has rarely been an issue for a Mac owner. However, as a Unix-based operating system, Mac OS X brings with it not only the advantages of a Unix operating system it also brings the disadvantages. Unfortunately, one of those disadvantages is security, which has always been a problem for networked, multitasking operating systems. This is unsettling at the beginning, but the simple fact is that nothing can be done to make any network-connected Unix machine (Mac OS X or otherwise) completely secure. To paraphrase one system administrator's feeling about securing a Unix machine, "I'd pull the plug and the network, put the machine in a safe, fill the safe with concrete, lock it, and drop it in the middle of Hudson Bay and even then I wouldn't be sure."

That take might be a little extreme, but you get the point. If your machine is on a network, and/or the hardware is physically accessible, it's vulnerable to something somehow. Your best efforts at security are only capable of increasing the effort, time, and creativity required for a cracker to access your hardware you simply can't make it impossible.

Crackers target anything they can find on your computer or network. Your Mac OS X computer runs a variety of server processes that enable it to communicate with the outside world. A single programming flaw in one of these daemons could open administrative access to anyone. If a cracker can't find a way in directly, he can direct attacks on your network hardware or your ISP's hardware. Switches, routers, and other devices are also susceptible to attack. If your computer has blocked access to outside networks, the intruder might resort to IP spoofing to fake his real location.

The threat of computer break-ins is very real and a very real concern even if you're doing everything right. For example, earlier versions of Sendmail (the mail server that used to be included with the Mac OS X installation) suffered from a bug that allowed remote crackers to send a specially formatted email to the server and force it to execute pieces of code with full administrator privileges. Imagine it: A person, anywhere in the world, could potentially take over control of your computer by sending it an email message. Even experienced administrators were at risk from this bug. Although Sendmail has since been patched, this is an excellent example of the type of attack that's possible. For more information about this particular exploit, check out http://ciac.llnl.gov/ciac/bulletins/h-23.shtml.

NOTE

Crackers, not hackers: There seems to be a popular misconception that the term hacker means someone who breaks into computers. This hasn't always been the case, and annoys the hackers out there who do not break into computers. To many hackers, the term hacker means a person who hacks on code to make it do new and different things frequently with little regard to corporate-standard good programming, but also frequently with a Rube Goldberg like, elegant-but-twisted logic.

Hackers, to those who don't break into computers, are some of the true programming wizards the guys who can make a computer do almost anything. These are people you want on your side, and dirtying their good name by associating them with the average scum that try to break into your machine isn't a good way to accomplish this.

So, to keep the real hackers happy, we refer to the people who compromise your machine's security as crackers in this book. We hope you follow our lead in your interactions with the programmers around you.

You might still be wondering, "Why bother?" If the machine can't be made completely secure, why try? When your machine is brand new and not very customized, perhaps you can afford to have that attitude. Reinstalling the operating system is not all that traumatic at that stage. However, because you've made it to this chapter, you've seen Mac OS X from many angles and perhaps even implemented some customizations to your system. Hence, you might not feel like doing it all over again.

So, if you can't be completely secure, what can you do? You can be reasonably secure. Exactly how secure is reasonable differs from case to case, and depends on a wide range of factors. Later in this chapter, we discuss some of these factors and how to assess your needs. For now, understand that when designing security measures, there's a threshold beyond which expending extra effort does not produce a sufficient increase in security to make that effort worthwhile. You can liken this to the somewhat facetious advice given for how to protect yourself when hiking in bear country "always hike with someone who runs slower than you do." Your goal in securing your system is to make your machine and your network less attractive than the next guy's system to the cracker. In this chapter, we look at some ways to accomplish that goal.

Types of Attackers

We divide the types of attackers you're likely to meet into three subsets. Although it might not seem obvious at first consideration, the variety that you're the most likely to meet regardless of the type of data you're protecting is frequently both the most and least dangerous.

The Motivated Cracker

The type of cracker you're probably least likely to meet is the dedicated and motivated professional or amateur cracker with a mission. This person might be an industrial spy trying to discover your company's trade secrets, a student trying to change his grade, or a hobbyist who simply finds your security measures a challenge.

CAUTION

Obscurity isn't to be relied on, but it sure doesn't hurt!

Many system administrators, and even OS vendors, have historically had a bad habit of attempting to implement security measures based only on the fact that people didn't know about the holes in them. This is a concept known as security through obscurity, and is generally considered a bad thing to rely on. On the other hand, obscurity in addition to other security measures certainly doesn't hurt. Keeping your system and security measures low profile is always a good idea. If you have a system with extra-tight security measures, telling people just how tight they are is the surest way to attract the person who delights in cracking systems "just because."

Rely neither on security through obscurity nor on the invincibility of your security measures. The best way to motivate a person to try to crack your system just because it's there is to let on that you think it can't be done.

The motivated cracker isn't likely to leave a large amount of evidence of his comings and goings. The types of this cracker vary between unlikely to do any significant damage (other than observing your data) to making insidious and difficult-to-detect modifications to the contents of your system.

To defeat this type of cracker, you must understand his motivation and either remove it or resign yourself to a constant battle to stay ahead. The only way to actually stop these people permanently is to track them down and pursue legal remedies against them.

The Casual Experimenter

The next type of attacker you're likely to meet is the casual experimenter. These individuals don't usually intend any significant harm and aren't usually very motivated to invade your system. They're frequently just a bit overcurious, and are trying out something that they stumbled across somewhere on the Internet. This doesn't mean that they're not dangerous their lack of intent can't prevent simple typing mistakes that can be disastrous to a person with root access. Thankfully, these individuals aren't usually too difficult to defend against because they're usually not particularly sophisticated. They also don't tend to be worth investing much effort in tracking down legally.

The Script Kiddie

The most common type of cracker doesn't even deserve to be called a cracker. Historically, crackers have been frequently thought of as Robin Hood characters, with a sort of romantic fascination with their exploits. Not to minimize the impropriety of the legendary crackers' actions, but you can appreciate the creativity and tenacity of these individuals without approving of their actions. By the standards set by the crackers of old, the vast majority of today's crackers barely qualify as cracker-wannabe wannabes.

Today's prototypical cracker is a young adult with too much free time who found a cracking script on a Web site somewhere and is trying to use it to show his friends he's an "lEEt HaCkEr dOOd." In fact, these new crackers are called script kiddies.

These individuals are both a trivial and significant concern. If you keep your system up-to-date and pay attention to the latest cracking scripts and to the patches against them, you're almost invulnerable to actual intrusion at the hands of these people. They don't generally try anything more complicated than running a script they've borrowed from someone else. So, if you keep your system secure against these scripts, you're usually secure against cracker wannabes. That doesn't mean they're completely innocuous because they can still consume your network resources while trying to break into your system.

However, they can be very dangerous if you don't keep your system completely up-to-date because there are so darned many of them, and because they're basically glory hounds interested in nothing more than self-aggrandizement. To give you a perspective on the magnitude of their numbers, here at The Ohio State University, we see unsophisticated cracking attempts of this sort multiple times every week, directed at the thousands of machines on campus. A Linux machine, installed out of the box and not immediately secured against intrusion, stands a better than 50% chance of being cracked within 24 hours if it's attached to the network here. Fortunately, a Mac OS X machine installed out of the box is a bit more secure, but that doesn't mean it's invincible.

Also, because their basic goal is self-aggrandizement and also because they don't get that much glory for using someone else's script, these people are rarely content to break into a system, tread lightly, and leave without a trace. Instead, they're more likely to erase the contents of your hard drive, or replace your corporate web page with pornography so that they have some evidence to show their "lEEt HaCkEr dOOd" friends.

Securing your system against these attacks is simply a matter of watching every security discussion list and cracker site for signs of trouble and postings of new cracking scripts, and then applying every security patch as quickly as it becomes available. Simple, no? As satisfying as it might be to track them down and squash them like the insects they are, it's usually impractical. Ninety percent of these attacks come from users with transient accounts, and the best you'll usually do is chase them to a different account. If you do happen to catch one, please do let the Internet system administration community know the newsgroup alt.sysadmin.recovery is a good venue public lynchings are always well attended.

Types of Attacks

Next, let's look at what methods attackers might use to access your machine. This is especially important if your machine is connected to an unprotected network or if it serves as a firewall.

Software and Operating System Flaws

The most common type of attack you'll encounter is one that attempts to exploit flaws in application or operating system software. There's probably not much you can do about most software flaws other than hope that the providers find and fix the problems promptly. Although this is a problem from a security standpoint, the positive side is that if you're spending the time to watch the cracking web pages and the security mailing lists, you'll know about the problems as soon as the crackers do. With the information you get from these sources and your understanding of the special risks your site incurs, you can assess whether leaving that software on your machine is an acceptable risk until the vendor fixes it. Note that Apple does put out semi-regular security updates via Software Update.

You need to be aware that some of these flaws require prior access to your system to exploit, whereas others can be exploited from a remote site over the network. Don't make the mistake of assuming that because no one has actually logged in to your machine, you can't be or haven't been attacked.

Brute Force Attacks

Although not a particularly elegant form of attack, the brute force attack is one that you can only partially prevent. In its simplest form, this attack is a cracker attempting to log in to a system by sitting at a machine and iteratively typing attempts at passwords into the prompt. There's not much you can do to keep people from trying this sort of thing.

Keep an eye on the system logs, and you'll see the trivial attempts as they occur. However, there's typically much more danger from this sort of attack when a cracker manages to get your password file and can attempt to crack the passwords on his own machine at his leisure. To prevent this, some systems use a shadow password facility to keep the password file from being readable by a normal user. The early releases of Mac OS X did not include a shadow password facility, making it important to consider restricting the executable permission on NetInfo utilities, such as nidump and niutil, to root only. Starting around Mac OS X 10.2, a shadow password facility was implemented.

Denial of Service

Denial of service (DoS) attacks are generally destructive attempts rather than attempts to access your system. When the attacks come from a network of multiple machines, they're known as distributed denial of service attacks. Both types of attacks are targeted at preventing you and your users from using your machines instead of allowing an intruder access. Because this can be effectively accomplished without the aid of your system, there's little that you can do about many of these attacks. Because a denial of service attack rarely results in an actual security violation or illegitimate access of your system, your best defense is detection and elimination.

Although the specific methods employed in different varieties of denial of service attacks vary, they share a common feature: the exhausting of some service or resource that your machines require or provide. Why do people do this? Good question. You might expect this sort of behavior from a disgruntled ex-employee attacking a former employer or from a student who thinks it's a funny practical joke. Less expected are denial of service attacks that seem to happen as random vandalism just because the attacker can do it.

Certain denial of service attacks can be mitigated or prevented with software or hardware updates. In general, these updates tend to be installation of operating system patches to disallow certain types of connections or installation of filtering hardware to block certain types of network traffic. Denial of service attacks range from flooding users' email, to absorbing all your HTTP server connections, to running your printer out of paper, to flooding your network with ICMP ping packets. Unfortunately, there's little you can count on to be reliably effective other than constant vigilance and swift retribution.

CAUTION

Not all denial of service attacks are devoid of security risks. Some attacks are targeted at services that are known to break inelegantly and that sometimes allow privileged access when broken. Just because a denial of service attack looks relatively harmless, don't allow yourself to be complacent. It could be less harmless than it looks, or it could be a prelude to more unpleasant attacks.

Most attacks can generally be thwarted by taking the following precautions:

• Vigilance is your best defense against denial of service attacks. Watching your machine's load and network performance are the best ways to discover an attack in progress.

• Consider building a monitoring web page that collects this information from all your machines and provides you with a continuously updated representation of the state of the world.

• A gateway between your cluster and the outside world can be used to deny traffic from a problematic outside host. Unfortunately, this can't prevent an outside host from effectively denying your network services just by banging away at your gateway until your network bandwidth is consumed.

• Enlist the help of system administrators upstream from your site in tracking and blocking denial of service attacks as they occur. Because you can't effectively prevent users on the other side of the world from running flood-ping against your machines, you'll need to find someone between you and them who can help.

Physical Attacks

Many administrators in charge of system security overlook this area of obvious weakness in their security strategy. Computers don't need to be logged in for a person to access their data. A person unscrupulous enough to crack your machines is just as happy to simply yank a hard drive out of your machine to steal the data on it. These sorts of attacks are usually easy to detect, but can cause significant downtime while critical hardware is replaced.

Although distributed computing and distributed storage are popular in certain environments, if security is a goal, especially data security, you should severely restrict access to all hardware with mission-critical data.

By far the easiest physical attack on your hardware is the power switch or reset button combined with the capability to boot the machine into single-user mode without a password or to boot off of a device specified at startup, also without a password. When in single-user mode, an attacker can get a dump of your passwords, change your root password, and so on.