Recognizing Other Critical Problems

Sometimes it's obvious that you need to go into "emergency mode" and try to preserve your data and rescue your systemfor example, if you suffer a complete hard drive failure or if you are unable to boot Linux at all. There are, however, two times when it is desirable to preemptively take drastic action to save your system even though your system still appears to be mostly functional:

• Situations in which your hard drive is displaying symptoms of file system corruption or approaching failure

• Situations in which you can determine that your Linux system might have fallen victim to network hacking or Trojan horse or worm programs

Left unchecked, either of these situations can lead to eventual unexpected downtime, data loss, or even data theft. The following sections detail how to spot these types of situations and what to do should they occur.

Recognizing File System Trouble

File system corruption occurs when the organization of the data on your hard drive is unexpectedly damaged, thereby causing Linux to begin to lose track of where some files begin or end, or of which files contain what data. After your file system becomes corrupt, continued access to the disk usually increases the spread of file system corruption, thereby endangering and potentially damaging still more files with every passing minute.

A few telltale signs indicate that you are likely beginning to experience file system corruption:

• You begin to encounter files that contain garbage, a mishmash of nonsensical data that doesn't represent the content that you were expecting a file to havethe content that you remember actually storing in the file in the first place.

• You begin to encounter directory problemsfilenames containing garbage, spontaneously appearing or disappearing files, files or directories that can't be removed or edited even when permissions would seem to indicate that such things should be possible.

• You find files that, when accessed, seem to crash your computer system or the program you're using every time, without fail, in the same way.

• You begin to lose files or directories entirely, even though you have not deleted them; they're just suddenly gone.

If you believe that you are experiencing file system corruption, follow the steps earlier in this chapter in the "Dealing with Catastrophic Failures" section to start the rescue tool and perform checks on your file systems using e2fsck. Doing this should repair the corruption that has occurred on your file system and make it safe for use againalthough any data that was corrupted is lost forever.

Hard drives can tell you when they're sick

Many hard drives in modern computers are capable of reporting on their status, and telling you when they're sick. Linux includes a command, smartctl, that can ask a hard drive to test itself and report on its health. To test a hard drive, type the following as root:

 /usr/sbin/smartctl -t long /dev/device 

Remember to replace /dev/device with the actual device in question, for example /dev/hda for the first IDE hard drive in the system. The smartctl command will tell you how long the test will take, and will then disappear into the background and run the test. Once the listed amount of time has elapsed, run the following command as root:

 /usr/sbin/smartctl -a /dev/device 

Again, remember to replace /dev/device with the actual device. This command will generate lots of output, but only the last few lines are important. If the Status column for the test you just ran shows Completed without error, then your hard drive is fine. If it reports any kind of error or failure instead, back your hard drive up and replace it immediately!

If you want to learn more about smartctl, read its manual page using the man command.

If e2fsck is unable to find any problems in your Linux file systems, whatever symptoms you are experiencing are not due to file system corruption. In some cases, they might be due to malicious activity (we deal with this topic in the next section); in other cases, they might simply represent an aspect of the normal functioning of the Linux operating system that is unfamiliar to you.

Data Corruption Is Usually a Warning of Things to Come!

If you are experiencing repeated bouts of file system corruption, you find that your system often hangs with the hard drive activity light on, or you begin to find log entries that refer to I/O errors or missing sectors on one of the devices used by your Linux file system, you are likely going to experience a catastrophic hard drive failure in the near future.

You should back up your data immediately and replace your hard drive to avoid unexpected downtime and/or data loss!

Recognizing Malicious Network Activity

There is one other type of critical problem that some unfortunate Linux users no doubt experienceparticularly those who are connected to busy networks or directly to the Internet. Linux systems are often targeted by hackers or other types of malicious network users. The reason is that most Linux systems on networks are not just PCs, but are typically serversconfigured to accept incoming requests while providing important services to many users.

In general, Linux should be very good at repelling attacks, especially if you have properly configured your firewall as described in Chapter 30, "Security Basics." However, from time to time, it is inevitable that some attacks are successful. Recognizing the symptoms of having been successfully attacked can help you avoid extensive amounts of data loss or unwilling participation in Internet crimes. As long as your Linux system is connected to a network, you should stay vigilant in watching for all the following:

• Newly appearing SUID/SGID files, which indicate that someone is trying to access or has already accessed root-level functionality on your system. For more information on SUID/SGID special permissions, refer to "Understanding Special Permissions" in Chapter 30.

• The appearance of new accounts in the /etc/passwd file or new groups in the /etc/group file that you did not create.

• System log records of users remotely logging in using accounts that don't seem to exist or that you did not create.

• Unexplained heavy network traffic that doesn't appear to be connected to any service you're running, or unfamiliar processes in the output of the ps command that always eventually return even after you kill them repeatedly.

• Any of the previously mentioned issues combined with symptoms of file system corruptiondisappearing files, undeletable files, or unreadable files or directories in spite of correct permissions, and so on.

If you find yourself experiencing any of these symptoms, your system has likely been compromised. Unfortunately, this counts as a catastrophic failure. When a computer system is compromised by a malicious network user, he usually replaces many of the operating system components with modified components, which allow them to steal your data, use your computer in attacks on other computers, or perform other unwanted behavior.

If you think your system has been compromised, you should immediately shut down your computer system to prevent further unknown malicious activity. Boot into rescue as described in "Dealing with Catastrophic Failures" earlier in this chapter, taking care not to enable networking. Save your important data files only (no programs or applications; they might have been replaced by harmful dupes) using the techniques described in "Backing Up and Restoring Your Data" earlier in this chapter. Then reinstall Fedora Core 4 from scratch as described in Part I of this book and restore your data from the backups you made.

After your Linux system is running again, review Chapter 30, "Security Basics," and implement the techniques described there. Afterward, refer to Chapter 32, "Keeping Fedora Core Updated," to ensure that all of the latest updates and security measures have been installed on your system.