The Linux Enterprise ClusterBuild a Highly Available Cluster with Commodity Hardware and Free Software

by Karl Kopper

San Francisco

Copyright © 2005 by Karl Kopper.

All rights reserved. No part of this work may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying, recording, or by any information storage or retrieval system, without the prior written permission of the copyright owner and the publisher.

Printed on recycled paper in the United States of America

1 2 3 4 5 6 7 8 9 10 - 07 06 05 04

No Starch Press and the No Starch Press logo are registered trademarks of No Starch Press, Inc. Other product and company names mentioned herein may be the trademarks of their respective owners. Rather than use a trademark symbol with every occurrence of a trademarked name, we are using the names only in an editorial fashion and to the benefit of the trademark owner, with no intention of infringement of the trademark.

Publisher: William Pollock

Managing Editor: Karol Jurado

Production Manager: Susan Berge

Cover and Interior Design: Octopod Studios

Developmental Editor: William Pollock

Technical Reviewers: Brian Elliott Finley, Peter Holzleitner, Simon "Horms" Horman, Chuck Lever, Joseph Mack, Matt Massie, Martin Pool, Alan Robertson, John Mark Walker, and Brian Ward

Copyeditors: Andy Carroll and Bonnie Granat

Illustrator for part openers: Nate Grundmann

Compositor: Riley Hoffman

Proofreader: Stephanie Provines

Indexer: Ted Laux

For information on book distributors or translations, please contact No Starch Press, Inc. directly:

No Starch Press, Inc.
555 De Haro Street, Suite 250, San Francisco, CA 94107
phone: [phone]; fax: [phone]; [e-mail]; http://www.nostarch.com

The information in this book is distributed on an "As Is" basis, without warranty. While every precaution has been taken in the preparation of this work, neither the author nor No Starch Press, Inc. shall have any liability to any person or entity with respect to any loss or damage caused or alleged to be caused directly or indirectly by the information contained in it.

Library of Congress Cataloging-in-Publication Data

 Kopper, Karl.   The Linux Enterprise Cluster : build a highly available cluster with commodity hardware and free software / Karl Kopper.        p. cm.    Includes index. 

ISBN 1-59327-036-4

1. Linux. 2. Parallel processing (Electronic computers) 3. Electronic data processing--Distributed processing. 4. Cluster analysis. I. Title.

QA76.58K67 2005

005.26'8--dc22

2003023352

This book is dedicated to all of the free software developers who wrote the software that made it possible for me to build a free, highly available cluster so I no longer had to work nights and weekends to keep the business system up and running.

Technical Reviewers

Alan Robertson (Heartbeat)

Simon "Horms" Horman (Heartbeat + LVS)

Joseph Mack (LVS)

Matt Massie (Ganglia)

Charles Lever (NFS)

Martin Pool (SSH + rsync)

Brian Elliot Finley (SystemImager)

Peter Holzleitner (Mon + SNMP)

John Mark Walker (RPM, FTP, Booting, Linux Administration)

Brian Ward (Linux Kernel)

ACKNOWLEDGMENTS

A courageous management team made it possible for me to deploy the cluster described in this book into production—to my knowledge the first of its kind to be used to support an American brick-and-mortar business. I would especially like to thank Kevin Michel and Bill Dettmann for the faith they placed in the people who worked for them, Jim Gallops for standing up and defending GNU software when it counted the most, Ken Paradox for patiently correcting the many mistakes I made when I was trying to do what I said I was going to do, and Francis Hamilton for testing the LPRng printing system at the peril of his own efficiency.

Technical guardian angels from each of the projects helped to correct me when I strayed into inaccuracies. I would especially like to thank Alan Robertson (Heartbeat) for helping me when I misunderstood the syntax and structure of the Heartbeat package; Simon Horman (The Linux Virtual Server, Heartbeat, ldirectord, and rpm dependencies) for going the extra mile (then two, then three; how many extra miles was it?); Matt Massie (Ganglia) for his gentle corrections and helpful ideas; Joseph Mack (The Linux Virtual Server) for his inspiration, advice, and support; Martin Pool (Open SSH and rsync) for putting up with my wayward ways; Charles Lever (NFS) for helping me to get over the really big hurdles; Brian Elliot Finley (SystemImager) for his terrific technical review; Peter Holzleitner (Mon and SNMP) for a truly selfless act; John Mark Walker (Linux utilities and the Linux boot process) for the grunt work; Brian Ward (The Linux Kernel) for saving me from my own folly; Dr. Patrick Powell (LPRng) for squashing every bug before they could do any harm; Mark Sherman (Linux novice) for lending a hand early on and being my kernel compile guinea pig; and Richard Stallman (Free Software Foundation) for clarifying the definition of the terms Linux, GNU, and free.

Also, a special thanks to all of the editors at No Starch press for catching the inaccurate, the obfuscated, and the unintelligible: William Pollock (editor and publisher, No Starch Press) for his uncompromising approach to every poorly formed idea; Susan Berge (production manager) for catching the pieces, keeping things on track, putting up with my eleventh hour scrambles, and then catching the pieces again; Karol Jurado (managing editor) for her graceful coordination of the cadre of technical reviewers; Riley Hoffman (compositor) for making the whole greater than the sum of the parts, redrawing every figure in the book, putting up with my criticisms of a job well done, and then doing it again; Patricia Witkin (communications manager), Leigh Poehler (rights manager/sales analyst), and Christina Samuell (administrative assistant) for smoothing out the bumps created by one delay after another, and then working hard to get the word out; Andy Carroll (editor) for a phenomenal job of cleaning things up and catching what so many others had already missed; and Bonnie Granat (editor) for seeing things all the way through to the end, and for her patience and understanding when she had to ask me and then ask me again.

On a personal note, I would like to thank my wife for letting me take time away from our relationship over the past four and a half years to sit alone at a computer, and my friends and family for helping me to stay on the path.

This book has been given so much support and attention that if any inaccuracies—technical or otherwise—remain they can certainly be attributed to me.

Karl Kopper
Sacramento, CA

UPDATES

Visit http://www.nostarch.com/cluster.htm for updates, errata, and other information.

ABOUT THE CD -ROM

The CD-ROM includes copies of the stock Linux 2.4 and 2.6 kernels with the LVS kernel modules; the ldirectord software and all of its dependencies; the Mon monitoring package, monitoring scripts, and dependencies; the Ganglia package; OpenSSH; rsync; SystemImager; and Heartbeat. The latest versions of all these software packages can be downloaded from the Internet for free.

The CD-ROM also includes copies of all the figures and illustrations used in the book so you can use and modify them to suit your needs.

CD LICENSE AGREEMENT

This CD-ROM bundled with the book The Linux Enterprise Cluster, ISBN 1-59327-036-4, contains electronic versions of all figures and illustrations ("Figures") that appear in the book, as well as a collection of relevant software. See the specific license for each software package located in the source files on the CD. All software is published under the GNU General Public License (GPL), http://www.gnu.org/copyleft/fdl.html, except Ganglia, which is released under the BSD license.

The Figures on the Linux Enterprise Cluster CD-ROM are released under the Creative Commons Attribution-NonCommercial-ShareAlike License (http://creativecommons.org/licenses/by-nc-sa/2.0). This license applies only to the Figures on the CD-ROM.

You are free to copy, distribute, display, and perform the Figures contained on the CD-ROM and to make derivative works from them but you must give the original author credit, you may not use this work for commercial purposes, and if you alter, transform, or build upon the Figures, you may distribute the resulting work only under a license identical to this one.

The penguin art is owned by Karl Kopper and released under the Creative Commons license.

For any reuse or distribution, you must make clear to others the license terms of this work. Any of these conditions can be waived if you get permission from the copyright holder.

No Warranty

The software provided is free of charge and without warranty. The software is provided "as is" without warranty of any kind, either expressed or implied. The software is provided without warranty for merchantability or fitness for any purpose. The entire risk as to the quality and performance of the software is with the user.

If any software provided proves to be defective or insecure or to cause damages of any kind, the user assumes all cost of repair.