The Architecture (Servers, Software, and Roles)


In traditional logging configurations, the server software and roles involved are somewhat minimal. It is expensive to juggle logs and, as such, servers tend to have multiple roles so there are as few moving parts as possible. Alongside your cluster of web servers, you'll see one "logging server" that is responsible for a great many things: collection, archival, analysis, and so on.

In a configuration where logs are published and subscriptions are inexpensive, the opportunity for delegating responsibility and adding redundancy increases as the challenge decreases. This makes our current tasks easier and affords us the opportunity to be innovative with our logs, which we will discuss in Chapter 10, "The Right Tool for the Job."

The previously discussed logging architectures change shape and form as depicted in Figure 9.4.

Figure 9.4. Logging based on group communication.


First, let's get some nomenclature in place so that we are all talking about the same thing. The word cluster is too generic, and in this case it is ambiguous. Instead we will refer to the servers according to their roles:

  • Web serversServers responsible for serving web pages and publishing logs or the operations that they perform.

  • Log serversServers responsible for journaling (to permanent storage) logs of the events published by web servers. They are responsible for abiding by the retention and archival policies the business dictates for this data.

  • Log monitorsServers that subscribe to the log stream and perform some form of analysis on the stream. These monitors feed back into the overall enterprise monitoring solution implemented for the architecture and supply passive metrics and alerts to be acted on appropriately.

  • Casual monitorsServers that subscribe to the log stream occasionally but serve no core infrastructure purpose. Subscribers of this fashion would typically be systems administrators attempting to troubleshoot a problem or developers attempting to retrieve logs (errors and access) during development, staging, and occasionally production.

In systems administrator terms, think of the web servers as the system, the log servers as sar or some system accounting and auditing service, the log monitors as SNMP and SNMP traps, and the casual monitors as tools such as top, vmstat, iostat, ps, and so on. All these services and tools are vital to running a healthy systemnow you have something comparable for web logs.

The irony is that when someone asks a systems admin what is going on or what happened at a certain point in time on his system, he can answer the question quickly and easily. Typically, no such answer can be had quickly or easily in large web environments. It is imperative that we rectify this situation.

By separating the roles, we allow them to be fulfilled by different servers. This has the obvious advantage of allowing each server to have separately defined uptime requirements, service levels, mission criticality, and management responsibility.




Scalable Internet Architectures
Scalable Internet Architectures
ISBN: 067232699X
EAN: 2147483647
Year: 2006
Pages: 114

flylib.com © 2008-2017.
If you may any questions please contact us: flylib@qtcs.net