Recipe 9.3 Defining Queue Groups

Problem

Special configuration is required to define different mail queues with unique characteristics.

Solution

Create the directories for the queue groups. The queue group directories must be subdirectories under the default queue directory. In this example, four queue group directories are created:

 #  cd /var/spool/mqueue  #  mkdir slowq fastq.1 fastq.2 fastq.3  #  chmod 700 slowq fastq.?  

Add QUEUE_GROUP macros to the sendmail configuration to use the queue group directories created in the first step. Here are sample QUEUE_GROUP macros:

 dnl Define a queue group QUEUE_GROUP(`slowq', `Path=/var/spool/mqueue/slowq') dnl Define a queue group QUEUE_GROUP(`fastq', `Path=/var/spool/mqueue/fastq.*, I=10m, F=f, R=3') 

Build sendmail.cf . Copy it to /etc/mail/sendmail.cf , and restart sendmail. See Recipe 1.8.

Discussion

The first step in Recipe 9.3.2 creates four queues in the /var/spool/mqueue directory: slowq , fastq.1 , fastq.2 , and fastq.3 . These queues are then referenced by the QUEUE_GROUP macros as the slowq and fastq queue groups. The directories for queue groups are created in the base path of the default queue. When a single directory is used for the default queue, the base path is the path specified by the QueueDirectory option. When QUEUE_DIR is used to define multiple queue directories, as it was in Recipe 9.1, the default queue pathname specified by the QueueDirectory option and the base pathname are not exactly the same. For example:

 define(`QUEUE_DIR', `/var/spool/mqueue/queue.*') 

This QUEUE_DIR define sets the default queue to /var/spool/mqueue/queue.* . The base pathname in this case is /var/spool/mqueue , not /var/spool/mqueue/queue.* . The directory that contains the subdirectories used for multiple queues is the base path in which the queues used for other queue groups should be placed.

Use the QUEUE_GROUP m4 macro to declare queue groups. The syntax of the QUEUE_GROUP command is:

 QUEUE_GROUP(`   groupname   ', `   equates   ') 

The groupname is an arbitrary name assigned to a queue. The groupname is used to reference the queue from within the sendmail configuration and in QGRP : records in the access database. The default queue is named mqueue , and it can be referenced by that name.

The equates are a comma separated list of queue attributes written in the form keyword = value . The equates field must be present, although it can be explicitly empty. The QUEUE_GROUP commands in Recipe 9.3.2 have values in the equates field. Here is an example with an empty equates field:

 QUEUE_GROUP(`normalq', `') 

Notice the single quotes enclosing the empty equates field in the example above. When the equates field is empty, the queue group inherits all of the attributes of the default queue, mqueue . Given the QUEUE_GROUP command shown above, normalq would use the same queue directory and all of the same attributes as the mqueue queue group.

All queue groups inherit the attributes of the default queue. The keyword = value equates defined in the equates field are used to override the default values for specific queue attributes. Table 9-1 lists the available keywords and describes the queue attribute associated with each keyword.

Table 9-1. Queue group attributes

Keyword	Function
Flags=	Sets optional runtime flags.
Interval=	Specifies the time interval between queue runs.
Jobs=	Limits the number of envelopes processed during a single queue run.
Nice=	Defines the nice value used for the queue run.
Path=	The full pathname of the queue directory used for this queue group.
Recipient=	Sets the maximum number of recipients allowed for one envelope.
Runners=	Defines the maximum number of queue processors that can be used in any one queue run.

The effect of the equates listed in Table 9-1 can be altered by other configuration options:

confFAST_SPLIT

The number assigned to this option sets the maximum number of envelopes that can be delivered during the initial delivery, without regard to the value set by Jobs . Additionally, assigning a positive number to this option prevents MX lookups of recipient addresses when the envelope is being split. Skipping the MX lookup can speed up envelope processing, and limiting the number of delivery processes can enhance performance on a heavily loaded system. By default, sendmail performs MX lookups when splitting envelopes, and it runs as many delivery processes as needed to deliver all of the envelopes in parallel.

confMAX_QUEUE_CHILDREN

The number assigned to this option is the maximum number of queue processes allowed across all queue groups. The upper limit set by this option is not affected by the values set for Runners or Flags . If confMAX_QUEUE_CHILDREN is set to 10 and the total number of Runners is set to 15, sendmail limits the number of queue processes to 10. By default, confMAX_QUEUE_CHILDREN is set to 0, which means that sendmail allows an unlimited number of queue processes (up to the limit set by Runners ).

confMAX_RUNNERS_PER_QUEUE

This option sets the default maximum number of queue runners allowed per queue group. The value set by Runners in a QUEUE_GROUP macro overrides this default value for a specific queue group.

confMIN_QUEUE_AGE

This option sets the minimum amount of time that a queued message must wait in the queue before a delivery attempt is made. This value is not affected by how frequently the queue is processed. If the Interval is set to 10 minutes (10m) and confMIN_QUEUE_AGE is set to 1 hour (1h), it is possible for a message to sit in the queue for 6 queue runs before sendmail attempts to deliver it. This option is used to reduce the load on systems that have very large queues, yet use a short queue interval. By default, sendmail attempts to deliver all of the mail in the queue (up to the limit set by Jobs ) every time the queue is processed.

confNICE_QUEUE_RUN

This option sets the default nice value used for queue runs. It can be overridden for a specific queue group by using Nice in a QUEUE_GROUP macro.

confMAX_QUEUE_RUN_SIZE

This option sets the maximum number of queue messages that will be processed during a queue run across all queue groups. By default, sendmail attempts to deliver all of the mail in the queue. This option sets an upper limit on the number of messages sendmail will attempt to process, much like Jobs does for a single queue group.

The sendmail configuration in this recipe contains two QUEUE_GROUP commands:

 QUEUE_GROUP(`slowq', `Path=/var/spool/mqueue/slowq') QUEUE_GROUP(`fastq', `Path=/var/spool/mqueue/fastq.*, I=10m, F=f, R=3') 

The first QUEUE_GROUP command defines a queue group and assigns it the name slowq . The path to the slowq group's queue directory is defined as /var/spool/mqueue/slowq . All of the other attributes used by slowq are the default queue attributes.

The second QUEUE_GROUP command defines the fastq queue group. The fastq group uses multiple queues, as indicated by the asterisk suffix on the queue directory's pathname. The asterisk is used to define multiple queues for any queue group in exactly the same way that it is used to define multiple queues for the default queue group. See Recipe 9.1 for more about multiple queues and the asterisk suffix.

In addition to the Path attribute, the second QUEUE_GROUP command defines three other attributes: Interval , Flags , and Runners . Notice that the keyword in each equate does not need to be spelled out; only the first character of the keyword is significant. Thus I=10m is equivalent to Interval=10m , and the Path attribute could have been set with P=/var/spool/mqueue/fastq.* . Also, notice that each item in the equate list is separated by a comma. The whitespace used in the example enhances readability, but it is not required.

Setting the attribute I=10m means that sendmail will process the queues in this queue group every 10 minutes. Setting the f flag, F=f , tells sendmail to fork queue runners for each queue, up to the limit of queue processes defined with the Runners keyword. f is the only value currently valid for Flags . Always use F=f when you use the Runners keyword.

Setting R=3 means that sendmail can use up to three separate processes to process the queues in this queue group. This recipe created three queues ( fastq.1 , fastq.2 , and fastq.3 ) for the fastq group. Setting R=3 means that each queue will have a single queue processor for each queue run. Setting R=15 would have dedicated five processes to each queue during each queue run. However, a large number of queue runners is not always a good thing. To find out why, see Recipe 9.5.

The QUEUE_GROUP macros create Q commands inside the sendmail.cf file. The Q commands created by this recipe are:

 #  grep '^Q' recipe9-3.cf  Qslowq, Path=/var/spool/mqueue/slowq Qfastq, Path=/var/spool/mqueue/fastq.*, I=10m, R=3 

Queue groups are used by queue management features. For example, you can configure a sendmail mailer to use a particular queue group as its default mail queue. The Q parameter of the mailer definition tells the mailer which queue group it should use. All mailers have an m4 define for the Q parameter; all of these defines are used in exactly the same way; and most have names in the form of NAME _MAILER_QGRP , where NAME is the mailer's internal name. For example, use PROCMAIL_MAILER_QGRP to set Q if the name of the mailer is procmail .

Additionally, queue groups can be used in the access database. Recipe 9.4 shows an example of this.

See Also

Recipe 9.1 covers the QUEUE_DIR command and discusses multiple queues. Recipe 9.2 covers the df , qf , and xf subdirectories, which can be created in the queue directory of any queue group. The sendmail book covers queue groups in Section 11.4, confQUEUE_FILE_MODE in 29.9.84, confFAST_SPLIT in 29.9.32, confMAX_QUEUE_CHILDREN in 29.9.65, confMIN_QUEUE_AGE in 24.9.72, confMAX_QUEUE_RUN_SIZE in 24.9.66, confNICE_QUEUE_RUN in 24.9.74, and confMAX_RCPTS_PER_MESSAGE in 24.9.67.