Page Cache Management > Pager Responsibilities
The pager is the only module that accesses (through your operating system's native IO APIs) native database and journal files. But, it neither interprets the content of databases, nor modifies the content on its own. (The pager may modify some information in the file header record, such as the file change counter.) It takes the usual random access/byte-oriented filesystem operations, and abstracts them into a random access page-based system for working with database files. It defines an easy-to-use, filesystem-independent interface for accessing pages from database files. The B+-tree module always uses the pager interface to access databases, and never directly accesses any database or journal file. It sees the database file as a logical array of (uniform size) pages.
Databases (except in-memory ones) normally reside on external storage devices like a disk, in the form of ordinary native files. When SQLite needs a data item, it reads it from the database file into the main memory, manipulates it in-memory, and, if needed, writes it back to the file. Normally, databases are very large compared to the available main memory. Because of the limited availability of the main memory, only a part of the memory is reserved to hold a (tiny) fraction of data from database file(s), and this reserved memory space is popularly called a database cache or data buffer; in SQLite terminology, it's called the page cache. The pager is the cache manager, among many other things (described next).
For each database file, moving pages between the file and the cache is the basic function of the pager as the cache manager. The page movement is transparent to the B+-tree and higher-level modules. The pager is the mediator between the native filesystem and those modules. Its main purpose is to make database pages addressable in the main memory so that those modules can access the page contents directly. It also coordinates the writing of pages back to the database file. It creates an abstraction so that the entire database file appears to reside in the main memory as an array of pages.
Apart from the cache management work, the pager does carry out many other functions of a typical DBMS. It provides the core services of a typical transaction processing system: transaction management, data management, log management, and lock management. As a transaction manager, it implements transactional ACID properties by taking charge of concurrency control and recovery. It is responsible for atomic commit and rollback of transactions. As a data manager, it coordinates reading and writing of pages in database files with the cache and does file space management work. As a log manager, it decides on the writing of log records in journal files. As a lock manager, it makes sure that transactions, before accessing a database page, have appropriate locks on the database file. In a nutshell, the pager module implements storage persistence and transactional atomicity. The interconnections between all the submodules of the pager are shown in Figure 3-1.
All modules above the pager are completely insulated from low-level lock and log management mechanisms. In fact, they are not aware of locking and logging activities. The B+-tree module sees every thing in terms of transactions, and is not concerned with how the transactional ACID properties are implemented. The pager module splits the activities of a transaction into locking, logging, and reading and writing of database files. The B+-tree module requests a page from the pager by the page number. The pager, in turn, returns a pointer to the page data loaded into the page cache. Before modifying a page, the B+-tree module informs the pager so that it (the pager) can save sufficient information (in a journal file) for possible use in future recovery, and can acquire appropriate locks on the database file. The B+-tree module eventually notifies the pager when it (the B+-tree) has finished using a page; the pager handles writing the page back to the file if the page was modified.
In this section, I present page cache management, and in the next, transaction management.