Maintaining Data Integrity and Accuracy

When you add, modify, or delete table data, it's important that the additions and changes you make to the data don't conflict with the normalization rules that you used to create the database. One of the most vexing problems facing users of large RDBMs is "unclean data." Over time, data-entry errors and stray records accumulate to the point where obtaining accurate historical information from the database becomes difficult or impossible. Software vendors and database consultants have created a major-scale "data cleansing" business to solve the problem. You can avoid the time and expense of retroactive corrections to your data by taking advantage of Jet and SQL Server features that aid in preventing errors during the data-entry process.

Referential Integrity

Maintaining referential integrity requires strict adherence to a single rule: Each foreign key value in a related table must correspond with a primary key value in a base (primary) table. This rule requires that the following types of modifications to data be prevented:

• Adding a record on the many side of a one-to-many relationship without the existence of a related record on the one side of the relationship, for example, adding a record to the Orders table with a CustomerID value of BOGUS when no such customer record exists in the Customers table

• Deleting a record on the one side of a one-to-many relationship without first deleting all corresponding records on the many side of the relationship, for example deleting Around the Horn's Customers record when the Orders table contains records with AROUT as the CustomerID value

• Changing the value of a primary key field of a base table on which records in a related base or linking table depend, such as changing AROUT to ABOUT in the CustomerID field of the Customers table

• Changing the value of a foreign key field in a linking table to a value that doesn't exist in the primary key field of a base table, for example changing AROUT to ABOUT in the CustomerID field for OrderID 10355

Note

You also must avoid changing the primary keys of or deleting one of two tables in a one-to-one relationship.

A record in a related table that doesn't have a corresponding foreign key value in the primary key of a base table is called an orphan record. For example, if the CustomerID value of a record in the Orders table is ABCDE and there's no ABCDE value in the CustomerID primary key field of the Customers table, there's no way to determine which customer placed the order.

Jet and SQL Server 2000 databases offer the option of automatically enforcing referential integrity when adding or updating data. Cascading updates and deletions are optional. If you specify cascading updates, changing the value of a primary key of table makes the identical change to the foreign key value in related tables. Cascading deletions delete all related records with a foreign key that corresponds to the primary key of a record in a base table that you want to delete.

To learn more about enforcing referential integrity in Jet databases, see "Establishing Relationships Between Tables," p. 189 and "Cascading Updates and Deletions," p. 192.

Entity Integrity and Indexes

When you add new records to a base table, entity integrity assures that each primary key value is unique. Jet and SQL Server ensure entity integrity by adding a no-duplicates index to the field you specify for the primary key. If duplicate values exist when you attempt to designate a field as the primary key, you receive an error message. You receive a similar error message if you enter a duplicate primary key value in the table.

For more information on Jet indexes, see "Adding Indexes to Tables," p. 194.

Indexes also speed searches of tables and improve performance when executing SQL statements that return data from fields of base and related tables.

Data Validation Rules and Check Constraints

Data-entry errors are another major source of "unclean data." In the days of punched-card data entry, keypunch operators typed the data and verifiers, who usually worked during the succeeding shift, inserted the cards in a punched-card reader and repeated the keystrokes from the same source document. This process detected typographical errors, which the verifier corrected. Keypunch operators had no visual feedback during data entry, so typos were inevitable; video display terminals didn't arrive until the mainframe era.

Note

Keypunch operators kept their eyes on the source documents, which gave rise to the term heads-down data entry. The term continues in common use to describe any data-entry process in which the operator's entire working day is spent adding or editing database records as quickly as possible.

Rekeying data leads to low productivity, so most data-entry applications support data validation rules designed to detect attempts to enter illegal or unreasonable values in fields. An example of a validation rule is preventing entry of a shipping date that's earlier than the order date. The rule is expressed as an inequality: ShipDate >= OrderDate, which returns False if the rule is violated. Similarly, UnitPrice > 0 prevents accidentally giving away a line item of an order.

Jet tables and fields have a Validation Rule property that you set to the inequality expression. SQL Server calls validation rules check constraints. Both Jet and SQL Server have a Validation Text property for which you specify the text to appear in an error message box when the entry violates the rule or constraint. It's a more common practice when working with client/server databases to validate data in the front-end application before sending the entry to the back-end server. Detecting the error on the server and returning an error message requires a roundtrip from the client to the server. Server round trips generate quite a bit of network traffic and reduce data-entry efficiency. One of the objectives of client/server front-end design is to minimize server round-tripping.

To learn more about Jet's validation methods, see "Validating Data Entry," p. 227.

Transactions

A database transaction occurs when multiple records in one or more tables must be added, deleted, or modified to complete a data-entry operation. Adding an order or invoice that has multiple line items is an example of a transaction. If an order or invoice has five line items, but a network or database problem prevents adding one or more item records, the entire order or invoice is invalid. Maintaining referential integrity prevents adding line item records without a corresponding order or invoice record, but missing item records don't violate integrity rules.

Transaction processing (TP, also called online transaction processing, OLTP) solves the missing line item problem. Requiring TP for order entry, invoice processing, and similar multi-record operations enforces an all-or-nothing rule. If every individual update to the tables' records occur, the transaction succeeds (commits); if any update fails, changes made before the failure occurs are reversed (rolled back). Transaction processing isn't limited to RDBMSs. Early mainframe databases offered TP and transaction monitors. IBM's Customer Information and Control System (CICS, pronounced "kicks") was one of the first transaction processing and monitoring systems and remains in widespread use today.

Jet and SQL Server databases offer built-in TP features. Jet has a Use Transactions property that you set to Yes to require TP for updates. SQL Server traditionally requires writing T-SQL statements BEGIN TRANS, COMMIT TRANS, and ROLLBACK TRANS to manage transactions, but Access 2003's ADP forms have a new Batch Updates property that lets you enforce transactions without writing complex T-SQL statements. You also can specify automatic batch updating for DAP connected to SQL Server databases.

For a brief description of the new batch update feature, see "Changes to ADP Features," p. 37.