Performing the Application Audit

These steps generally refer to controls specific to the application and do not address general controls at the level of the network, operating system, and database management system. Please refer to other sections of this book for general controls at these levels and consider the frameworks and concepts enumerated earlier in this chapter as you approach developing the audit program for your application.

Part 1-Input Controls

Incorrectly implemented or checked input controls consistently remain top reasons why applications suffer vulnerabilities. Examples include buffer overflows and injection attacks.

1 Review and evaluate data input controls.

As much as possible, online transactions should perform upfront validation and editing in order to ensure the integrity of data before they are entered into the system's files and databases.

How

Verify that invalid data are rejected or edited on entry. The auditor will need to understand the business function being supported by the system and the purpose and use of its various data elements. This likely will require discussion not only with the developers but also with the end users. Once the purpose of the system and its data are understood, it will be much easier for the auditor to think through the various data-integrity risks associated with the application. In some cases, a code review may be appropriate if the developers are available and the auditor is a knowledgeable coder. Poorly written, commented, or formatted code is often a red flag that suggests that a deeper review is needed. Some basic examples of good data input controls include

• Fields that are intended to contain only numbers should not allow entry of alphanumeric characters.

• Fields that are used to report such things as dates and hours should be set up to either require input in the correct format (such as MMDDYY or HHMM) or to transform input into the correct format.

• Where applicable, transactions should perform "reasonableness" checks on inputs. An example would be preventing users from reporting labor of more than 24 hours in a day or more than 60 minutes in an hour. Another example would be disallowing entry for time, costs, etc. for an employee who has been terminated or who is on LOA.

• When there are a finite number of valid entries for a field, entries that are invalid should not be allowed. In other words, input screens should validate such things as cost centers, account numbers, product codes, employee numbers, etc. against the appropriate database(s) or file(s).

• Duplicate entries should not be allowed for data that are intended to be unique. For example, the transaction should not allow a product code to be added to the product database if that code already exists on the database.

• Where applicable, the transaction should perform "logic" checks. For example, if there were a transaction used by ticket agents to record how many seats were sold on a flight and how many no-shows there were, the transaction should not allow the agent to input numbers indicating that there were more no-shows than seats sold.

• Each input screen generally has certain fields that are required for the transaction to be processed accurately. Execution of a transaction should not be allowed until valid data are entered into each of those fields.

• Where applicable, transactions should perform "calculation" checks on inputs. For example, the system should ensure that journal-entry credits and debits balance to zero before processing a transaction. Another example would be a labor-entry system where hours charged for the week need to add up to at least 40.

• Programmed cutoff controls should be in place to help prevent users from recording transactions in the wrong period. In other words, the screen should not allow users to record transactions in prior accounting periods.

• Database operatives such as *, =, or select) should be disallowed as valid input.

2 Determine the need for error/exception reports related to data integrity and evaluate whether this need has been filled.

Error or exception reports allow any potential data-integrity problems to be reviewed and corrected when it's not feasible or practical to use input controls to perform upfront validation of data entered into the system. For example, while it may not be inherently wrong for an employee to enter 80 hours of overtime for one week into a labor system, this sort of unusual event should be captured on a report for review by the appropriate level of management.

How

Discuss the application's error and exception handling with the developer or administrator. Based on the results of the analysis from step 1, look for opportunities for additional data integrity checks (that may not have been feasible to perform with "hard" upfront input requirements). Again, discussions with the end users can be very helpful here. Ask them what sorts of reporting would be helpful for them in catching anomalies and errors. In some cases, a code review may be appropriate if the developers are available and the auditor is a knowledgeable coder. Poorly written, commented, or formatted code is often a red flag that suggests that a deeper review is needed.

Part 2-Interface Controls

3 Review and evaluate the controls in place over data feeds to and from interfacing systems.

When an application passes and/or receives data to or from other applications, controls need to be put in place that ensure that the data are transmitted completely and accurately.

How

Discuss data-feed controls with the application developer or administrator. Expect to see basic controls such as

• Control totals from interface transmissions should be generated and used by the system to determine whether the transmission completed accurately. If a problem is found, reports should be issued that notify the proper people of the problem. Some examples of control totals that may be applicable are hash totals, record counts, and total amounts (for numerical records). Another type of control total could flag missing record numbers when records are transmitted in a sequential fashion.

• The system should handle items that did not transmit successfully in such a manner that reports and/or processes enable these items to be resolved quickly and appropriately with audit trails as appropriate.

• Data files that contain interface source or target information should be secured from unauthorized modifications. This may mean appropriate authentication controls, authorization controls, or encryption where necessary.

• When it is not feasible to use control totals to verify accurate transmission of data, reconciliation reports should be generated that allow users to compare what was on one system with what was received on another system.

• Where applicable, data validation and editing, as described in the "Input Controls" section of this checklist, should be performed on data received from outside systems. Error/exception reports should be generated that allow any data-integrity problems to be corrected.

4 In cases where the same data are kept in multiple databases and/or systems, ensure that periodic sync processes are executed to detect any inconsistencies in the data.

How

Determine with the help of the application developer or application administrator where this sort of control is applicable, and review for its existence and effectiveness.

Part 3-Audit Trails

5 Review and evaluate the audit trails present in the system and the controls over those audit trails.

Audit trails are useful for troubleshooting and helping to track down possible breaches of your application.

How

Review the application with the developer or administrator to ensure rinformation is captured when key data elements are changed. This information should include in most cases the original and new values of the data, who made the change, and when the change was made. This information should be kept in a secured log in order to prevent unauthorized updates. The logs should be retained for a reasonable period of time, such as three or six months, in order to aid investigations into errors or inappropriate activities.

6 Ensure the system provides a means to trace a transaction or piece of data from the beginning to the end of the process enabled by the system.

This is important to verify that the transaction was fully processed and to pinpoint any errors or irregularities in the processing of that data.

How

Review the application with the developer or administrator and evaluate the existence of this ability.

Part 4- Access Controls

7 Ensure the application provides a mechanism which authenticates users, based, at a minimum, on a unique identifier for each user and a confidential password.

Failure to authenticate users or just having a poor authentication scheme presents an open opportunity for curious users and malicious attackers.

How

Review the application with the developer or administrator and verify appropriate authentication measures exist commensurate with the type of data on the application. For example, two-factor authentication might be required in some cases to authenticate users in sensitive environments or for end-users accessing your network from their homers.

8 Review and evaluate the application's authorization mechanism to ensure users are not allowed to access any sensitive transactions or data without first being authorized by the system's security mechanism.

The system's security mechanism should allow for each system user to be given a specific level of access to the application's data and transactions.

How

Employees should only be given the amount of access to the system which is necessary for performing their jobs. Review the application with the developer or administrator, and verify this functionality in the application. In other words, it should be possible to specify which specific transactions and datasets or files a system user will access. In general, it also should be possible to specify what level of access (e.g., display, update, and delete) the user will receive to application resources.

9 Ensure that the system's security/authorization mechanism has an administrator function with appropriate controls and functionality.

The administrator user function should exist to help administer users, data, and processes. This account or functionality should be tightly controlled in the application to prevent compromise and disruption of services to other users.

How

Evaluate the use of the administrator function with the developer or application administrator. The user of this function should have the ability to add, delete, or modify user access to the application system and its resources. The security mechanism should also provide the ability to tightly control who has access to this administrator function. Also ensure that the system's security mechanism provides the system's security administrator with the ability to view who has access to the system and what level of access they have.

10 Determine whether the security mechanism enables any applicable approval processes.

The application's security mechanism should support granular controls over who can perform what approval processes, and then lock data that has been formally approved from modification by a lower authority. Otherwise, a lower authority or malicious user could modify or corrupt data in the system.

How

Verify with the developer or application administrator that appropriate controls are in place. For example, if there is a need for sign-off of journal entries before they can be passed on to the general ledger, the system's security mechanism should provide a means for defining who is authorized to perform this sign-off. Any data that have been through this sort of approval process should be locked from any further modifications.

Interviews with system users are a good mechanism for helping the auditor to determine the need for this sort of ability. It is critical for the auditor to understand not only the technical aspects of the application being reviewed but also the business purpose.

11 Ensure that a mechanism or process has been put in place that suspends user access on termination from the company or on a change of jobs within the company.

Poor deprovisioning processes may leave a user with inappropriate access to your application long after the access or authority should have been removed.

How

Verify appropriate deprovisioning processes are in place with the developer and application administrator. Be sure to look at both the application and the procedures around the application to ensure they are being followed and are capable of being followed as written.

For applications that have been in "production" for some time, select a sample of system users, and validate that their access is still appropriate. Alternatively, if possible, select a sample of system users who have changed jobs or left the company, and ensure that their access has been removed.

12 Verify that the application has appropriate password controls.

The appropriateness of the password controls depends on the sensitivity of the data used within the application. Overly weak passwords make the application sensitive to compromise, and overly strong passwords often force users to write them down in plain sight or to never want to change their password.

How

Verify appropriate password controls with the help of the developer or the application administrator. For example, three-digit PIN numbers probably are inappropriate for applications that store credit-card data, and a 20-character password probably is overly paranoid for someone trying to access his or her voicemail. Ensure that the security mechanism requires users to change their passwords periodically (e.g., every 30 to 90 days). When appropriate, the security mechanism also should enforce password composition standards such as the length and required characters. Additionally, the security mechanism should suspend user accounts after a certain number of consecutive unsuccessful log-in attempts. This is typically as low as 3 and can be as high as 25 depending on the application, other forms of authentication required, and the sensitivity of the data.

13 Review and evaluate processes for granting access to users. Ensure that access is granted only when there is a legitimate business need.

Users should have intentional access granted and governed by the application to prevent unauthorized access to areas outside the intended scope for the user. The application should have controls in place to prevent users from having more access than is required for their role. This step embodies the concept of least-privileged access.

How

Review the application with the developer or administrator. Possibly select a sample of users, and ensure that user access was approved appropriately. Verify that the authorization mechanism is working appropriately.

14 Ensure that users are automatically logged off from the application after a certain period of inactivity.

A person could have access to the application if they walk up to a logged-in workstation where the previous user didn't log off and the application is still active.

How

Review the application with the developer or administrator to evaluate the existence of this ability.

15 Evaluate the use of encryption techniques to protect application data.

The need for encryption is determined most often by either policy, regulation, the sensitivity of the network, or the sensitivity of the data in the application. Where possible, encryption techniques should be used for passwords and other confidential data that are sent across the network. This prevents other people on the network from "sniffing" and capturing this information.

How

Review the application with the developer or administrator to evaluate the existence of encryption where appropriate.

16 Evaluate application developer access to alter production data.

In general, system developers should not be given access to alter production data.

How

Discuss with the developer or administrator, and evaluate the separation of duties between developers and administrators.

Part 5- Software Change Controls

Software change management (SCM), used by a trained software development team, generally improves the quality of code written, reduces problems, and makes maintenance easier.

17 Ensure that the application software cannot be changed without going through a standard checkout/ staging/testing/approval process after it is placed into production.

It should not be possible for developers to update production code directly. Should a failure in the application occur without enforced software change controls, then it might be difficult to impossible to track down the cause of the problem. Additionally, developers should not have access to data in production applications. This is particularly true if the application data are sensitive.

How

Evaluate this capability with the developers and application administrator.

18 Evaluate controls around code checkout, modification, and versioning.

Strong software controls around code checkout, modification, and versioning provide accountability, protect the integrity of the code, and have been shown to improve maintenance and reliability.

How

Verify with the developers that the software-change mechanism requires developers to check out code that they wish to modify. If another developer wishes to modify the same code while it is still checked out, he or she should be prevented from doing so. Alternatively, the second developer could be warned of the conflict but allowed to perform the checkout. In such a case, a notification of the duplicate checkout should be sent automatically to the original developer.

Additionally, ensure that the software-change mechanism requires sign-off before code will be moved into production. The system should require that this sign-off be performed by someone other than the person who developed or modified the code. In addition, the software-change mechanism should allow for specific people to be authorized to perform sign-off on the system's programs. The people with this authorization should be kept to a minimum.

Evaluate controls in place to prevent code from being modified after it has been signed off on but before it has been moved to production. Ensure that the software-change mechanism ‘versions’ software so that past versions of the code can be retrieved, if necessary.

19 Evaluate controls around the testing of application code before it is placed into a production environment.

Improperly tested code may have serious performance or vulnerability issues when placed into production with live data.

How

Determine whether the software-change process requires evidence of testing, code walkthroughs, and adherence to software-development guidelines. These should occur before the approver signs off on the code. Testing of any software development or modifications should take place in a test environment using test data. Determine whether this is the case.

Part 6- Backup and Recovery

20 Ensure that appropriate backup controls are in place.

Failure to back up critical application data may severely disrupt business operations in the event of a disaster.

How

Determine whether critical data and software are backed up periodically (generally weekly full backups with daily incremental backups for the data) and stored off-site in a secured location. If cost beneficial and appropriate, duplicate transaction records should be created and stored in order to allow recovery of data files to the point of the last processed transaction. Also ensure that the application code is backed up and stored offsite in a secured location, along with any tools necessary for compiling and using the code.

21 Ensure that appropriate recovery controls are in place.

Recovery procedures and testing are necessary to ensure that the recovery process is understood and functions operationally as intended.

How

Discuss with the application administrator and appropriate personnel to ensure that detailed recovery procedures are documented that define what tasks are to be performed, who is to perform those tasks, and the sequence in which they are to be performed. Testing of the recovery from backup tapes using the documented recovery procedures should be performed periodically.

Part 7- Data Retention and Classification

22 Evaluate controls around the application's data retention.

Data should be archived and retained in accordance with tax and legal requirements.

How

Evaluate the appropriateness of the controls with the developers and application administrator. These requirements will vary based on the type of data and should be acquired from the appropriate departments within your company.

23 Evaluate the controls around data classification within the application.

All application data should be assigned a business owner, and this owner should classify the data (e.g., public, internal use only, or confidential).

How

This classification should appear on any reports or transactions that display system data. Determine whether this has been done.

Part 8-Operating System, Database, and Other Infrastructure Controls

Detailed guidelines for controlling the operating system, database, and other related infrastructure components are beyond the scope of this chapter. However, security of the infrastructure on which the application resides is a critical part of application security. The applicable audit programs from this book's other chapters should be executed in addition to the application-specific steps provided earlier in this chapter.