Unforeseen problems arise after the implementation of almost any new system. Without a robust method for capturing and resolving those issues, there likely will be instances where issues will "slip through the cracks" and not be resolved in a timely fashion. Also, an issue-tracking system is needed to ensure that issues are being prioritized and worked according to their importance.
Review the issues database, issues spreadsheet, or whatever other method has been established for recording and tracking postimplementation issues. Ensure that the issue-tracking tool records adequate information regarding each issue, including description of the issue, priority level, due date, latest status, and resolution information. Ensure that controls exist over the tool used to track issues, such as backups and security to prevent unauthorized updates. Review processes for escalating issues and for ensuring issues are tracked to resolution. Review the issues list for evidence that issues are being closed. Interview customers to ensure that the process is working.
If the project being reviewed involves replacing an existing system, there will need to be a point at which users switch over to the new system. It is critical that existing data be converted successfully to the new system prior to this cut-over to ensure a smooth transition.
Review the conversion plan, and look for elements such as the following:
Ensure that all critical data are identified and considered for conversion.
Review controls for ensuring that all data are converted completely and accurately. Examples of these control mechanisms could be control totals on key fields, record counts, and user reconciliation procedures.
Determine whether all conversion programs are fully tested with user involvement and that the test results are documented.
If historical data are not converted, ensure that a method is developed for accessing these data if needed. For example, if financial data are involved, historical financial data may be needed in the future for IRS reporting.
Review and evaluate plans for parallel processing or other fallback methods in case difficulties are experienced during cut-over to the new system.
Ensure that the conversion process includes establishing data that were not used in the legacy systems. For example, a record in the new system may contain fields that were not contained in a similar record on the legacy systems. Consideration should be given to populating these new fields.
Review and evaluate the plan for "conversion weekend." There should be a detailed plan containing criteria and checkpoints for making "go/no-go" decisions.
This list should not be used as a mechanical checklist. The absence of one of these items should not automatically result in an audit issue. Instead, look at the conversion process as a whole, and determine whether enough of the key elements are present to provide a reasonable comfort level that adequate and controlled conversion is taking place.
Once the project has been completed, it is likely that project personnel will be redeployed to other projects. It is therefore critical that run/maintain support personnel be trained properly in the functionality of the system so that they will be prepared to support it when users identify issues or request enhancements.
Through interviews or review of documentation, look for evidence that support personnel have been identified, adequately involved in the project, and appropriately trained on the system and its functionality.
Incomplete or out-of-date technical and user documentation could increase costs and cycle time to maintain the software, increase support and training costs, and limit the system, process, or software's usefulness to the customer.
Obtain copies of existing documentation, and evaluate its adequacy. Look for evidence that would indicate that documentation has been updated when the system has changed, and review processes for ensuring ongoing maintenance of the documentation. Ensure that files containing documentation are locked down and can be modified only by appropriate personnel (using techniques described in Chapters 6 and 7). Interview appropriate personnel to understand processes for changing critical documents. Ensure that an approval process is required before changes are made to significant documents and that the approval process cannot be circumvented.