Evidence Collection: Evidence Is not just Evidence

 < Day Day Up > 



There are three techniques used by auditors in collecting evidence that allows them to understand an organization and its application systems:

  • Make a judgment about the levels of inherent risk associated with an organization's management and its application systems.

  • Obtain an understanding of an organization's controls sufficient to make a judgment about the types and levels of controls in the applications system.

  • Design and perform tests of the existence and reliability of controls on which the organization can depend.

Auditors commonly use the evidence collecting techniques of interviews, questionnaires, and flowcharts to complete their audits.

Interviews

Auditors use interviews to obtain qualitative and quantitative information during their evidence collection efforts. Their objectives are to elicit candid, complete and honest answers from the interviewees. At this point, it is important to differentiate between interviews and interrogations. The reason behind an interrogation is to elicit information about some wrongdoing. Inherently, it is an intrusive method of obtaining information using accusatory language and demeanor. Interviewing is a technique eliciting information from someone who has more information than the interviewer who is requesting a response from a fellow professional. It is a kinder, gentler approach to eliciting information than an interrogation.

Auditors must conduct effective interviews, but first, they must understand the interviewee's motivation for answering the auditor's questions. Usually, the respondent's motivation to reply to questions asked during the interview is a function of how they perceive the interview to be a means of reaching their goals or something the respondent wants. For example, if a respondent sees the audit as a process in assisting them in attaining their performance goals, they will likely answer questions frankly and directly. However, if the interviewee views the auditor's interview as a process hindering their work, it is possible their answers will be evasive, incomplete, and even antagonistic. Wise auditors ask themselves, "What's in the interview for the respondent?"

Interview Preparation

Auditors may control the amount of interview stress by limiting the number of difficult questions asked. In this fashion, more stressful interviews should be shorter. Experienced auditors take sufficient steps to alleviate any respondent fears before the interview begins. Interviewers should be aware of the interviewee's desire to pursue topics that interest them if they perceive the auditor to be a responsible person. The auditor's task is one of establishing a professional rapport as quickly and effectively as possible. This is another one of those good judgment areas for auditors. Adept auditors clearly communicate the purpose and intent of the interview at the outset to show empathy, professional and responsible demeanor, and promote mutual trust and respect.

Experience Note 

Taking a moment to talk to the interviewee about something other than work is a technique that works well.

Doing Your Homework

Before beginning an interview, auditors should be mindful that the information they require is not available from anywhere else. Frankly, if interviewees perceive the interview is a waste of their time, they may become disinterested and less than forthright. Doing their homework involves auditors identifying those employees who can provide them with the best information on a particular topic. Organizational charts are usually the first source.

Another good source of information is the organization's line-of-authority documentation and brief job descriptions. Through senior managers, auditors may obtain an idea as to the division of business units and corresponding employee responsibilities. Additionally, senior managers may wish to make introductions between their employees and the auditors. Senior managers can be very helpful in locating facilities for performing interviews where the atmosphere is not disruptive and scheduling mutually convenient times.

Interview content must be thoroughly prepared before beginning the interview. Nothing will leave a respondent colder than an auditor who has no idea about what they want to do during the interview. Auditors should make a list of goals they wish to achieve during the interview. Some auditors go so far as having a script of questions they want to ask divided by specific topic area.

Auditors may use open or closed questions in their interviews. Closed questions merely require a yes or no answer. Open questions usually begin with the words: how, why, when, who, or what. Open questions may be asked at the beginning of topic areas followed by closed questions where more clarification is needed. For example, "What are the types of controls you have over the entry of data from credit card applications?" This question might be followed by "Do you have manual or automated data input quality inspections?"

Interview Steps

At the outset of the interview, the auditor should state the purpose of the interview and confirm with the respondent that the interview corresponds to the arrangements made earlier. No one likes to be surprised in showing up for an interview just to find it is not the interview they were expecting. Here are a few steps to facilitate productive interviews:

  • Take a moment to state the purpose of the interview. Auditors may wish deliver the goals of the interview at the beginning.

  • Take another moment to briefly establish a personal rapport.

  • Do not digress during the interview, stay on track.

  • Mirror the voice, tonality, volume of speech, eye contact and rate of the respondent

  • Ask if the auditor can take notes. Ask if they want copies of the auditor's interview notes.

  • Be a good listener; wait for the respondent to reply.

  • Allow the respondent time to think.

  • Answer any questions from the interviewee.

  • Keep it professional, avoid familiarity.

  • Review responses for accuracy.

  • Typical questions should cover at least the following basic topic areas:

    • What is the nature of your business? Briefly describe your business unit's operations.

    • How many employees are there? What are their duties? Where are their workstations and their respective locations?

    • What was the date of the last hardware/software inventory? Where is the location of the inventory documentation?

    • What was the date of the last risk analysis? Is there a risk management program? Where is the documentation? When was the last test of your disaster recovery/business resumption plan?

    • Provide documentation for these business processes:

      • Policies, procedures, standards

      • Human resources, data and facilities safeguards

      • Last audit documentation

      • Legal and regulatory issues

      • Human resources issues

      • Employee training and continuing education

      • Security awareness program

      • Business unit or organizational issues

Interviewing for Evidence of Controls

Interviews can be very useful in obtaining evidence of the existence of controls and the procedures. By way of widely accepted measures, there are already formulated audit programs to address internal controls and their implementation. Examples of these programs are ISO 17799, available at www.iso.ch and COBIT (Control Objectives for Information and related Technology) and ISACA, available at www.isaca.org. These programs attempt to provide generally accepted internal control guidance for auditors and are worth reviewing before beginning an IT audit engagement.

The object of the auditor's employee interview is to determine the present condition of the system and compare this condition with the audit program criteria. In the pursuit of interviews, the auditor should ask employees for evidence of the controls in the form of questions:

  • May I see it?

  • Please show me how you _______.

  • May I observe you working?

  • How do you perform the process of _______?

  • Is it possible for you to delete transaction logs?

  • Please show me if you can do this _______ operation.

An experienced auditor will interview several employees having similar jobs in order to compare results and decide whether policies and procedures are being practiced or not.

Interview Analysis

As soon as possible after the interview, the auditor should prepare a written report of the interview from the notes taken. Be certain to separate fact from inference and speculation.

Facts are those things that the interviewee has heard, seen, or in which she has materially participated. They know the facts because they were there. Inference is a logical extension of the interviewee's mind - for example, if a cat and a mouse were placed in a box and the top closed and placed where it is under constant observation. A few minutes later the box is opened; the mouse is discovered to be gone. It is inferred that the cat ate the mouse even though no one actually observed it. Speculation is merely that the interviewee is guessing about something. For example, if there is a sudden increase in system processing time and the interviewee indicates the reason is attributable to increased input error rates, but cannot offer any observation or other substantive proof, then the response is speculative.

Do not discount the value of speculation. Wise auditors give speculation due consideration depending on the credibility, experience, and training of the interviewee. Many experienced auditors include speculation at the end of their interview report accompanied by proper qualifications.

Interview notes and written reports should be retained as permanent parts of the auditors' work papers. Senior managers should not be surprised when knowledgeable attorneys or investigators request audit interview notes and reports as legal processes. They are looking for evidence that the audit work papers can reconstruct audit events.

Experience Note 

In some cases, the destruction or failure to retain auditor work papers is a violation of law and regulation, depending on the industry. In other cases, destroying work papers after being notified of a pending investigation or legal action is running the risk of contempt of court.

Questionnaires

Auditors may use written questionnaires as effective means to collect evidence. Responses obtained to questions asked on questionnaires indicate the presence or absence of controls or the incorrect application of controls. They can elicit users' comments about the effectiveness and efficiency of a system or subsystems. There are basically three major aspects of their design:

  1. Design of questions

  2. Design of responses

  3. Design of layout and structure

The primary focus on questionnaire design is the crafting of questions to ensure the respondents understand the facts required. It is not unreasonable that some questions are redundant ensuring the respondents understand which facts are being requested. Questions need to be self-explanatory. If the question asks about input field limits, then it is expected the anticipated respondent already knows what input field limits are and when they apply. Here are a few questionnaire best practices:

  • Make certain the questions are specific. Rather than ask if input fields are controlled, ask which applications examine input field correctness.

  • Use simple, plain language.

  • Avoid technical jargon.

  • Avoid abbreviations; use specific language instead.

  • Avoid ambiguous language.

  • Avoid leading questions. Leading questions suggest answers that respondents should reply. For example, "Do employees use the human resources system interface?" A much better question is phrased as, "How is human resources information obtained online, and by whom?"

  • Avoid hypothetical questions. Stick to the facts. Do not ask questions based on assumptions. For example, "How often would you use the human resources system in a month's time?" This presupposes that the respondent knows about the human resources system and uses it monthly.

  • Avoid questions that require extensive and accurate recall. "How many times did you use the human resources system during the past two months?" This is a question that cannot be easily answered. Instead, ask the respondent if he/she keeps a record of their use of a particular system.

  • If the questionnaire has a scale of responses, make certain they are applicable to the topic. If the questionnaire asks the location of fire extinguishers in the warehouse, then ask if fire extinguishers are present, how many are present, and their locations. Make certain the questionnaire is directed to the right employees. Asking the finance unit questions about the location of fire extinguishers in the warehouse is not going to produce relevant answers.

The layout and structure of a questionnaire affect its accuracy. If the questionnaire is mailed, its layout and structure will likely affect the response rate. Its objective is to be well-received with clear, simple, logical, and appealing construction. The length of a questionnaire also affects the success of completion. If questions and questionnaires are too long, respondents lose interest and provide answers that may not accurately reflect their observations. Care must be taken to craft the flow of questions through the questionnaire. At the beginning, general questions should be asked placing little stress on the respondent with more difficult questions placed toward the middle or end of the topic areas.

Flowcharts

Control flowcharts illustrate that controls exist in a system and where these they are located in the system. There are basically three purposes of flowcharts for auditing purposes:

  1. Comprehensive. The construction of this type of flowchart highlights areas where auditors do not have a thorough understanding of either the system or the controls located in the system.

  2. Evaluation. Auditors use control flowcharts to recognize patterns that show control strengths or weaknesses.

  3. Communication. Auditors may use control flowcharts to communicate their understanding of the target system and its related controls to other parties.

Experience Note 

Auditors ask respondents for their flowcharts before creating their own. It is likely most business units already have control flowcharts.

Types of Flowcharts

There are many different types of flowcharts that can be crafted. There are flowcharts for analysts, designers, engineers, managers, or programmers detailing individual understanding.

Document flowcharts have the purpose of showing existing controls over document-flow through the components of a system. These flowcharts are typified by their vertical structure. The chart is read from left to right and documents the flow of documents through the various business units. An example of document flowchart is shown in Exhibit 3.

Exhibit 3: Document Flowchart

start example

click to expand

end example

The second popular type of flowchart is the data flowchart. This diagram has the purpose of showing the controls governing data flows in the system. Data flowcharts are used primarily to show the channels that data is transmitted through the system rather than how controls flow. It is important to note that data flowcharts are not particularly useful in gaining an understanding of controls placed in the physical or resource level of a system. In other words, data flowcharts do not illustrate controls in prevention of detection of errors (Exhibit 4).

Exhibit 4: Data Flowchart

start example

click to expand

end example

System flowcharts are the third type of illustration showing the controls located at the physical or resource level. System flowcharts show the flow of data to and through the major components of a system such as, data entry, programs, storage media, processors, and communication networks. These types of flowcharts demonstrate how the controls are placed to ensure the correct functioning of the named components (Exhibit 5).

Exhibit 5: System Flowchart

start example

click to expand

end example

The fourth type of flowchart, the program flowchart, shows the controls placed internally to a program within the system. For example, illustrating the process modules within a program aids the auditor in gaining an understanding of the means by which data integrity is preserved during processing (Exhibit 6).

Exhibit 6: Program Flowchart

start example

click to expand

end example

Experience Note 

Unless there is some very pressing reason to create flowcharts, they are very time consuming and frequently are not understood or appreciated by the intended audience. Think twice about spending the time before doing them.

Taking Care of the Stakeholders

Although this practice is not really part of the collection of evidence, it is relevant. In any audit process, no one likes surprises. Experienced auditors will initiate and foster dialogues with stakeholders during the audit. It is unwise for auditors to play "Gotcha" with audit results, besides most folks lose their sense of humor resulting from this behavior.

Experience Note 

A "stakeholder" is someone who has a professional interest in the audit and its outcome.

Here are a few best practices in the care and feeding of stakeholders:

  • Keep audit stakeholders briefed during the audit process. Keep them briefed of any serious negative trends or indications of fraud or abuse. Ensure that verbal briefings reflect exactly the same terminology that is going to be found in the audit report. Discrepancies of this nature cast serious doubts on auditor credibility.

  • Keep the audit manager briefed regularly throughout the audit process.

  • Reports should not be overly brief nor should they be overly verbose. They should be concise and supported by brief and relevant narratives.

  • Auditors should feel they have the independence to stray from the audit program, but a logical explanation is necessary if they spend significant time outside the formal program.



 < Day Day Up > 



Critical Incident Management
Critical Incident Management
ISBN: 084930010X
EAN: 2147483647
Year: 2004
Pages: 144

flylib.com © 2008-2017.
If you may any questions please contact us: flylib@qtcs.net