Background

Electronic Administration

Web-Based Surveys

Web-based surveys involve computer-to-computer communication via the Internet (Dillman, 2000), where information requests are made of individual users or groups of users. Bradley (1999) suggests a three-category classification for Web-based questionnaires:

• "Open" — The survey is available to any user since access is non-restricted.

• "Closed" — The survey is only available to invited respondents with knowledge of the instrument's URL.

• "Hidden" — A 'pop-up' survey appears when triggered by a user's selection of linked information.

An example of an open survey is the Microsoft Network (MSN) polls available to any and all visitors choosing to respond. A closed survey example was implemented in a recent research study. The closed Web-based survey design was used to collect data from five independent organizations on user trust in information systems technology (Lippert, 2001). A unique URL was provided to each organization, thereby, segmenting the sample populations by organization.. Closed Web-based questionnaires enable a researcher to control administration and access to the survey instrument. Dedicated URLs can control unwanted respondent access through selected invitation. Malhotra (1999) suggests 'pop-up' surveys offer a better response rate than a simple banner invitation. Hidden or 'pop up' surveys appear on portals such as Yahoo or Lycos and are aimed at obtaining a specific end user's immediate attention and response. Hidden surveys are triggered by keyword searches or banner links initiated by the user and generate a stratified sample based on the HTML connection.

Email Attached Surveys

Email attached surveys involve an electronic survey request made to a group of potential respondents within a given population. Electronic mail questionnaires appeared concurrently with the advent of the Internet and email for communication purposes (Bachmann, Elfrink & Vazzana, 1996). Bradley (1999) suggests a three-category classification for email questionnaires:

• Simple email — An email message containing a list of questions, which appear within the email text body.

• Email attachment — An email message containing a cover email message with the questionnaire provided as a file attachment.

• Email URL embedded questionnaires — An embedded URL within an email message requesting recipient participation through a hyperlink.

The email survey approach can be disseminated via the Internet or on a closed network through an Intranet or Extranet. Email and Web-based surveys both involve computer-to-computer communication. Email surveys, while easier to compose than Web-based surveys, are more limited in their visual stimulation and interaction capabilities (Dillman, 2000).

Manual Administration

Paper Based Surveys

Non-electronic surveys are divided into two broad classifications: questionnaires and interviews (Singleton, Straits & Straits, 1993). Paper and pencil questionnaires seek information from the respondent on a hardcopy document using a marking implement. Interviews are completed via telephone or face-to-face by an individual or group of researchers. Paper based surveys are limited to text questions and lack the dynamic capabilities of response bridges found in Web-based administration. A response bridge facilitates the sequencing from an initial question to the appropriate follow-on segment. For example, user trust of the technology may function as the initial question. Based on the individual's response — low trust or high trust — the response bridge will automatically redirect to an appropriate follow-on question.

Like Internet surveys, both open and closed ended questions can be employed in paper-based administrations. Researchers primarily used mailed questionnaires to obtain respondent information until the late 1970s (Dillman, 1978). Blattberg and Glazer (1993) suggest that through the development of advanced electronic technology, two-way data exchanges dramatically increased efficiency.

Schmidt (1997b) suggests that Web-based surveys offer the ability to implement interactive or dynamic instruments — a unique characteristic of this data collection medium. The variety of question formats and presentation styles enables flexibility in questionnaire design matched to specific research goals (Schmidt, 1997b).

Definitions of Instrument Performance Parameters

Instrument performance parameters discussed in this chapter are defined in Table 1. These parameters form the basis for comparing administration media by output performance measures. Current trends in administration usage are directed toward electronic surveys. Researchers should consider all output parameters prior to selection of a data collection medium.

Table 1: Survey Instrument Performance Parameters

Parameter	Definition
Response Rate	The ratio of completed and usable instruments divided by the total number of distributed instruments.
Participation Ease	The subjective perception that the survey instrument is user friendly to the respondent.
Attractiveness of Survey	A subjectively pleasant and visually stimulating presentation.
Novelty Effect	The appeal of a new or unusual feature, which impacts the respondent's willingness to participate.
Administrative Costs	The costs associated with instrument development, survey dissemination, questionnaire follow-up, data entry and organization of results.
Response Flexibility	The degree of individual control available in the design and completion of a survey instrument.
Response Time	The time from the initial dissemination to the arrival of completed instruments, including the time necessary for actual administration.
Population Size	The boundary or totality representing the limit of all potential respondents.
Sample Bias	A type of sampling error that decreases generalizability of results.
Instrument Validity	The quality of a survey instrument to measure the intended phenomena.
Management of Non-Response Data	The data provided by a survey respondent in which no response, a spurious response or a clearly false one is offered.
Response Error	The class of all potential responses that are either incorrect, inaccurate, intentionally deceiving, misguided, or methodologically misclassified.

Parametric Comparison of Web-Based Versus Paper-Based Administrations

Response Rate

Response rate is generally defined as the ratio of completed and usable instruments divided by the total number of distributed instruments. Scholars exploring response rate variance differ in findings regarding whether electronic-based or paper-based administrations yield higher response rates. Often response rate variance is attributable to sampling frame limitations rather than actual differences in survey media (Couper, 2000). Sampling frame limitations are the inequity to access the entire sampling frame as a function of technology accessibility issues. A 1996 survey by Bachmann, Elfrink and Vazzana of business school deans and chairpersons reported no significant difference between responses to email and regular mail instruments. Miller, Daly, Wood, Brooks and Roper (1996) found response differences between the email and paper surveys of professional computer scientists, which they attributed to sampling issues, rather than actual differences in survey media. Shaw and Davis (1996) reported significant differences in responses between their electronic and paper groups — attributable to demographic differences rather than differences in survey media.

Response rates for most American national surveys, regardless of type, have fallen over the last 40 years (Cook et al., 2000). Response rates for email surveys have significantly decreased between 1986 and 2000 (Sheehan, 2001). The decline in response rates may also be a function of the general over saturation of surveys within society (Marits, 1998). Response rates for sampled populations may vary, based on a combination of factors (Turley, 1999). Turley suggests that response rates may differ depending upon topic relevance to the respondent and/or the respondent's socio-demographic characteristics. Men tend to respond more to Web-based formats than women (Tomsic, Hendel & Matross, 2000). In general, response rates tend to increase over time for Web-based administrations (Tomsic et al., 2000).

Response rates appear to be lower for Web-based surveys than for equivalent mail surveys (Allen & Fry, 1986; Cook et al., 2000; Crawford, Couper & Lamias, 2001; Handwerk, Carson & Blackwell, 2000; Manfreda, Vehovar & Batagelj, 2001). Web-based surveys conducted from listservs produce lower response rates than mail administrations (Couper, 2001). In a general study, postal mail produced three times the response rate of email (Schuldt & Totten, 1994). Although different in experimental treatments, subjects, and type of electronic medium employed, a summary of nine studies (see Figure 2), shows agreement across eight of the nine studies that paper-based response rates exceed electronic response rates. In a recent study to develop standard techniques for email surveys, Sheehan and McMillan (1999) discovered a slight improvement in email survey response rates.

Figure 2: Response Rate Comparisons Between Electronic and Paper and Pencil Surveys

Response rates for paper and pencil instruments increase through proctored administration because respondents are monitored throughout the survey completion process (Krysan, Schuman, Scott & Beatty, 1994). Response rates for Web-based surveys can be increased through instrumentation design that accommodates ease of use and attractiveness.

Dillman (2000) suggests that many of the techniques used to increase paper-based survey response rates will not produce the same results with Web-based surveys. He further suggests using multiple contacts for email surveys as a mechanism for increasing response rates. Similar strategies to improve response rates can be employed with paper and pencil administration. Personal contact through pre-notification, reminder emails or notices, and/or voicemail messages may contribute to higher rates (Fox, Crask & Kim, 1988). Higher rates are clearly found to exist for Web-based administrations when incentives are utilized (Tomsic et al., 2000) and/or when samples are selected from targeted populations on interest sites (Cheyne & Ritter, 2001).

Turley (1999) suggests there are three influences to mail survey response rates: sponsorship, subject matter, and population. Sponsorship refers to the perceived influence of the individual or institution requesting respondent participation. Subject matter refers to the content specific theme of the data collection. Population refers to the unit of analysis under investigation. Table 2 provides response rate advantages and disadvantages between Web-based and paper and pencil administrations.

Table 2: Response Rate - Advantages and Disadvantages Comparing Web-based and Paper and Pencil Surveys

Response Rate - Advantages and Disadvantages
	Web-Based	Paper and Pencil
Advantages	Response rates have been found to increase as an inverse proportion to the volume of junk email (Weible & Wallace, 1998) Increased data reliability for proportional response rates without increasing costs (Quality Progress, 1999) Progress indicators (triggers to offer a respondent's survey completion position) reduce respondent abandonment (the respondent's leaving the survey prior to completion) (Couper et al., 2001)	Response rates tend to improve through the preservation and control of the confidentiality of respondents' identity, both real and perceived (Kuhnert & McCauley, 1996) Responses tend to be higher overall when compared to electronic administrations (Crawford et al., 2001; Cook et al., 2000; Allen & Fry, 1986; Manfreda et al., 2001; Handwerk et al., 2000)
Disadvantages	Respondent lack of familiarity with technology may reduce response rates for Web-based administration (Dillman & Bowker, 1996)	Long self-administered questionnaires produce lower response rates (Heberlein & Baumgartner, 1978; Dillman, Sinclair & Clark, 1993)

Cook et al. (2000) report recent research indicates lower response rates may be more accurate than surveys yielding higher response rates. This suggests an inconsistency with a survey research standard that data quality (both validity and reliability) is improved through higher response rates.

Response rate does not function as an independent, isolated, or linear variable; rather, it is affected by many direct and indirect intervening variables. A simple association between response rate and administration media is probably naive. Given the existing data, response rate, as a single performance variable, may not be an adequate comparison variable between paper and pencil and Web-based surveys (Underwood, Kim & Matier, 2000).

Participation Ease

Participation ease is the subjective perception that the survey instrument is user friendly to the respondent. User friendliness may include ease of completing the questions under inquiry. A review of existing research on ease of participation generally favors Web-based over manual survey administration. Web-based instruments enable keyboard based response devices or point and click mouse actions. The attractiveness of quick single keystroke entries, the ease of working with radio buttons, check boxes, or drop down lists are key features facilitating participation ease. Audio components or instructions permit a dual sensing source of survey questions. The potential multi-sense stimuli associated with Web-based administration supports simultaneous kinetic, auditory and visual understanding. Specialized feedback offered to Web-based respondents (Schmidt, 1997a) may assist participants in survey completion. The addition of help menus or tool tips can facilitate respondent inquiries in the absence of a proctor or principal researcher. While many respondents are computer literate, a segment of the population lacks basic keyboard entry skills or suffers from computer anxiety (Igbaria & Parasuruman, 1989). Participation is inhibited because the use of technology becomes a greater obstacle to the respondent than the perceived value of survey completion.

A disadvantage associated with some Web-based survey administration is the difficulty of returning to a previously unanswered question (Dillman & Bunker, 1996), or the need to complete the survey in a single sitting (Lippert, 2001; Tomsic et al., 2000). Serial-order constraints may impede participation ease and affect response rates. Time issues may encumber Web-based participation. For example, system boot-up times become a factor for individuals with older machines. Dial-up access speeds and limited bandwidth may inhibit participation ease. In one study, university students preferred paper and pencil formats because they had a greater assurance of the confidentiality of their responses (Tomsic et al., 2000). Table 3 provides participation ease differences between Web-based and paper and pencil survey administrations.

Table 3: Participation Ease - Advantages and Disadvantages Comparing Web-Based and Paper and Pencil Surveys

Participation Ease - Advantages and Disadvantages
	Web-Based	Paper and Pencil
Advantages	Web-based administration has been found to be more enjoyable, thus resulting in an increased perception of participation ease (Dillman et al., 1998) Radio buttons and check boxes provide greater control over how respondents answer questions (Matz, 1999) Interactive surveys may offer the respondent specialized feedback throughout the administration process (Schmidt, 1997a) Help menus and tool tips are readily available	A standard and familiar format requiring minimal instructions for survey completion A high set of potential response categories based in presentation media (Rockwood, Sangster & Dillman, 1997) Respondents' perception of a higher level of confidentiality in responses (Tomsic et al., 2000)
Disadvantages	Diminished degree of user trust in IS technology (Lippert, 2001) Lack of Internet familiarity or computer experience may result in user frustration associated with Web-based surveys (Dillman & Bowker, 1996) Web-based surveys often require a forced response choice, even when no choice appears appropriate (Dillman & Bowker, 1996) Respondents with older systems may experience system problems or crashes resulting in lost survey responses (Dillman et al., 1998) The need to scroll up or down the page to view all answer choices (Dillman & Bowker, 1996) Disabled response features may appear within the online questionnaire (Dillman et al., 1998) A single response sometimes requires multiple entry actions - e.g., clicking an answer choice and scrolling down to reveal the next instruction (Dillman & Bowker, 1996)	Penmanship issues can impact the legibility of survey responses

Participation ease may function based on a sequential decision matrix. Figure 3 depicts the possible sequencing. Experience with technology, as the underlying factor, may determine a respondent's overall perception that a Web-based survey is easier to answer. When instant access to technology is available, a stronger relationship exists between participation ease and response rate (Tomsic et al., 2000). Easy access to computers and Internet links favored Web-based surveys (Handwerk et al., 2000).

Figure 3: Participation Ease Sequencing Activities

Attractiveness of Survey

Attractiveness is generally accepted as a subjectively pleasant and visually stimulating presentation. Attractiveness of survey instruments combines clarity of content, balanced graphics and reading ease. What constitutes attractiveness, however, varies by respondent. A few general attractiveness features, such as message personalization and screen attractiveness, possess broad appeal. Message personalization facilitates Web-based survey attractiveness to many who might otherwise not respond to a survey instrument. This is accomplished through personalized email attachments or even, in the case of sophisticated designs, the use of IP identification when a respondent logs on to participate in an administration. Screen attractiveness or 'wiz bang' (panache) also was cited as an attractiveness feature for Web-based administration. Limited graphics and professional layout have been identified as attractiveness variables. Paper-based instruments can be designed with special response technology that uses 'special marking pens' or 'rubouts.'

In general, Web-based surveys have a greater potential to enable attractiveness. Web surveys may contain a more refined appearance than email surveys (Dillman, 2000). Redline and Dillman (2000) found that visual design elements, such as color, graphics, and interactive features, can facilitate or distract from the task of completing the survey — hence, the relationship between attractiveness and distraction. Couper (2000) suggests that a Web-based survey can vary from respondent to respondent based on different browser usage and/or settings. Web surveys enable dynamic interactions between the respondent and the survey (Dillman, 2000). Web questionnaires can incorporate invisible response bridges (Dillman, 2000).

Self-administered surveys, whether presented on paper or on the Web, rely on both verbal and visual information to communicate with respondents (Redline & Dillman, 1999). Smith (1995) found that differences in layout format affect both self-administered and proctored surveys. Formatting issues, such as typeface, the use of lines, and black ink, affect a survey participant's willingness to respond (Carroll, 1994). Table 4 provides attractiveness of survey advantages and disadvantages of both media.

Table 4: Attractiveness of Survey - Advantages and Disadvantages Comparing Web-based and Paper and Pencil Surveys

Attractiveness of Survey - Advantages and Disadvantages
	Web-Based	Paper and Pencil
Advantages	Visual design elements such as color and graphics can serve as attractors or detractors (Redline & Dillman, 1999) Web surveys enable dynamic interactions, such as HTML and Java-applets, between the respondent and the survey (Dillman, 2000; Dillman et al., 1998) Availability of various media types (text, sound, graphics, animation and video) facilitates format diversity Sequence and question skips are easily integrated into the survey with little impact on the respondent (Dillman, 2000) Greater potential for respondent personalization (McCooey, 2000) Presentation format variance enables the development of questionnaire designs matched to specific research goals (Schmidt, 1997b) The end user can manipulate screen formats - font, font size, color, and resolution	Survey length/size is immediately visible No knowledge of computer technology is required
Disadvantages	Greater transmission time for graphic intensive questionnaires (Dillman & Bowker, 1996)	Dynamic interactions between the respondent and the survey are not possible (Dillman, 2000) Unable to accommodate audio, animation and video media elements

A greater potential for visual attractiveness exists within Web-based surveys. The opportunity for inclusion of multimedia, particularly the use of sound, graphics, and video is facilitated in electronic formats. Hyperlink activities, enabled in Web-based surveys, permit wide variance in questionnaire design, visual presentation, and provide the potential for a multi-sensing experience.

Novelty Effect

Novelty effect is defined as the appeal of a new or unusual feature, which impacts the respondent's willingness to participate. For some respondents, participating in a Web-based survey for the first time may constitute a novel experience. Novelty effect has a short half-life and is only a factor when a new response technology is introduced or prototyped. The novelty of using a Web-based interactive administration appeals to those with moderate familiarity of electronic media, both email and Internet. Web-based surveys have higher response rates as a function of novelty effect (Cheyne & Ritter, 2001). By contrast, Sheehan (2001) found that over time the novelty effect of electronic email surveys has diminished. The result of the reduced novelty effect is a decrease in response rate (Sheehan, 2001). The over saturation of surveys within society (Marits, 1998) may offer a reverse novelty effect associated with paper and pencil surveys. Table 5 offers novelty effect advantages and disadvantages of Web-based and paper and pencil administrations.

Table 5: Novelty Effect - Advantages and Disadvantages Comparing Web-based and Paper and Pencil Surveys

Novelty Effect - Advantages and Disadvantages
	Web-Based	Paper and Pencil
Advantages	A relatively new means of data collection provides respondents with a novel alternative to traditional paper and pencil administrations	Involves a single marking device, e.g., a pencil or pen, for question response User motivation is modulated by subject matter interest (Rogleberg, Fisher, Maynard, Hakel & Horvath, 2001)
Disadvantages	User frustration from lack of Internet familiarity or computer experience (Dillman & Bowker, 1996) New response features - drop-down menu access, selecting a single radio button versus clicking multiple check boxes - make questionnaires more difficult for some respondents to access and complete (Dillman & Bowker, 1996; Dillman et al., 1998)	A resistance to participate due to societal over saturation of paper and pencil surveys (Marits, 1998)

Administrative Costs

Costs vary greatly for survey instrument management. Administrative costs include instrument development, survey dissemination, questionnaire follow-up, data entry and costs associated with organizing the results. Web-based surveys may contain multiple media elements — audio, video and animation — resulting in development costs, which far exceed those of paper-based instruments. The most cost efficient administration method (advantage) for large samples is Web-based, because of the speed and ease of accessing large and geographically dispersed populations. Web surveys offer opportunities for low-cost self-administered surveys (Bachmann et al., 1999/2000; Cook et al., 2000; Couper, 2000; Matz, 1999; Parker, 1999; Schmidt, 1997a; Solomon, 2001; Weible & Wallace, 1998).

Email based administration is also cost effective and fast, in both dissemination and data analysis (Sheehan & McMillan, 1999). When compared to telephonic survey techniques, Web-based administration continues to be more cost effective, although far less flexible. Activities to increase response rates are more cost effective when conducted via email (Cook et al., 2000), rather than through telephone or postal mail follow-up (Matz, 1999; Weible & Wallace, 1998). Automatic data compilation obtained through Web-based collection means can dramatically reduce research costs (Dillman et al., 1998). Sample size decisions may influence the type of administration selected. Cost trade-off will always be a consideration in decisions on the use of Web-based vs. paper and pencil administration.

Web-based administration may incur additional costs related to survey design, coding, and development. Server hosting costs do not exist for paper and pencil surveys. Web management costs, depending upon the hosting site, may add additional costs for Web-based administration. The trade off may be the volume of survey activity, which can be efficiently handled through online administration. Table 6 suggests administrative cost differences between Web-based and paper and pencil administrations.

Table 6: Administration Costs — Advantages and Disadvantages Comparing Web-Based and Paper and Pencil Surveys

Administration Costs - Advantages and Disadvantages
	Web-Based	Paper and Pencil
Advantages	Offers opportunities for low-cost self-administered surveys (Couper, 2000; Weible & Wallace, 1998; Bachmann et al., 1999/2000; Matz, 1999; Cook et al., 2000; Schmidt, 1997a; Solomon, 2001) Can reach large audiences with minimal expense permitting rapid replies (Cook et al., 2000) No copy or postage fees (Matz, 1999) Enables automatic result compilation (Dillman et al., 1998; Stanton, 1998), thus dramatically reducing research costs	Less costly to create if electronic surveys contain multimedia elements
Disadvantages	High cost of code design and development Server hosting costs Increased Web management costs	More costly in terms of financial, administrative, and researcher effort High costs of survey postage (Dillman, 2000) High follow up costs (Weible & Wallace, 1998; Matz, 1999)

Data entry can be generally classified in three forms: manual — keypunch entry into a database for processing; semi-automated — optical scanning which enables automated data entry for inclusion into a database for processing; and, automated — entries automatically entered into the database at time of keyboarding. The per unit cost decreases the more automated the administration. In a small sample survey, a manual administration may be overall less expensive, but the per unit cost will be greatest for paper and pencil administration (Manfreda et al., 2001).

Response Flexibility

Response flexibility is the degree of individual control available in the design and completion of a survey instrument. Web-based administration provides for a variety of response formats — check boxes, radio buttons, slider bars, scroll bars, text boxes, and drop down alternatives — in the design of survey instruments. These response formats provide greater design control, thus limiting potential data type mismatches. The limiting of mismatch errors further reduces result compilation time, effort and costs.

Paper and pencil administration provides flexibility through a uni-mechanistic response process achieved via a marking implement. Individuals, with limited computer familiarity, may find a greater sense of respondent control over paper and pencil administrations. Expert computer users may find the notion of pencil and paper administration less flexible since many of their communication interactions are conducted electronically. In Web-based administration, forced choice responses can be controlled through non-response defaults, in which the respondent is not permitted to go forward or to submit, when a question remains unanswered. In paper and pencil administrations, this control mechanism is unavailable. Response flexibility is an important variable in instrument design and a due consideration in the choice of survey administration. Table 7 provides response flexibility advantages and disadvantages between Web-based and paper and pencil administrations.

Table 7: Response Flexibility - Advantages and Disadvantages Comparing Web-Based and Paper and Pencil Surveys

Response Flexibility - Advantages and Disadvantages
	Web-Based	Paper and Pencil
Advantages	Availability of programming languages to support flexible presentation display (Schmidt, 1997b) Easy integration of sequence and response bridges into the survey (Dillman, 2000) Increased flexibility in data collection structures, i.e., radio buttons, check boxes, etc. (Matz, 1999)	Availability of additional space for respondent comments throughout the instrument Greater perceived control for respondents with limited computer familiarity
Disadvantages	Screen presentation formats can vary across respondents based on the browser used and the established settings (Couper, 2000)	Researcher control is diminished as a function of unrestricted response formats

Response Time

Response time refers to the time from the initial dissemination to the arrival of completed instruments, including the time necessary for actual administration.

Response time is more efficient with Web-based deployment when delivery time is included. Internet surveys enable rapid return of completed surveys (Matz, 1999; Schmidt, 1997a; Sheehan & McMillan, 1999; Solomon, 2001; Weible & Wallace, 1998). However, when response time is broken down into actual respondent time for completion of the survey, paper-pencil and Web-based are virtually identical (Handwerk et al., 2000). When using the postal system, mail surveys are significantly slower than email based surveys (Oppermann, 1995). Cobanoglu, Warde and Moreo (2001) found that mail surveys (paper and pencil) were 3.5 times slower than Web-based surveys. Overall response time increases as a function of the complexity of survey design (Dillman et al., 1998). Table 8 provides response time advantages and disadvantages associated with Web-based and paper and pencil survey administrations.

Table 8: Response Time - Advantages and Disadvantages Comparing Web-Based and Paper and Pencil Surveys

Response Time - Advantages and Disadvantages
	Web-Based	Paper and Pencil
Advantages	Rapid return of completed surveys is enabled (Matz, 1999; Weible & Wallace, 1998; Schmidt, 1997a; Sheehan & McMillan, 1999; Solomon, 2001)	Total response time decreases for single sitting administrations
Disadvantages	Increased download time associated with multimedia intensive survey designs (Dillman et al., 1998)	Slower administration times as a function of postal distribution systems (Oppermann, 1995)

Population Size

Large sample surveys strongly favor Web-based administration. However, Dillman et al. (1998) suggest a Web culture has emerged that ignores the scientific underpinnings of survey design in favor of larger potential population access. An advantage of Web-based surveys for large-scale administration includes ease of administrative management and direct recording of results. Automatic analysis routines associated with Web-based surveys further facilitate the data analysis process. Additionally, geographically dispersed populations are better addressed in Web-based or email based (electronic format) administration. Electronic formats permit survey researchers to define larger and more diverse populations because of the ease of distribution, administration and analysis. The same disadvantage may exist in both Web-based and paper-based data collections because respondent address lists may be outdated. Table 9 provides population size advantages and disadvantages associated with Web-based and paper and pencil survey administrations.

Table 9: Population Size - Advantages and Disadvantages Comparing Web-Based and Paper and Pencil Surveys

Population Size - Advantages and Disadvantages
	Web-Based	Paper and Pencil
Advantages	Access to large populations of individuals with concurrent time and cost savings (Schmidt, 1997a; Dillman & Bowker, 1996) Availability of mass populations formerly limited to governments and large corporations (Couper, 2000)	Facilitates administrations in which populations are small and intact Useful when administration time is fixed
Disadvantages	Ignoring systematic survey processes in favor of easy access to large populations (Dillman et al., 1998) Requirement to access respondent address lists or specific URLs. Oftentimes, access data are difficult to locate or are maintained on proprietary databases	Difficulties in accessing large populations

Sample Bias

Sampling bias is a type of sampling error that decreases the generalizability of results. Sample bias can have a profound impact on survey administration. Different forms of sampling bias exist. Volunteerism — the respondent's option to select in or out — is reduced when paper and pencil administration occurs under the supervision of a data collector or monitor.

In Web-based administration, only computer literate users with technology access are reachable. Therefore, populations are parameterized without regard to respondents who choose not to use or access electronic media. Internet based surveys are likely to contain sampling errors since not all members of the frame population are measured (Basi, 1999; Couper, 2000). Couper (2000) suggests Web-based surveys are plagued with external validity issues through a potential target population-frame population mismatch. Generalizability of study results may limit the effectiveness of Internet-based surveys.

Surveyphiles are individuals who find surveys/polls appealing. Incentives used to lure Website viewers to an online survey may influence the type of respondents surveyed, thus biasing the responses (Tierney, 2000). An example of a Website survey solicitation is provided in Figure 4.

Figure 4: Example of Website Survey Solicitation

Frequently, surveyphiles are sophisticated in their ability to respond. They may be attracted to Web-based surveys; however, the identification of these individuals is often difficult and expensive. Complex routines to classify and treat sampling bias are required. Research on self-administered surveys suggests that the design of the instrument may be important in obtaining unbiased answers from respondents (Couper et al., 2001). Table 10 describes sample bias differences between Web-based and paper and pencil survey administrations.

Table 10: Sample Bias - Advantages and Disadvantages Comparing Web-Based and Paper and Pencil Surveys

Sample Bias - Advantages and Disadvantages
	Web-Based	Paperand Pencil
Advantages	Potential of a large number of completed surveys improve sample validity (Tierney, 2000) Reduced sampling bias (Solomon, 2001)	Greater control over sampling bias (Epstein, Klinkenberg, Wiley & McKinley, 2001)
Disadvantages	Not all members of the frame population are measured (Couper, 2000; Basi, 1999) The mismatch between target and the frame populations (the largest generalizability issue) (Couper, 2000) Non representative survey populations are common (Matz, 1999) Incentives to lure survey participants influence the demography of final respondents (Tierney, 2000) Two types of respondents are most attracted: those who like to answer surveys (surveyphiles) or those who have a public agenda (Parker, 1999) A need for greater control of response reliability (Stanton, 1998) Greater concern for participant motivation (Stanton, 1998)	Voluntary participation by members within a sampling frame may produce sampling bias

Web-based administrations entail many more control requirements than paper based surveys. Far greater attention is required in Web-based administrations to minimize sampling error.

Instrument Validity

Instrument validity is equivalent between Web-based (electronic) and paper and pencil (manual) survey administration, since content and construct validity are based on survey design (Cronbach, 1990). Threats to internal validity are easier to control for Web-based surveys (Davis, 1999). Most survey designs can be accommodated in Web-based, email or paper and pencil formats. Improved instrument validity is marginally in favor of Web-based administration as a function of better response and completion rates (Buchanan & Smith, 1999). (See Response Rates, Table 2). Table 11 provides advantages and disadvantages of instrument validity issues associated with Web-based and paper and pencil survey administrations.

Table 11: Instrument Validity — Advantages and Disadvantages Comparing Web-Based and Paper and Pencil Surveys

Instrument Validity - Advantages and Disadvantages
	Web-Based	Paper and Pencil
Advantages	Better control of instrument validity through availability of respondent tracking mechanisms Easier control of internal validity issues (Davis, 1999)
Disadvantages	Measurement of instrument validities is time consuming, costly and subject to greater error (Marits, 2002)	Measurement of instrument validities is time consuming, costly and subject to greater error (Marits, 2002)

Management of Non-Response Data

Non-response data are those data in which a survey respondent either provides no response, a spurious response or a clearly false one. Survey research convention typically ignores non-response data by discarding obvious erroneous or sabotaged answers. Lack of clarity of a respondent's intent is also a potential issue of survey data management. Through the use of electronic techniques and the use of automatic analysis routines, non-responses can be measured and provided as usable data. Errors produced by non-responses are better controlled in Web-Based surveys (Couper, 2001). Electronic data collections can incorporate forced pauses, which will not advance until a response is provided (Schmidt, 1997a).

In a recent study comparing online and Opscan (paper and pencil) for a U.S. Government agency, non-response items were found to be greater for online administration (Church, 2001). "Don't know" responses were slightly greater for Opscan (paper and pencil) than for the online administration (Church, 2001). Table 12 provides management of non-response data advantages and disadvantages between Web-based and paper and pencil survey administrations.

Table 12: Management of Non-Response Data — Advantages and Disadvantages Comparing Web-Based and Paper and Pencil Surveys

Management of Non-Response Data - Advantages and Disadvantages
	Web-Based	Paper and Pencil
Advantages	Can include a no response choice to differentiate respondents' intent The underlying code can catch unacceptable responses through real-time error checking (Schmidt, 1997a) Programs can guard against multiple submissions by the same respondent (Schmidt, 1997a) Significantly lower levels of missing data (Stanton, 1998)
Disadvantages	Increased variability of missing data associated with differences in data collection mechanisms - entry box versions produce more missing data than radio buttons (Couper et al., 2001) Missing, unacceptable, incorrect or duplicate data and increased security risks (Schmidt, 1997a)	Respondents can quickly skip through questions and provide spurious data to expedite survey completion Increased frequencies of routing and skipping errors (Marits, 2002)

Response Error

Response error is defined as the class of all potential responses that are either incorrect, inaccurate, intentionally deceiving, misguided, or methodologically misclassified. Incorrect responses represent answers that are different from that which are being asked. An incorrect response may include an inaccurate response such as the percentage of time a respondent utilizes the computer for daily activities. The respondent inaccurately indicates a level that is either above or below the actual amount. This is sometimes classified as a misguided response, distorted response (Potosky & Bobko, 1997), or mischievous response (Schmidt, 1997a; Smith & Leigh, 1997), based on intention — either intentional or unintentional. Response error is equivalent, between Web-based and paper and pencil administrations, in short answer surveys (Liefeld, 1988), while for longer or more complex response requirements, computer or Web-based surveys showed greater inaccuracies (Liefeld, 1988; Manfreda et al., 2001).

Response error can be generated from social desirability phenomena, whereby individuals have the propensity to answer in a manner thought to be most socially acceptable rather than how he/she actually performs (Schwab, 1999). While response error through social desirability is generally unintentional, there is also response error from intentional deception — a response intended to skew data in a given direction — mischievous response (Schmidt, 1997a; Smith & Leigh, 1997). Response error may occur when the readability index exceeds the reading level of the respondent. Methodologically misclassified responses are those in which the types of data are miscoded. Response error can be partially controlled through expanded explication of responses, either through follow-up or the use of multiple questions aimed at assessing the same construct.

Martin and Nagao (1989) showed that a degree of respondent honesty exists when comparing computer-administered, paper and pencil, and face-to-face interviews, but the difference was not statistically significant. Less response error was found in the computer-administered scenarios. Table 13 provides management of response error advantages and disadvantages between Web-based and paper and pencil survey administrations.

Table 13: Response Error - Advantages and Disadvantages Comparing Web-Based and Paper and Pencil Surveys

Response Error - Advantages and Disadvantages
	Web-Based	Paper and Pencil
Advantages	Ease of generating subroutines to capture some inaccurate responses Internal consistency is easily dealt with at the time of data collection Computer-administered surveys tend to have less response error (Martin & Nagao, 1989; Rosenfeld, Giacalone, Knouse, Doherty, Vicino, Kantor & Greaves, 1991; Solomon, 2001)	Ease of correcting a response at any time during the survey completion process Paper and pencil administrations tend to contain lower response error (Allen & Fry, 1986)
Disadvantages	Computer administered surveys tend to contain greater inaccuracies for longer or more complex response requirements (Liefeld, 1988) Inability to backtrack, preview or change responses (Lautenschlager & Flaherty, 1990)	In optical scanning data entry, response error increases for paper and pencil survey responses as a function of the sensitivity of the optical scanning equipment

Comparative Summary of Web-Based and Paper-Based Survey Administrations

Table 14 offers a comparison between Web-based and paper-based administrations. Academic and public media sources are included. A comprehensive review of the literature suggested the treatment of 12 parameters for consideration in comparing and contrasting Web-based administrations to paper and pencil administrations. The table below was generated through examination of the literature sources for the parameters and its focus on either Web-based, paper and pencil or mixed mode. In some cases, parameters were not tagged because treatment was peripheral to the ancillary investigation.