CROSS-CULTURAL RELIABILITY

While the validity of an instrument reflects the correlations between measures of the same construct when the measures are maximally independent, its reliability concerns the degree of agreement among maximally similar methods . In attitude measurement there are several techniques for evaluating the reliability of a measure. They can be split into three major classes: measures of stability, measures of equivalence, and a measure of discriminant reliability. The first aspect is the measure of stability, or "test-retest" reliability. In order to overcome the limitations of measures of stability, such as the problems of "reactivity," measures of equivalence have turned to become a more and more valid alternative. Discriminant reliability refers to the use of scale scores to distinguish between two or more existing groups in a population.

For the purposes of international market research we need to take the application which gives information about the ability of the instrument to distinguish between two or more groups. This test of the discriminatory power of the instrument under investigation is especially important in cross-cultural research, the validity of which stands or falls by the differences found between the cultures involved.

Problem Areas in Cross-Cultural Market Data Collection

Many scholars have pointed to the difficulties of cross-cultural data gathering, presenting important sources of error in the measurement of meaning (Berrien, 1967; Berry, 1980). However, in international market research there are ways to minimize sources of error that are specifically related to data-gathering in cross-cultural environments. Increasingly you need to become aware of the humble attitude needed.

First effects can be taken to reduce errors in the administration of the questionnaire. In some cultures the use of questionnaires for market research is familiar and laws protect people from the abuse of data gathered in this type of research. In others it is a unique event and, quite frequently, anonymity is questioned at every step. Some developed societies , such as Germany, often question the purpose of research in order to make certain that data and results are used properly. This is even amplified when questionnaires are done orally through, for example, the telephone. Some cultures don't need to know the person on the line; others, however, want to know the individual to whom they are revealing their intimate thoughts about a product.

In the use of a questionnaire the assumption is made that people are individuals with a clear attitude and opinion about a certain product or situation. When THT were doing research in Japan we found that subjects wanted to have a discussion about the question before answering it. Obviously there are individual opinions in Japan, but these are formed after a collective discussion.

In general, clarity about the reasons and purpose of research helps significantly in most cultures. People appreciate getting a better insight into the why and how of the research. Some cultures, however, insist on getting feedback, while others see it as a duty to give their opinions and feedback is not asked for or may even be rejected.

Defensive Responsiveness

The defensiveness of respondents is a roadblock across the path of virtually all research programs that use either the interview or questionnaire technique. It is not at all unusual to encounter defensive evasiveness relative to some types of questions in groups of respondents or by individuals. For example, inquiries about sexual relations or religious beliefs might be accepted and answered fully in one culture, whereas in another they would be considered so threateningly intrusive that the very asking of them could disrupt the interviewer-interviewee relationship. Generally, in any survey research where the investigators become actively involved in generating data, they are, in principle, able to control certain sources of error. This can be done by informing the participants about:

• The purpose of the research. For what organization is it being carried out and what will be done with the end results?

• The role of the investigator. Which organization is the investigator working for and what are the codes of integrity that apply?

• The role of the participant. Subjects need to feel secure, for example, about their anonymity and privacy, and about the anonymity of their answers.

Questionnaires can be designed in ways that reduce the possibility of their creating some forms of defensive responsiveness. First of all, in sections which introduce a specific set of questions, interviewers can remind the subjects of the fact that there are no right or wrong and no good or bad answers. Secondly, the section on subjects' personal data can be kept for the end of the questionnaire, or even done separately.

The content of the questions needs to be chosen with the utmost care. As Guthrie observed , generalized questions referring to a wide range of similar concrete patterns are a far better means of minimizing the effects of defensiveness than are particularized questions (Guthrie, 1971). Even more precautions can be taken without reducing the degree to which one measures meaning. Kluckhohn and Strodtbeck (1961) suggested avoiding questions in areas of behavior where defensiveness might be expected. Third-person phrasing can be used in the written presentation of problematic life situations in order to attain a situation in which interviewees are invited to give their opinions about statements made by third parties. This reduces the possibility of personal identification, which might lead in certain cultures to various forms of defensive responsiveness.

Despite all precautions market researchers will still retain areas where defensive responsiveness might occur in some cultures. It may furthermore be helpful to construct questionnaires where the commonsense content of the messages is such that interpretation is quite invariable to the interpreter (Drever, 1974).

Ethnocentrism

The very idea of cross-cultural market research, we are afraid, reflects most probably a western, universalistic value position. In a dictionary of psychology, ethnocentrism is defined as an "exaggerated tendency to think the characteristics of one's own group or race superior to those of other groups or races." To avoid ethnocentrism in data collection, you should develop instruments for cross-cultural use cross culturally. A way of culturally decentering the data collection methods is to work in multicultural market research teams .

Data Analysis

One of the objects of analyses of data coming from market research instruments is to gather essential meaning in the buying behavior of those people researched. In the endeavor of gathering meaning there are basically two ways in which, out of a complex mass of information, one can find underlying relationships: by reflection and by computation. In the design of research instruments reflection on qualitative relationships needs to precede computation.

A process of computation of a quantitative nature follows this reflection on qualitative relationships. The quantitative account includes all that is contained in the corresponding qualitative one, and as such both are necessary, not antithetical or alternative: "Quantities are of qualities, and a measured quality has just the magnitude expressed in its measure... [it is] not a matter of ontology" (Kaplan, 1963).

Statistical Analysis of the Data

We need to analyze the problematic methodological issues that underlie the type of data treatment applied to the meaning of human behavior. In this type of market research some characteristics of the obtained data need to be analyzed , such as the levels of analysis, within-culture regularities, and the level of measurement.

Level of Analysis

In market research data are often collected from individual responses on questionnaires. These data come from individuals within a specific culture, within a specific society. Before analysis, the data needs to be transformed into societal or cultural data through a process of calculation, either from mean values of variables for each location or, in the case of dichotomous variables , from percentages. What we observe here is the existence of two possible levels of analysis: the individual or ideographic level and the societal or nomothetic level, as mentioned in Chapter 2.

Correlation analysis of market data, in which we study the relationship between the meaning of a product and the behavioral orientations of its potential buyers , leaves the following choice (Hofstede, 1980):

• a global correlation between all individuals regardless of the society they are in;

• a number of within-society correlations, one for each society, between those individuals belonging to that society, and

• a between-society correlation, or ecological correlation, based on the mean scores of the variables for each society.

In taking the first alternative of a global correlation between individual scores we disregard the culture from which the individuals are drawn. This aims to explain the variance in the individual scores of the various value orientations. In itself this procedure is quite valuable , but it does no more than analyze the relationship between the individual's "personality" in terms of the individual set of value orientations and conception of the product. It does not reflect on the relationship between the group's set of value orientations, nor its (shared) conception of the product. We should resist the temptation to generalize the findings that result from this procedure to the cultural or ecological level, because in that case we would be trapped in an "individualistic" fallacy or what Hofstede has termed the "reserve ecological fallacy," that is, the "construction of ecological indices from variables correlated at the individual level" (ibid.). It involves inferring relations at higher levels of generality on the basis of observed relations recorded at lower levels (Scheuch, 1966; Smelser, 1976). This type of fallacy is the result of the false assumption that cultures can be treated as individuals, while in fact they cannot: "Cultures are not individuals: they are wholes, and their internal logic cannot be understood in the terms used for personality dynamics of individuals. Ecologic differs from individual logic" (ibid.).

A second alternative statistical treatment that refers to an analysis that results in a number of within-society correlations, one for each society, between those individuals belonging to that society, is of interest from a cultural point of view, but is not to be confused with ecological correlations. A confusion between these two types of correlations is known as the "cultural or ecological fallacy" (Alker, 1966). It is the fallacy "which involves the inference of correlations at the level of individual persons on the basis of correlation among aggregated attributes" (Smelser, 1976). The results of the validity and reliability tests of the questionnaires within and between cultures, therefore, are only applicable for that specific purpose and should not be allowed to be translated into ecological conclusions. For attaining ecological correlations a third alternative statistical treatment has to be executed.

This third alternative, which is referred to as a between-society correlational or ecological analysis, is the major statistical instrument for the treatment of cultural data and thus for international market research. It concerns the relationship of culture and market behaviors that is assumed to occur on the ecological level.

The variables that concern the conception of the product or service on the one hand, and the variables that concern the organization of meaning or culture on the other, are similar to what Lazarsfeld and Menzel referred to as "structural properties" of social units - those properties "which are obtained by performing some or all of the others" (Lazarsfeld and Menzel, 1961). Reference is made to regularities in the meaning a population assigns to a product and to behavioral orientations. It is in the sharing of meaning that we transcend from the individual to the ecological level.

In order to be able to establish ecological indices from collected data of individuals from a society both the conception of products and the various value-orientations are shared by its members . This information can only be achieved by a within-culture analysis.

Within-Culture Analysis

The analysis of data obtained from the individuals in each specific location gives one the necessary information on which one can determine whether the individuals share - to a certain significant extent - the meaning assigned to the product or service on the one hand, and to value orientations on the other. It refers to "intra-cultural regularities."

Level of Measurement

The application of certain statistical techniques not only depends on the nature of the research questions that they are designed to answer, but also on the nature of the data to which they may be applied. The nature of the data determines, to a large extent, the appropriate statistic and the appropriate interpretation of the results. The most basic information in this context of the level of measurement of each of the variables is the data set.

The specific characteristics of measurement in the social sciences are one of the basics on which rests the choice for particular statistical tests out of the numerous correlational analyses that are available. In the scoring or measurement of social variables the social scientist, like the physical scientist, assigns numbers to observations in such a way that the numbers are amenable to analysis by manipulation according to certain rules. The operations that are allowed on a given set of scores are dependent on the level of measurement achieved.

In view of the fact that most instruments used for international market research are aimed at performing ordinal-level measurements - the specific country scores can ultimately be ordered in ranks only - we are committed to the use of available non-parametric statistics. Studies using statistical methods for determining the relationship between quantified variables across cultures are called halogeistic ones. The method we use for this purpose at THT is partial correlational analysis (Robinson, 1950).