« Comment Here on "Taking the Leap from Dyads to Triads" | Main | Comment Here on "How to Do Research" »

September 01, 2008

Comments

Antonio

Who is the key informant for an organization? I think it depends on sampled industry characteristics. As my empirical data usually come from Greater Pearl River Delta region (PRD), my samples are mainly small and medium enterprises in Hong Kong (still the top FDI party in the PRD). After more than 30 personal interviews and a few years industrial experience, the one that really understand a company's supply chain operations is the "Owner", who ventures and manages the company over so many years. I remember that, in one case, when I discussed purchasing strategy with a purchasing manager of a 300-people company in PRD, the purchasing manager did not know it, but the owner knew. The purchasing manager was just another "doer". Thus, asking functional manager for a "big picture" may not be good, but he/she may know the practices well. Thus, I suggest that, for big picture, ask the senior; for operations, ask the functional management.

I know that different industries/sectors have different practices. Please share your view on it.

Multiple informants and objective data are the best choice. However, it is not the "cost" issue, it is the "possibility" issue. It is difficult, if not impossible, to ask non-public listed companies for objective data. Multiple respondents are possible only if you can reach the company's senior management. I tried it once, but failed. In one case, I discussed with some company seniors about a multiple informant approach that I mail two questionnaires to each company's operations manager and purchasing manager respectively. Those seniors said that if you got one response, would you use it? If the returned questionnaire had no contact information, how did match the two questionnaires? Numbered questionnaire might work but they didn't like to be recongized and might not return the questionnaire. There are just a few practical issues. Who can teach me how to tackle them?

Regards,
Antonio

Kris

Theoretically, multiple respondents approach sounds great. However, practically, especaially for the 'green' researchers with not much resource (time, money and manpower), it is nearly impossible to collect multiple samples from a single organization.
Thus I think we should prepare different sets of questions for practitioners at different levels (more focusing). However, this may lead to the small sample size for statistical analysis. I thus always am in this kind of dilema- the response rate/ sample size and the appropriateness of questions.

Any advioe?
Best,
Kris

Irfan Ertugrul

Although multiple respondents approach is ideal, it’s so difficult for the supply chain research. During the survey there are no responses from the companies many times. Another aspect is filling the survey by somebody other than the person who is targeted. This is an unavoidable situation because the people in the industry are always busy. In this manner researchers must be satisfied with single respondent. To overcome this problem, researchers must make the industry conscious of the responses by them because responses from them are very important and affect the result of research.
Another important point is explaining the concepts about the survey, when the key respondent is determined. While filling the survey by key respondent, the researcher must be there in case of answering the possible questions of the respondent.

Paul Skilton

I don't think how difficult the process of getting multiple respondents is the right criterion. The fact is that as a reviewer, a single respondent survey is a signal that the validity of the study is questionable. Multiple respondent studies, on the other hand, are often regarded as better on that basis alone. As Pagell and Krause point out, it is not always a fatal flaw, but it often is. What would you rather do, spend your time doing something easy that won't be published in a high quality journal, or make the effort to do work that will be recognized as high quality from the outset, and stands a vastly better chance of being published in JOM or a similar outlet?

There are two other issues I want to bring into the discussion. The first is the structure of survey questions, which should be directed toward recall of specific events or actions the respondent was personally involved in. You don't have to specify the event or action in advance, just prime the respondent to choose one specific object: A project, a supplier or a product. I would tend to disagree that high level questions on topics 'all managers' know about are more valid to ask when using single respondents. Many a firm has been managed by collectively deluded teams - as a teams/groups scholar, I would even propose that group beliefs may even be more likely to be incorrect than individually held ones. There is good empirical evidence of this (Chattopadhyay et al., 1999; Miller et al., 1997; Wright et al. 2001)

My second issue has to do with the ways that authors try to justify single respondent surveys. No form of factor analysis can demonstrate that there is no bias in a set of responses from a single respondent. The fact that responses load on factors is also a function of the design of the survey. Survey questions are designed to produce factors, and do so even if respondents (as they often do) have an overall optimistic or pessimistic range of response. Unless authors are prepared to admit that their survey instruments are a set of unrelated questions, factor analysis of single respondent surveys can only be said to detect how well a survey is designed.

C Chet Miller, Laura B Cardinal, William H Glick. (1997). Retrospective reports in organizational Research: A reexamination of recent evidence. Academy of Management Journal, 40(1), 189-204.

Patrick M Wright, Timothy M Gardner, Lisa M Moynihan, Hyeon Jeong Park, et al. (2001). Measurement error in research on human resources and firm peformance: Additional data and suggestions for future research. Personnel Psychology, 54(4), 875-901.

Prithviraj Chattopadhyay, William H Glick, Chet Miller, George P Huber. (1999). Determinants of executive beliefs: Comparing functional conditioning and social influence. Strategic Management Journal, 20(8), 763.

Antonio Lau

Shall we withdraw from adopting single informant approach in survey study? How do we ensure completely that the survey respondent(s) understand the meaning of each question and understand they know how to answer the question and answer it correctly within 10 mins? Could we validate survey data through statistical analysis? If not, then why do we pay so much effort on the analysis, getting magic number p-value< 0.05 with CFI/IFI/ etc. over 0.90?

I have no answer yet. Could we find someway to truely get rid of them?

Antonio Lau

The comments to this entry are closed.

Blog powered by Typepad