« Welcome to the Operations & Supply Management (OSM) Forum | Main | Comment Here on "Taking the Leap from Dyads to Triads" »

May 01, 2008

Comments

Sambhavi Lakshminarayanan

Business related "research" necessarily is tightly related to the business environment and specific challenges that companies face and attempt to resolve. Thus, a major change factor in the business environment, that significantly affects a majority of businesses, would immediately generate related scholarly work. In the initial stages, when the change is new, businesses attempt to leverage it to their advantage. At that time, there is great scope for research to analyze, explain and model the impact of this change. Subsequently, as the change is absorbed into the environment, research related to it matures as well.

We can track this pattern of change in the case of supply-chain management. From the initial stages of innovation, when a few companies began utilizing it to differentiate themselves in the market place, it has now become de rigueur. Research methodologies mirrored this process - moving from surveys and case-studies of particular businesses to more general, analytical models that classify and link.

In the case of Marketing, there are have been changes in the related business environment, such as the impact and influence of technology, that have renewed and regenerated research. As and when businesses utilize supply chain management in new ways, the area will see a corresponding mix of methodologies that address the situation.

Jan Holmström

Even though Kuhn is best known for his paradigms he makes an other interesting point about the possible development paths of a research domain. He makes a distinction based on what is the researcher attitude towards knowledge. His two categories are research towards what the researcher wishes to know and research based on what the researcher knows. The more researchers shift towards seeking answers to well formulated problems the more of a puzzle solving exercise the research becomes. Only when researchers manage to keep their minds open to the unexpected research can continuously renew and stay innovative.
Nevertheless, the discussion on methodological approaches in supply chain management is welcome. There is a need for methodologies that fit the maturity of a research problem, both established and new problems. As the essayists point out the move towards econometrics is not a problem unless it comes to dominate. The shift towards survey based empirical research that has already happened in the supply chain management field to me illustrates the cost of domination. Domination of survey based empirical research has left little room for research on emerging problems and solutions that cannot (yet) be surveyed. Puzzle solving is what the field of supply chain management is already in danger of becoming due to the domination of a methodological approach, and tipping once more towards econometrics will not make it much worse.

Kenneth Green, D.B.A.; Pamela Zelbst, Ph.D.; Victor Sower, Ph.D., C.Q.E.

We often use the term paradigm shift too freely. What the authors describe could hardly be called a paradigm shift. What they describe is the reaction of researchers to changes in the research environment. However, their comments also address the need for multiple methodologies in SCM research to address the weaknesses of single methodologies so that the resulting research is robust.

McGrath (1982) points out that all research is inherently flawed because researchers are attempting to maximize generalizability, precision of measurement or realism of setting. With the ability to place the focus on only one of these areas, the others are reduced. For example, a survey maximizes generalizability but lacks precision and realism. As McGrath points out there exists a need for triangulation using multiple methods as well as replication of studies. When triangulation is used there can be a convergence of information to compensate for inherent limitations. With this in mind then there is room in SCM research for all methods including econometrics and modeling.

The authors are concerned that econometrics will take over as the preferred methodology for supply chain management related research. Econometrics incorporates the use of multiple regression and secondary data. From a very practical perspective, low response rates to surveys may be driving the shift from SEM to econometrics. It is becoming increasingly difficult to collect the requisite 200 responses to support SEM. A fall back position when sample sizes are small may be to adopt a regression-based methodology. It may be necessary for SCM researchers to abandon SEM in favor of path analysis and PLS when sample sizes fall below 100.

The authors are concerned that economics will take over as theoretical basis for the SCM research agenda. While economic theories may serve as the foundations for SC activities, SCM requires that behavioral theories be layered on top of the economic theories to give a full picture of the necessary linkages between supply chain partners. If the key to successful supply chain competition is the relationships among the supply chain partners, then the behavioral aspects of building win-win relationships that result in global optimization ultimately leading to improved organizational performance must be investigated. Economics alone does not provide a full picture. SCM researchers must incorporate both economic and behavioral theories and supporting methodologies.

The authors indicate that we are currently in a relatively peaceful interlude in the SCM researcher arena. We do not believe that this is the case. While we believe that SCM researchers have done a relatively thorough job of describing supply chains and supply chain management and describing strategies and tactics that support supply chain management efforts, we have not fully assessed the relationships through our research efforts. For example, there currently exists no good measure of supply chain performance in the literature. It is also very difficult to gather data related to supply chain management strategies and supply chain performance throughout the expanded supply chain from supplier’s supplier to ultimate consumer. Much of what we currently assess relates to relationships with direct suppliers and customers. We have talked extensively about supply chains, now we must carefully assess our theories with data from the real world. This is not a time of peaceful interlude.

References

McGrath, J.E., 1982. Dilemmatics: the study of research choices and dilemmas. In: McGrath, J.E., Martin, J., Kulka, R.A. (Eds.), Judgment Calls in Research.

Mikko Ketokivi

Carter et al. implicitly equate methodology with specific statistical methods, such as structural equation modeling and econometrics. Kuhn’s treatment was, however, much more fundamental. Methodology is not the same as method, it is a branch of philosophy concerned with the fundamental structure of scientific reasoning, which includes explication of one’s ontological and epistemological premises. SEM is a collection of statistical techniques used to analyze correlations, and as such makes no ontological or epistemological commitments; SEM is a tool.

I don’t think we have had a single Kuhnian paradigm shift in OM for as long as I can remember, and I don’t see any impending shifts in the immediate future either. To be sure, going from SEM to econometrics is not a paradigm shift.

In my view, the essence of contemporary OM methodology in the philosophical meaning of the word is as follows:

1. We are ontologically realists: we assume that the reality we study is not dependent on cognitive states. Whether a supply chain is efficient or not is established with reference to objective and impartial measurement, not subjective interpretation. A supply chain cannot simultaneously be efficient and inefficient.

2. We are committed to the operationalization of unobservable concepts (e.g., competences, competitive priorities, competitive advantage, commitment, customer satisfaction), and in our conclusions often also make claims about these unobservables. We thus embrace a rather strong version of scientific realism.

3. Potential truth claims notwithstanding, the most important de facto standard for evaluating empirical research is assessment of empirical adequacy, that is, the ability of our theories to account for observable phenomena. This is where statistical inference is often invoked: we establish empirical adequacy by looking at statistical associations--a model that fits is empirically adequate.

4. We are epistemologically objectivists in the sense that we engage in scientific inquiry as outsiders: we assume that we can get the data we need by having people fill out surveys or by interviewing them. We assume objectivism also in the sense that we trust our informants to provide us with impartial and honest, indeed objective descriptions of the state of affairs in their respective organizations.

All of these premises can be challenged from a philosophical perspective: we could, with good reason, be criticized of being naive realists who rely on outdated epistemology. But the goal of methodology is not to provide the "correct" or epistemologically flawless set of rules and procedures (Kuhn was very clear on this), the goal is to enable scientific inquiry as a collective effort. Without a paradigm there is no scientific community: two researchers who disagree on methodology can, at best, "agree to disagree." This is the essence of Kuhnian paradigms: Einstein’s theory can be accepted only with the recognition that Newton’s was wrong (although that Einstein is in turn right is not implied).

To be sure, combining different data collection methods and using multiple-informant research designs ("triangulation") in an attempt at more reliable and valid measurement is a great idea. But calling this "multiple methodologies" must be tagged with the disclaimer that we are using the word "methodology" in a manner that is inconsistent with its use in philosophy. Why not just say "multiple methods" or "multiple sources of data" to avoid confusion?

JACK HAYYA

The authors err in postulating that Supply chain Management (SCM) is a new paradigm and that it is threatened by a noxious methodological invasion (they call it revolution) from areas such as Marketing (Heaven forbid!). In fact, SCM is an evolution of older paradigms in multi-echelon design and control. See for example Clark (1974). What spurred interest in this so-called new field is globalization and then the term or acronym SCM, just like JIT or lean manufacturing, has a nice ring to it.

I do not subscribe to what I consider to be artificial boundaries between methodology and theory, for, after all, good methodology derives from theory. And I am irked by those who insist on these divisions and who create their own jargon to further their private interests. Thus there unfortunately are myopic or dogmatic editors or associate editors who condemn us through their egocentric predicament: “don’t bother us with any research we are unfamiliar with.” Hence I argue that it is intellectually dishonest to claim that this article or that is too theoretical or too methodological to receive consideration – that this article is SCM and ought to go to a SCM journal, whereas this other article is JIT and should go to a JIT journal. And the same applies to the authors’ argument about methodology, an argument that I consider specious. After all, research is good if it attempted to answer questions (in our field) heretofore unanswered. Look at the oral histories of Studs Terkel (wikipedia.org/wiki/Studs_Terkel). Who is to say they are not good research and who cares whether they use case study methodologies.

I was in a department called Management Science and Information Systems, but our ex-Dean (Judy Olian, now Dean of the Anderson School at UCLA) got fed up with our internal wrangling and threw us into Business Logistics in a department called Supply Chain and Information Systems. Now the courses we used to call MSXXX or QBAXXX have been transformed to SCMXXX. I asked some instructors what they taught in these SCM courses and they all said ‘Excel.’

On the bright side, SCM has spurred fresh innovative research with dissertations (for example, Chatfield (2001) and journal articles and new ways of addressing things. So SCM may be “old wine in a new bottle,” but it is vintage wine. Besides, our Center for Supply Chain Research (CSCR) keeps accepting my working papers and sponsoring my conference excursions.

References

Chatfield, D.C. (2001). SISCO and SCML – Software tools for supply chain simulation modeling and information sharing. Unpublished Ph.D. dissertation, Department of Management Science and Information Systems, Penn State University.

Clark, A.J. (1974). An informal survey of multi-echelon inventory theory. Naval Research Logistics, 19, 621-650.

wikipedia.org/wiki/Studs_Terkel

Byron Keating, Ph.D.

I was a little surprised by the assertion that supply chain management is in the throes of a paradigm shift in its dominant methodologies. Carter, Sanders, and Dong assert that supply chain management, like the closely related discipline of marketing, is being overtaken by econometrics at the expense of the existing survey-based methods. While many may question their assertions based on epistemological or evidentiary grounds, there remains an intriguing question: why are we increasingly turning to econometrics?

If we entertain the notion that Carter et al. may actually be observing a genuine shift in the methodological foundations of our discipline, then I can’t help but ask, why? For those that subscribe to the view that supply chain management, like the natural sciences, can be described by a set of common truths, and that our understanding of these truths advances according to the same structures of scientific revolutions that Kuhn (1970) observed in the natural sciences, then we should also be able to observe the circumstances that contribute to these revolutions. Kuhn asserts that paradigm shifts occur when science is confronted with an anomaly or crisis that cannot be addressed by existing methods and knowledge (p. 90). Carter et al. seem to have missed an important point: if we are indeed confronting a shift in method paradigms, it is because the old paradigm no longer meets the needs of our discipline.

Singhal et al. (2008) allude to several shortcomings in our existing methods paradigm in their response to a call by Boyer and Swink (2008) for multiple methods in operations and supply chain research. Some of the challenges identified include (i) the difficulty of obtaining survey data; (ii) the inability of existing methods to effectively combine primary and secondary data; (iii) the need to avoid common method and respondent biases; (iv) the need to control for cultural sensitivities; and (v) the need to achieve congruence in the way that key OSM constructs are defined and measured.

Could it be that econometrics enables us to resolve some of these issues? In this regard, I would like to draw attention to a new breed of econometrics pioneered by Nobel Prize–winning economist Daniel McFadden (Berkeleyan, 2000). Commonly referred to as choice modeling, this family of methods incorporates the principles of econometrics within a behavioral framework to resolve many of the above issues. The use of sophisticated experimental designs results in robust parameter estimates from relatively modest sample sizes. The models used can easily accommodate secondary data, and avoid typical method and respondent biases, including cultural bias, by asking respondents to make choices between realistic bundles of predefined attributes or features. By defining the attributes or features, researchers avoid many of the problems associated with scale development and refinement, and can have confidence that people are responding to the same thing.

Carter et al. may be concerned that we are on the tipping point of a methods revolution—encouraging us not to abandon our tried-and-proven methods, but if we are in fact on the verge of a paradigm shift, then the needs of the discipline will ultimately prevail over the needs of any particular researcher. They must do so. Ironically, while Kuhn indicates that newer paradigms are usually incompatible with older ones, he does not suggest that it is necessary to abandon the former (p. 91). Kuhn implies that in some cases a new paradigm may apply to, and consequently replace, only a small part of an existing paradigm. Hopefully this will provide some solace to researchers who feel anxious at the prospect of a methodological paradigm shift.

References

Berkeleyan 2000. McFadden receives this year's Nobel Prize in economics, http://www.berkeley.edu/news/berkeleyan/2000/10/18/mcnobel.html, accessed on 18 May 2008.

Boyer, K.K., and Swink, M.L. 2008. Empirical elephants—why multiple methods are essential to quality research in operations and supply chain management, Journal of Operations Management 26, 337-344.

Carter, C.R., Sanders, N,R., and Dong, Y. 2008. Paradigms, revolutions, and tipping points: the need for using multiple methodologies within the field of supply chain management, Operations and Supply Management Forum, Journal of Operations Management, http://nebula.bus.msu.edu/JOM/osm.asp, accessed on 18 May 2008.

Kuhn, T.S. 1970. The structure of scientific revolutions, University of Chicago Press, Chicago, IL.

Singhal, V., Flynn, B.B., Ward, P.T., Roth, A.V., and Gaur, V. 2008. Response and comments: Empirical elephants—why multiple methods are essential to quality research in operations and supply chain management, Journal of Operations Management 26, 345-348.

Elliot Bendoly

The risk suggested by the authors regarding a shift in preference towards econometric modeling is a real one, though unfortunately only a natural consequence of the methodological orientation of “top schools”. While it is true that we as a field of research should embrace new methods, it only makes sense to do so when appropriate. This then leads to the main question of when econometric models and their traditional reliance on secondary data should be viewed as appropriate. A double barreled statement? – Perhaps… In practice however it’s rare to find econometric models applied without the reliance of secondary data. So, narrowing the scope here, let’s consider some of the myths / assumptions that surround secondary data often used in econometric modeling:

(1) Myths of Objectivity and Accuracy
A frequent assumption made by individuals that don’t have a rich history of working with secondary data is that such data is in some way "cleaner" than say survey data (as an example of contrast). That’s a fairly big assumption. Secondary data is ultimately still subject to the collection of "people" (either with direct influence or indirect influence in the case of automated programs developed by people to collect such data from other systems). Still potentially more problematic is the risk that unforeseen organizational factors may play in such secondary reports. In this day and age, how much faith can practice place on our ability to advise them regarding the use of complex prescriptions based on numbers that they themselves may know to be subject to flaw (deliberate or otherwise)?

(2) Myths of Completeness and Criticality
Certainly there are key critical elements implied by theory and research evidence, as well as directly discussed in anecdote, that are far from being captured in secondary data sources today. These are often "perceptions" of individuals, and aside from likely being embedded within secondary "objective" reports, can not be isolated through such reports. A predominant reliance on such data (even within a single study) would force the interpretability of results to be bound to market-influences (which feed both selection and reporting biases), rather than challenging the assumption that other forms of information should also be collected in practice. This is entirely against the very fundamentals of our discipline and its interest in advancing practice.

(3) Myths of Robust Appropriability
An additional risk associated with a shift towards models relying heavily on secondary data as a "dominant form of OM empiricism" is the inevitable and inappropriate reuse of objective data for multiple alternate proxies of reality. We have already seen much of this in existing archival research (eg. note the common use of inventory turns as both a proxy of ‘performance’ and a proxy for ‘policy’ along which performance is viewed to be dependent). This of course is related to the fundamental problems with often linked to "convenience" sampling. Much reinterpretation of data that takes place in econometric modeling is simply inappropriate – but gets little attention from dominant academic establishments. Yet this is akin to a subjective abandonment of the very objectivity that researchers often associate with such data and methods. Selective rigor isn’t rigorous.

John E Boylan and Aris A Syntetos

Research Paradigms in OM and OR


John E Boylan1 and Aris A Syntetos2

1. Buckinghamshire New University, UK
2. Salford University, UK

The discussion on what counts as valid and rigorous research in Operations Management (OM) is important. There has been a similar debate within the field of Operational Research (OR) in the UK, which may shed some light on the question of appropriate methodologies and paradigms for OM.

We agree with all that has been said about the need for methodological diversity within Operations Management (OM). The question is whether this diversity should be based on a single paradigm, or on multiple paradigms. In this commentary, we take a broad view of the concept of a paradigm, which incorporates theories, standards and beliefs, as well as methodologies.

Firstly, it should be recognized that not all methods are based on the same paradigms. Much modelling work is based on a positivist paradigm. In the British OR community, this is known as the hard OR paradigm. It underpins such methods as optimization, Monte Carlo simulation and queuing theory. Working within this paradigm, models are designed, based on well-specified assumptions, and the implications of the models are evaluated. Empirical observations are used to test such models.

Other OR modelling work is based on a phenomenological paradigm. In the UK, this is known as the soft OR paradigm, and underpins approaches such as Soft Systems Methodology and Strategic Options Development and Analysis. In this paradigm, empirical data is used to support theory-building, rather than theory-testing. Soft OR models engage with the beliefs and understandings of those involved with real-world problems, using well-defined processes and problem-structuring methods. Insights are gained in this context from case-study research that may also be applicable to other organizations. Until the 1980s, this paradigm was almost non-existent in British OR and is still neglected in American OR. However, multiple paradigms in OR are currently well represented in the British Journal of the Operational Research Society and are supported by the major UK Research Councils.

Clarification on the underlying paradigmatic approach has two benefits. Firstly, it helps to clarify the purpose of the research. Is the research descriptive or normative? If it is normative, are the objectives taken as given (as is usually the case in positivistic research) or are they the subject of investigation (as is sometimes the case in phenomenological research)? Is the purpose to reach generalizable conclusions (positivistic) or to identify new insights that pertain to at least one case and may apply to others (phenomenological)? Secondly, it helps to clarify the standards by which the research should be judged. Phenomenological research should not be judged by the standards of positivistic research. Rather, it should be assessed according to the recoverability of the research process and the depth of analysis.

Operational Research has gained much over the last twenty years by the sustained development of ‘soft’ approaches, to complement the more traditional ‘hard’ approaches. The richness of real world problems is better addressed from a range of perspectives, some of which may be better suited to earlier or later stages of an investigation. Consequently, multi-paradigmatic approaches have led to a broader range of issues being addressed, and to work over the last decade on the interactions between ‘hard’ and ‘soft’ OR methodologies. Many OR applications are in the OM field and, in our view, OM can only benefit from paradigmatic diversity. Currently, Operations Management is at risk of losing the phenomenological paradigm. This paradigm needs to be regained.

Craig Gustin

The authors cite the Journal of Operations Management and Journal of Supply Chain Management as premier supply chain management (SCM) journals. There are other publications that also should be included, such as the Journal of Business Logistics, International Journal of Physical Distribution and Logistics Management, International Journal of Logistics Management, and the Journal of Transportation Management. The Supply Chain Management Review is another source of SCM articles that often contains case studies and results of survey research. In addition, there are numerous trade periodicals widely read by SCM professionals that regularly publish the findings of industry surveys and case studies.

Industry analysts are another source of SCM research. Companies such as the Gartner Group, Forrester Research, ARC Advisory Group, AMR Research and Aberdeen Group regularly conduct research in the areas of SCM and information technology. Their reports are generally based on primary data obtained from survey research of SCM practitioners, system/software vendors and third-party service providers.

Several professional organizations also provide a forum for SCM education through annual conferences, local chapter activities and publications. Example organizations are the Council of Supply Chain Management Professionals (CSCMP), American Production and Inventory Control Society (APICS), Institute of Supply Management (ISM), and the Supply Chain Council. The educational materials offered by these organizations typically focus on the day-to-day activities of SCM professionals and the practical applications of technology, with little or no mention of econometric models.

In summary, while there may be some indications of a trend toward greater use of econometric-type modeling in SCM journals, I believe “less-sophisticated” methodologies such as survey research, case studies and regression analysis will continue to be used for the foreseeable future. Thus, we may not even be approaching a “tipping point.”

Prakash J. Singh

Carter and his fellow essayists invoke Thomas Kuhn’s evocative model of ‘paradigm change’ to posit the types of changes in research that are likely to occur in the field of SCM in the foreseeable future. In line with Kuhn’s idea of revolutionary change, Carter et al. suggest that the existing research methodologies that are essentially based on interrogating primary empirical data will be replaced by other forms that rely on econometric and mathematical modelling. Carter et al. find the notion of the replacement of one paradigm by another to be lamentable, if not dangerous, for how the field will develop into the future. As opposed to paradigm change, they call for paradigm expansion where pluralism in methodologies would be a highly desirable feature. But, as traditionally interpreted, the Kuhnian model does not allow for this. For this reason, I believe that this model is inadequate for explaining how knowledge has, and is likely to develop in the SCM area. Further, a better model is needed. Fortunately, one such model does exists, this being Imre Lakatos’ (1974) concept of ‘research program’. In this piece, I will briefly explain what the Lakatosian research program is, followed by how this applies to conceptualising SCM research, and conclude with the benefits of taking this approach. In doing this, I am taking an altogether different tack - while most other commentators have worked largely from within Kuhn’s model to show how the ideas in the essay could be flawed or limited, I am proposing that the model itself should be rejected, and a better one be used in its place.

For quite some time, philosophers of science have debated how knowledge development and scientific change takes place. Karl Popper’s (1959) and Kuhn’s (1962) ideas have been popular. However, as Lakatos states, the Popperian tradition of theory development through falsification and the Kuhnian concept of revolutionary paradigm change provide rather limited and constrained explanations for scientific change and knowledge growth. Lakatos contends that researchers study a phenomenon using many different theories and proposes that these theories operate within what he describes as a ‘research program’. Within the research program, there is a ‘hard core’ of theories which are positively appraised because they supply predictive gratification about a phenomenon. Beyond the hard core, there are other theories located in what he calls the ‘protection belt’ that produce novel shifts in knowledge. Lakatos’ model thus distinguishes between a hard core and a protection belt. The research program guides researchers as to which paths to research. This approach ensures that researchers are aware of where their research is being directed, the hard core or the protection belt.

Looking at the state of research in the SCM field from an ontological and epistemological perspective, I would describe it as a multi-disciplinary, meta-theoretic research area that is still largely in a pre-paradigmatic state. Superimposed on this is a set of dominant characteristics, the most notable of these being: a reliance on the manufacturing industry for empirical and analytical illustration; predominance of economics based theoretical grounding; and, strong positivist paradigmatic stances in the research methods that are employed. These dominant characteristics appear to have repressed the emergence of a plurality of ideas in terms of how the area is conceptualized, theoretically described and researched, making the development of the field a narrowly concentrated one.

Applying the Lakatosian research program lens, the hard core is based in the ‘manufacturing – economic theory - positivist’ nexus, while numerous other activities including those that Carter and his colleagues class as being part of the Kuhnian paradigm change are coalescing increasingly within the protection belt. Further, while the hard core is well defined, this is not the case for the protection belt. Given the nature of SCM as described earlier, the Lakatosian research program appears to aptly describe how research is presently conducted in the area. The extra advantage in adopting the Lakatosian approach is that it provides a logical framework to conduct research in the manner in which it presently appears to be evolving, thereby helping reduce the shortcomings in research identified previously in a non-revolutionary manner. The Lakatosian approach provides a way of staying connected with the historical core of SCM in both content and research methods while, at the same time, also engaging with new areas and methods. It thus provides a way of potentially overcoming existing barriers and assisting the speed of development of the SCM body of knowledge by encouraging and fostering new approaches, enabling discussion and exchange with a wider community of researchers and practitioners whose views may be presently marginalized.

If one applies the Kuhnian paradigm change concept to a body of knowledge that is organised in Lakatosian form, then change would involve replacing the current hard core with elements that may or may not be present in the protection belt. Carter et al. warn of the dangers of this type of change for SCM research, and suggest that the hardcore (and indeed, the protection belt) needs expansion, not replacement. The Lakatosian model is about evolutionary change, pluralism and inclusiveness. Therefore, with all due respect, I believe that the ideas that Carter and his colleagues are attempting to express in their essay is best conceptualised by invoking the Lakatosian model, and not the Kuhnian model.

Kuhn, T., 1962. The Structure of Scientific Revolutions, University of Chicago Press, Chicago, IL.
Lakatos, I., 1974. "Falsification and the methodology of scientific research programs" in Criticism and the Growth of Knowledge, Lakatos, I. and Musgrave, A. (Eds.), Cambridge University Press, Cambridge, UK, pp. 91-196.
Popper, K., 1959. The Logic of Scientific Discovery, Hutchinson, London.

Craig R. Carter, Nada Sanders, and Yan Dong

“On Paradigms, Revolutions, and Tipping Points: A Rejoinder”

We were quite pleased to learn that our comment, entitled, “Paradigms, Revolutions, and Tipping Points: The Need for Using Multiple Methodologies within the Field of Supply Chain Management” has generated so much interest within the supply chain management community of scholars. This is evidenced by the large number of visitors to the Journal of Operations Management’s Operations and Supply Management Forum website where our comment first appeared, along with the number of researchers who took the time to carefully craft a response to our comment. Some researchers agreed with us and built upon our initial message, while others vehemently disagreed with at least some part of our original essay. However, all of the responses seemed to concur with the message that our discipline needs to ensure the use of multiple research methods and data sources within our field, and avoid the domination of a single data source and method.

Additionally, the responses afforded us the opportunity to appreciate where we were unclear in our initial essay and where we might extend our original thoughts. To succinctly summarize the message of our original essay: 1) surveys, archival data, and case studies are data sources, and structural equation modeling (SEM), econometric modeling, qualitative data analysis, and regression are analysis techniques or statistical tools to analyze those data, 2) we encourage the use of multiple data sources and analysis techniques, and welcome the use of archival data and econometric tools to analyze that data, and 3) our fear, however, is that the pendulum may swing too far, such that the use of surveys and case studies are unduly excluded in favor of these “new,” “sexy,” and “sophisticated” tools.

While it seems as though a few of the respondents may have misinterpreted this message, there is generally strong agreement that we need to have multiple data sources and statistical techniques to analyze those data. Correspondingly, we were pleased to see that respondents agreed that we should avoid the use of a single data source and statistical technique, whether it be SEM to analyze survey data (as noted by Holmström) which has been true in the past in our discipline, or econometrics to analyze archival data.

The major additional criticisms of our original essay that are found in the responses center upon the follow issues: 1) we have not correctly employed the term “paradigm”, as defined and conceptualized by Kuhn (1962, 1970), 2), we point to and perhaps criticize the field of marketing, when this may not be an appropriate analogy, and 3) we have only scratched the surface of a phenomenon that is extremely important to our field, and we need to go much further in explaining what is occurring along the lines of methodological changes in the discipline. A fourth issue, which is related to the third, was addressed in private conversations with colleagues who have read our original essay: 4) we should explain and expand upon additional possible reasons why the use of survey research has become less common. We address each of these issues below.

We believe that the use of the term paradigm is technically correct. As noted in our comment, Kuhn (1970, 1996) defines a paradigm as consisting, in part, of a consensus concerning research methods within a field of inquiry. However, we believe that Singh’s suggestion to employ Lakatos’ (1974) model of a research program to allow for our advocacy of open acceptance and encouragement of multiple data sources and analysis techniques is quite appropriate, and may be a better means to view the shift in sources and techniques that we are seeing in our field.

Our essay was not intended in any way to be an indictment against the field of marketing, nor was it our intent to suggest that our field is in any way subordinated to marketing. Rather, to paraphrase our original comment, we chose the field of marketing as a point of comparison because marketing as a field examines issues of substance that somewhat overlap with the field of supply chain management. In addition, marketing has used similar analysis techniques and data sources in the past, as are currently being used in supply chain management. Finally, marketing is the field that has most recently experienced a surge in use of data sources and statistical tools from the economics discipline. As such, marketing, with a history of academic rigor, offers an excellent point of comparison.

One colleague suggested that our original treatment of the field of marketing was too cursory, and that we should perform a much more in-depth examination of the extant literature in that field. However, the purpose of our essay was not to perform an in-depth analysis of the literature in another field. Instead, we next briefly qualify the analysis of the marketing literature data presented in our original comment, and present additional evidence from literature in our own field, which appeared in a recent special issue of Manufacturing and Service Operations Management (MSOM) centering on empirical research in operations management.

A further examination of the data from our original essay – specifically, articles published in the Journal of Marketing (JM) in 1997 and 2007 – suggests that the proportion of channels related articles decreased between 1997 and 2007. However, the reduction in the proportion of channels articles using survey data in 1997 versus 2007 is quite similar to the overall reduction in the use of surveys across all articles appearing in JM between these two time periods. Further, our interviews with senior marketing faculty corroborated these findings. A recent special issue in MSOM, focusing on the application of empirical research to the field of operations management, serves as one data point to support the assertion that we are beginning to see a trend toward the use of archival data and econometric analysis within our field. As noted by Roth (2007, p. 362), “the dominant approach found in this (special) issue was the application of various types of econometric analyses to secondary data.”

There are several likely reasons for the decreased use of surveys in fields outside of supply chain management, and we will most probably see a diminished use in our own field for these same reasons. First, we have seen a continued lowering of response rates over time. Researchers in our field tend to use many of the same databases and membership lists of trade organizations such as the Institute for Supply Management and Council of Supply Chain Management Professionals. Potential respondents, who are members of these organizations, have experienced ‘survey fatigue’, to the point where it is becoming increasingly difficult to generate a sufficient sample size of responses. At the same time, we are seeing an increasing availability of archival databases that are relevant to our own field from marketing research companies and research centers, as well as the application of traditional secondary data (e.g., COMPUSTAT) to empirical supply chain management research (Roth 2007). Second, the use of single informants is increasingly being viewed as a limitation of traditional survey research. While we disagree that the use of a single, key informant is categorically invalid in survey research, there are certainly research questions and hypothesized relationships that do require the use of multiple informants in order to provide a valid treatment. Finally, reviewers, associate editors, and other gatekeepers in the review process are increasingly questioning whether the use of a single survey instrument to measure both independent and dependent variables can avoid the issue of common methods bias, despite various tests that researchers conduct to assess this bias. To the extent that researchers address these issues (e.g., through the use of new sampling frames, multiple informants where needed, and multiple data sources such as surveys and archival data, surveys and interviews, or surveys of multiple informants), this increased rigor should help to improve the quality of research in our field. We never suggested that the decreased dominance of survey research is something to be avoided, but rather that we should avert the emergence of a new data source and analysis as a dominant paradigm within our field, to the biased exclusion of other valid approaches. Again, we need the balanced use of multiple methods in our field.

This advocacy should not be viewed as a threat to survey researchers, but rather as a strengthening and maturation of our field. Additionally, our discussion does not suggest that all survey research must use multiple informants and different data sources to measure the independent and dependent variables. For instance, there are certainly subject areas for which a single, key informant can knowledgeably provide valid data. Conversely, a survey which attempted to assess external conditions in both the upstream and downstream supply chains would probably require answers from at least two informants: one who spans the boundary of the focal firm and its upstream supply chain and another respondent who interacts with customers and other members of the downstream supply chain. This discussion does suggest, however, that researchers must carefully consider and address non-response bias, the key informant issue, and common methods bias. These issues have always been present. Thus, the bar is not being changed, but rather raised as our field continues to advance along the lines of both theory and methods.

Finally, is the field of supply chain management at a methodological tipping point? The purpose of our essay was not to forecast such a change per se, but rather to make researchers in the discipline aware of this issue, so that we can have an open and constructive forum for dialogue. If we are not at such a tipping point, then there is no better time to discuss these issues than now. Further, we disagree with Keating’s suggestion that if we are at a tipping point, there is little that can be done to stop such a change in our field. As noted by Roth (2007, p. 355-357), thought leaders in our field do have the ability to change the tide to allow for multiple methods and approaches.

In conclusion, we thank our colleagues for their responses and contribution to this dialogue of change in data sources and analysis techniques in our field. Not only did their responses assure us that our original comment provides a useful contribution to the field of supply chain management and a starting point for discussion, but they have also afforded us the opportunity to further develop this dialogue. For this we are indebted.


References

Holmström, J. (2008), OSM website, http://jom.typepad.com/my_weblog/2008/05/comment-here-on.html.

Keating, B. (2008), OSM website, http://jom.typepad.com/my_weblog/2008/05/comment-here-on.html.

Kuhn, T., 1962. The Structure of Scientific Revolutions, University of Chicago Press, Chicago, IL.

Kuhn, T.S., 1970. The structure of scientific revolutions, University of Chicago Press, Chicago, IL.

Lakatos, I., 1974. "Falsification and the methodology of scientific research programs" in Criticism and the Growth of Knowledge, Lakatos, I. and Musgrave, A. (Eds.), Cambridge University Press, Cambridge, UK, pp. 91-196.

Roth, A.V. (2007). Applications of empirical science in manufacturing and service operations. Manufacturing and Service Operations Management 9 (4), 353-367.

Singh, P.J. (2008), OSM website, http://jom.typepad.com/my_weblog/2008/05/comment-here-on.html.

Kevin Dooley

This is a really interesting thread to read, both the essay and comments. Dare I say, some dialogue actually took place? Thank you all for your insights.

I am somewhat surprised though at the few number of overall comments on the site. This mode of communication is legitimate and exciting and our large part of our future, I wish more would embrace it.

The comments to this entry are closed.

Blog powered by Typepad