Research Methods in Corporate Reputation

Secondary research is the synthesis of data and information that others collected for other purposes to answer research questions. Secondary research is also called desk research because it can be analyzed at a researcher’s desk rather than out in the field. Secondary research has a number of advantages. Primarily, it is a quick, efficient, and inexpensive way to learn from multiple and large studies. However, it does have its limitations and disadvantages. Because secondary research was collected for other research purposes, the research may not be specific enough to one’s primary topic of interest, there is often little information about the research methodology and data collection, and the research may not have sufficient depth. In all cases, one must also be concerned about source credibility and whether there are conflicting or alternative viewpoints on the subject matter.

Numerous types of secondary research are important for corporate reputation research. Some of these include syndicated audience rating measurement systems, like Nielsen or Arbitron, and syndicated multimedia audience measurement systems, like the Simmons Market Research Bureau (SMRB), Mediamark Research Inc. (MRI), and Scarborough. Academic journals are also a form of secondary research. Academic journals that publish a number of studies on corporate reputation include Administrative Science Quarterly, Academy of Management Journal, Organizational Science, Public Relations Review, Journal of Communication, and Journal of the Academy of Marketing Science. There are also a number of trade publications, such as PR Week, and industry association publications, such as Communication World by the International Association of Business Communicators and Tactics by Public Relations Society of America, that publish industry research related to corporate reputation. Other sources include syndicated trend and consumer behavior data available from firms such as Yankelovich, Ketchum, and Nielsen, such as the Harris Reputation Quotient. There are also reports from government agencies such as the U.S. Census Bureau and the U.S. Department of Labor. Finally, there are blog and social media analyses available, such as Nielsen BuzzMetrics.

Secondary research serves numerous purposes in corporate reputation research. It is good to use it when the source offers thought leadership or the industry standard, when broad background information about a subject area needs to be known, when there are known facts that are needed, and to avoid duplicating the mistakes and errors that other people have made and to offer suggestions or feedback on how to correct them.

Archival Research

Archival research includes materials found on-site in company archives or in the records of individuals or other institutions. This can include formal reports, informal reports, digital files such as presentation slides, annual reports, and even news articles and press clippings. One study using archival data found that a firm’s environmental ratings from Franklin Research increased return on assets.

Syndicated Research

One special type of secondary research is syndicated research. Syndicated research differs from other types of research in that it is not often singly sponsored. A number of organizations can pool their resources together to fund research in ways that would not be possible individually. Syndicated research is the backbone of much of the media industry. MRI helps advertising agencies in defining target audiences and understanding the media habits of the larger public for their clients. SMRB operates similarly, but it provides useful psychographic information that can be cross-referenced against media and product categories. Scarborough provides research similar to that of MRI and SMRB, but it focuses more on local markets whereas MRI and SMRB focus more on national markets. Syndicated research helps organizations not only by telling them how many people were exposed to their messages through various media outlets but also by identifying specific demographic groups within the larger audience. Thus, syndicated research also helps by compiling audience profiles of various media channels. In addition, it can help develop target audience media profiles and audience profiles. But knowing about the past is often not enough; sometimes data are gathered that provide clues and insights on trends.

Fortune’s “Most Admired Companies” listings and other rankings (also called league tables) are one common example of syndicated research. The Fortune most-admired-companies study has existed for 30 years and is still running annually. Denis Bromley identifies a number of limitations for rankings. First, reputational attributes in these studies are abstract and are not operationally defined. In addition, the attributes are not equally important from an objective or subjective point of view. There are questions about the nature of the sample, how heterogeneous or homogeneous it is, and whether it constitutes a “collective representation.” Then, there are also questions about the timing and the circumstances of the surveys’ data collection efforts, which often go undefined.

Historical and Legal Research

Many of the other methods in this entry can be used for historical research. Historic resources refer to persons or items studied that existed during the time period under examination, such as original documents of people who lived through the event or the era studied. Historical items, on the other hand, include other secondary research sources that are about the time period under examination but were written by those who were not present during the time. Oral histories are a special type of personal interview that focuses on people who lived through particular times or who have particular insights on those times, to make the information available for future researchers. The participants share their experiences, perceptions, and opinions of the time, but they do not speak for others tied to the event studied. These interviews are recorded and transcribed. As with other research methods, the credibility of the sources is tantamount; this is especially the case with oral histories used for historical research. Attention must be given to the sources of the information, their backgrounds and vested interests, and the various influences that may shape their views or production.

Legal research is concerned with finding and establishing precedence for past legal actions and decisions. Legal research is particularly specialized and is best undertaken by researchers who have some form of training, background, and experience in the law. Legal research relies on a number of primary sources, including the laws themselves and precedents from court decisions. There are a number of legal journals, law digests, law books, and legal reports that serve as secondary research. Legal research, like historical research, is an underutilized resource in corporate reputation research but one of particular importance. Quasi-judicial case studies are one example.

Quasi-judicial case studies adopt a “legal science” approach and apply it to social science issues that are not amenable to a “natural science” approach. They use the substantive logic procedure developed by Stephen Toulmin, Richard Rieke, and Allen Janik, enabling researchers to construct or dissect complex arguments using simple diagrams. Applying the quasi-judicial case method to a series of cases enables researchers to establish and argue for a pattern. The “facts” presented as evidence do not speak for themselves. Researchers must offer a theory or an argument, making their assumptions explicit as well as their concepts, rules, and any generalizations applied to impose meaning onto the available evidence. Thus, the method relies on rhetoric and argumentation, rather than establishing causality through more empirical methods. Quasi-judicial cases place an emphasis on eliminating unsubstantiated, unverified sources and evidence in a journalistic/legalistic sense. Quasi-judicial case studies do more than offer narratives. They move beyond storytelling and narration to argumentation.

A body of research employing historical and legal research methods is just beginning to emerge. A criticism of existing historical research that implicates the reputation of organizations is the taken-for-granted nature of reputation and the tendency to treat it as uncomplicated. Moreover, the tendency to treat as facts reputational assessments, claims, and constructions in historical research also presents a complication for historical research. The use of historical methods in the case of corporate reputation is best when combined with theory and the rigorous, spelled-out processes that accompany other research methods.

Qualitative Research

Qualitative research is designed to provide rich, contextual insights and deep descriptions. Quantitative research often relies on qualitative research during an exploratory stage, such as when one is getting familiar with the scope, dimensions, and parameters of a particular problem area. But qualitative research is often conducted and used in its own right. Qualitative research is helpful for providing deeper insights into the nature of concepts and ideas, their meanings, and their value. There are a number of qualitative research methods used in corporate reputation research. This entry covers personal interviews, focus groups, projective techniques, and framing studies.

Personal Interviews

Personal interviews are interviews that take place between people. But this does not mean necessarily that the interviews are face-to-face. They can also be conducted over the phone or through the Internet. Moreover, the interviews do not necessarily have to be one-on-one; they could also be in small groups. Interviews are to be used when one needs to gather information on respondents’ thoughts, ideas, beliefs, opinions, and expertise.

In-depth personal interviews take longer to schedule, longer to conduct, and longer to transcribe or convert the findings into a usable form than other types of interviews. Nevertheless, they provide tremendous insights because they are more interactive, allowing the researcher to go off the topic if new insights or directions emerge. The more common personal interview technique is the in-depth interview. In-depth interviews are good at uncovering insights not available through other methods, particularly from individuals whose insights are unique or exclusive to them. Dyadic interviews involve two people, where the focus is on the relationship or on a particular topic. The Delphi technique allows insights to emerge from experts and thought leaders on a particular topic area. In corporate reputation research, personal interviews have been used to identify organizations, issues, attributes, or topics worthy of deeper exploration. They have also been used to answer questions concerning “why” or “how.”

Focus Groups

The focus group could be considered a particular type of personal interview, but of a group of people with shared characteristics, often with a group size of 8 to 10 persons. The researcher uses a trained moderator to lead the discussion following a protocol. The focus group has the advantage of allowing a client to observe the focus group behind a one-way mirror or, if the interview is recorded, by watching the recording. The protocol is loosely structured so that participants can insert their opinions. The focus group relies on group dynamics to generate insights, but it is also sensitive enough to involve all the participants and ensure that none are left out. Questions start off as general, moving to more specific questions during the process. A focus group can last between one and two hours. Focus groups have the advantage of allowing the findings to be extracted relatively quickly from a written summary of the proceedings. Focus groups are usually conducted at the beginning stages of research. No organization bases its decisions solely on the focus group findings, but instead, organizations use the findings to develop more sophisticated methods and questions for generating insights from larger groups via surveys, for instance.

Projective Techniques

Qualitative research relies on the use of projective techniques, which allow for participants to become more engaged in the research. Some common techniques are word associations, storytelling, and card-sorting exercises. In card-sorting exercises, a series of ideas are placed on index cards, and respondents are asked to sort the deck into piles (often in the shape of a forced bell curve) and then explain the story about how the cards are sorted to tell their particular story of how they fit together and represent their view. Other forms of projective techniques include usability tests, which allow respondents to work with a company’s product to show how they use it or have difficulty doing so. Media monitoring of stakeholders and social media such as blogs (and their comments), Facebook, and other platforms also serve as projective techniques that organizations use.

One particular type of projective technique used in corporate reputation studies is the free-description method. This entails respondents listing a series of attributes, based on their personal interests and experiences. The respondents would be drawn from selected interest groups, such as a group of investors, consumers, or supply chain members. The more frequently used attributes could be used to construct a league table, or a set of rankings. Differences in their frequency of occurrence could reveal their relative importance.

Framing Studies

Content analysis is primarily thought of as a quantitative technique, but in the early stages it is often qualitative. The use of framing methodology often remains at this level. Framing studies are concerned with how meanings are constructed both within and across texts. Framing studies are useful for identifying the factors that shape the production of texts and meanings, the dimensions of the issue, or the frame within a text or texts, and for linking the texts to particular qualitative outcomes. In addition, framing studies are useful for conducting audience analysis by working with audiences to uncover the frames they use in the interpretation or evaluation of texts.

Case Studies

As a research method, a case study focuses on one particular case that offers a holistic description of a complex, real-world social phenomenon with something to teach and learn and generally offers a rigorous and fair presentation of the data. A case study used as a secondary source requires assessing the original case author’s intent because as a method for teaching, the need for rigor and fair presentation are not primary concerns, as much as providing a platform for debate and discussion. Case studies have a long history in business research. Case studies are particularly helpful in studying companies whose attributes, ratings, or rankings deviate significantly from the average of their peers; companies where there appears to be a mismatch between their reputation and their relative measure of performance on which their reputation is based; or those facing particular risks or crises.

A key concern is the bracket of time that identifies the beginning and ending time period of the case. Most case studies focus on one particular case. Kimberly Elsbach’s 2002 case study of the Apple 4 illustrates the profound effect that the bracketing of the time period has on the case. She identifies and analyzes four distinct reputation management strategies employed over a matter of months and underscores the importance of considering history. Many social scientists believe that case studies should only be employed as an exploratory research method and that they cannot be used for testing propositions or hypotheses. Some researchers are beginning to challenge this assertion. Current efforts are under way to identify a series of research designs for answering this challenge.

Quantitative Research

Quantitative research is designed to convert data to numbers, making them amenable to measurement and evaluation using statistics. The primary methods in quantitative analysis include surveys, content analysis, and experiments.

Surveys

A survey is a detailed method to gather data on attitudes, impressions, opinions, satisfaction level, and so on, by polling a subsection of a population. Surveys are designed to collect larger amounts of information from a larger number of people coming from a selected population of interest than a focus group. Each question asked is designed to elicit information on topics of interest. The questions may be self-administered or conducted in person, on-site, via phone, or through the Internet, using computer-assisted methods such as e-mail or directly on the web. Surveys can be cross-sectional or longitudinal. Cross-sectional surveys involve a sample of respondents collected at one time; longitudinal surveys involve a sample answering the same questions at different junctures over time.

A mail survey is a self-administered survey mailed to a sample of the population. After the respondent has completed the questionnaire, the survey is mailed back to the researcher. Researchers are not able to ask questions in addition to those on the survey. The survey must have directions that are clear and simple to follow, with questions that are clear and worded simply and a simple, clean design, all of which help improve the response rate. Either respondents are asked in advance to participate in the study or their solicitation is included with the survey. Mail surveys involve the additional step of feeding the respondents’ answers into the computer.

Electronic surveys by e-mail and by website have become more popular over the past 10 years for a number of reasons. They have a higher return rate than mail surveys, but they are not able to reach people who do not have access to computers or e-mail addresses. They have the benefit of reaching a larger number of people within minutes of distribution, they are less expensive, and they also allow for additional follow-up in ways that are faster and less expensive. It is usually easier and faster to compile data with electronic surveys than mail surveys. E-mail surveys have many of the same limitations as mail surveys. Respondents may not follow the directions precisely, or depending on the software, they could provide two answers to the same question or skip questions entirely. Web-based surveys have the added benefit of preventing respondents from going further in the survey until they answer each question. Electronic surveys have the disadvantage of additional concerns of privacy, confidentiality, and anonymity created by using e-mail and the Internet for distribution and compilation.

Content Analysis

Content analysis is often employed for the systematic analysis of messages and characteristics within texts. Content analysis involves the analysis of manifest or latent characteristics found in texts or other types of content, where inferences that are valid and replicable can be made. These characteristics may focus on an attribute found within the text or on the relationship between two or more attributes within the text. Manifest content refers to content within the text that any observer can see if he or she knows what to look for. Latent content can be patterned or projective. Latent patterned content refers to content that emerges due to a pattern of repetition or association that occurs in the text. Projective content relies on human judgment, expertise, or a definition/rule book that specifies the criteria by which to identify the content. Manifest content can be observed by computers or by humans, but it is more cost-effective to use computers. On the other hand, projective content will almost always rely on human coders to identify it.

Content analysis usually involves the coding of objects (e.g., issues or company names), attributes (e.g., firm attributes, demographic characteristics, or company actions), or associations or relationships (e.g., firms’ linkages to people, other firms, news values, or other organizations). The most significant advance in content analysis concerns the employment of network analysis to look at the interlinkages between a set of objects, attributes, and issues.

Content analysis is often combined with other methods, such as archival research, including the analysis of historical public opinion polls. Through such triangulation, scholars have been able to establish media influences on various dimensions of corporate performance and various company attributes and attributes from the larger environment that shape the news coverage of organizations.

Q-Sort Method

The Q-sort method is rooted in Q methodology, often called a technique of inverted factor analysis. Q methodology was developed as an alternative measurement technique to the existing scales in psychology. The Q-sort method can be used in any situation in which subjectivity is to be studied. This includes impression and attitude measurement. The Q-sort method involves the rank ordering of a set of statements into a forced bell curve, a near-normal distribution, ranging from agree to disagree, with unclear, ambiguous, or ambivalent statements placed near the center of the curve. The set of statements should contain all of the statements that characterize the discussion of a topic, at best in mutually exclusive and exhaustive terms. These statements may be compiled through archival research, literature reviews, personal interviews, focus groups, or content analysis.

The Q-sort method is often called an inverted factor analysis because the data matrix is inverted. In this method, the respondents are the variables, and the items in the set of statements are the cases. Thus, it is the respondents who are correlated, instead of the items. The Q-sort method provides a rich way of helping organizations uncover multiple reputations at work from the perspective of their stakeholders. One example of the Q-sort method is the Organizational Image, Identity, and Issues Audit, which relies on a comprehensive set of issues, identity, and reputation claims from the point of view of each of a focal organization’s stakeholders. Then, a subset of stakeholders from each group sorts the produced set of statements into the forced curve from their perspective. The Q-sort method combines well with qualitative methods such as in-depth interviewing and surveys. For instance, with surveys, the Q-sort method provides a way to pretest the wording and selection of the statements to be included in surveys. Surveys supplement the Q-sort method by providing an indication of how large the sample size is from within a population, who might subscribe to a particular “Q factor” produced from a Q-sort study.

Experiments

An experiment is a research method that enables researchers to examine causal relationships, more than any other research method. The goal of experiments is to test hypotheses, or the assumptions derived from a more or less formal theoretical explanation of a phenomenon. Three criteria have to be met for an experiment. First, changes in one variable (the cause, or the independent variable) must cause changes in the other (the effect, or the dependent variable). Second, the effect must follow the cause in time or sequence. And third, no third variable must influence the relationship.

The basic elements of an experiment are the following. First, the independent variable must be manipulated by the experimenter. If it is not easy or possible to manipulate, then the independent variable may be measured. The independent variable may take on two or more values, which leads to different treatment conditions. And the independent variable must cause change in a dependent variable. Second, the critical feature of the dependent variable is that the dependent variable is dependent on the independent variable for its value. Then, the dependent variable is measured by the experimental researcher. Third, nuisance variables are variables other than the independent variable that cause changes in the dependent variable. Nuisance variables have to be eliminated or controlled through randomization, where subjects are randomly assigned to either the experimental condition or the control group.

There are three basic ways by which experiments can be differentiated. First, they can be differentiated by the presence or absence of randomization. A true experiment involves randomization, while a quasi experiment must often measure the value of the independent variable and form groups post hoc and then control for other variables that may affect the relationship between the independent and the dependent variables. Second, experiments may be differentiated by their location—a laboratory experiment or a field experiment. Third, they may be differentiated by the number of variables. An experiment with multiple independent variables is a multifactorial experiment; an experiment with multiple dependent variables is a multivariate experiment.

Validity, both internal and external, is a central concern for experiments. Internal validity refers to how well the study was run and how confidently the conclusion can be drawn that the change in the dependent variable was produced solely by the independent variable. External validity refers to the extent to which a study’s results can be generalized or applied to other people or settings.

The strength of experiments is that they are the most rigorous of all research methods because they are standardized and show cause and effect. The downside of experiments is that they take a long time to prepare, involving a considerable amount of precision and meticulousness, and because of their artificial nature, they lack a degree of realism. In fact, many scholars question whether corporate reputation is something that can (or should) be studied at all. On the one hand, it is questionable whether a reputation can truly be constructed for a fictitious company, and on the other, reputational manipulation through an experiment has the potential to pose real harm to an organization or perhaps its stakeholders.

Often, it is difficult, impossible, or unethical to randomly assign subjects to the different treatment conditions—for example, finding out the effect of corporate reputation on the organizational stress levels of employees. One solution may be to conduct a quasi experiment and measure the value of the independent variable, forming groups post hoc. However, one problem is that there is no control for possible nuisance variables. A partial solution is to hold variables that are thought to have an influencing effect constant in the analysis, using a procedure available in most statistical software programs. However, even here, it is often difficult to judge which variables these are and impossible to control for everything. Nevertheless, quasi experimentation is often the only valid and possible solution for testing a hypothesis in corporate reputation studies.

Laboratory experiments are often criticized for their artificial nature and lack of realism. Many respected social scientists, including Muzafer Sherif, Edgar Schein, and Leon Festinger, relied on qualitative data from real-world contexts to supplement and ground their work in theory building.

Online and Mobile Research

Online research has also provided numerous opportunities for doing research that used to be done off-line, if at all. Today, panel studies, focus groups, and in-depth interviews are all possible via online research. Organizations can use computers, mobile phones, and personal digital assistants for researching content via blogs, forums, websites, social networking sites, online videos, online games, and virtual reality worlds such as Second Life. There are also sources available online that are not available off-line, such as websites that track people’s online behavior, including web searches, online buying behavior, and participation and advocacy via social media platforms. Organizations can also use online research to measure and evaluate online campaigns. Website optimization, for instance, provides a way of attracting attention to company websites and to see who visits them. Social media analyses of platforms such as Facebook and Twitter also provide numerous insights for companies.