Sentiment analysis (SA) has grown considerably in organizational science research over the past two decades, particularly in the last few years. While enthusiasm for integrating advanced natural language processing algorithms is encouraging, authors are not reaping the benefits of such tools fully. Our systematic review of SA application in the organizational sciences suggests that authors struggle to appreciate all of the decisions that are inherent to SA, the choices that are available at each decision point, and the consequences of each choice. To address this gap, we use a working example to illustrate four critical decision points authors confront when conducting SA, and the subsequent impact different choices can have on one's conclusion. Decision points include selecting the SA method, computing a sentiment score, preprocessing the data, and using an appropriate level of analysis. We conclude with a framework outlining five dimensions (e.g., accuracy, interpretability, computational cost) to guide the selection of an SA approach based on study goals and needs, along with seven recommendations to authors wishing to apply SA.

Careless responses by survey participants threaten data quality and lead to misleading substantive conclusions that result in theory and practice derailments. Prior research developed valuable precautionary and post-hoc approaches to detect certain types of careless responding. However, existing approaches fail to detect certain repeated response patterns, such as diagonal-lining and alternating responses. Moreover, some existing approaches risk falsely flagging careful response patterns. To address these challenges, we developed a methodological advancement based on first-order Markov chains called Lazy Respondents (Laz.R) that relies on predicting careless responses based on prior responses. We analyzed two large datasets and conducted an experimental study to compare careless responding indices to Laz.R and provide evidence that its use improves validity. To facilitate the use of Laz.R, we describe a procedure for establishing sample-specific cutoff values for careless respondents using the “kneedle algorithm” and make an R Shiny application available to produce all calculations. We expect that using Laz.R in combination with other approaches will help mitigate the threat of careless responses and improve the accuracy of substantive conclusions in future research.

In graded paired comparisons (GPCs), two items are compared using a multipoint rating scale. GPCs are expected to reduce faking compared with Likert-type scales and to produce more reliable, less ipsative trait scores than traditional binary forced-choice formats. To investigate the statistical properties of GPCs, we simulated 960 conditions in which we varied six independent factors and additionally implemented conditions with algorithmically optimized item combinations. Using Thurstonian IRT models, good reliabilities and low ipsativity of trait score estimates were achieved for questionnaires with 50% unequally keyed item pairs or equally keyed item pairs with an optimized combination of loadings. However, in conditions with 20% unequally keyed item pairs and equally keyed conditions without optimization, reliabilities were lower with evidence of ipsativity. Overall, more response categories led to higher reliabilities and nearly fully normative trait scores. In an empirical example, we demonstrate the identified mechanisms under both honest and faking conditions and study the effects of social desirability matching on reliability. In sum, our studies inform about the psychometric properties of GPCs under different conditions and make specific recommendations for improving these properties.

Qualitative meta-studies (QMS) have emerged as a promising methodology for synthesizing qualitative research within organization and management studies. However, despite considerable progress, increasingly fragmented applications of QMS impede the advancement of the methodology. To address this issue, we review and analyze the expanding body of QMS in organization and management studies. We propose a framework that encompasses the core decisions and methodological choices in the formal QMS protocol as well as the reflective—yet often implicit—meta-practices essential for deriving meaningful results from QMS. Based on our analysis, we develop two guidelines to help researchers reflectively align formal methodological choices with the intended purpose of the QMS, which can be either confirmatory or exploratory.

Organizational research increasingly relies on online review data to gauge perceived valuation and reputation of organizations and products. Online review platforms typically collect ordinal ratings (e.g., 1 to 5 stars); however, researchers often treat them as a cardinal data, calculating aggregate statistics such as the average, the median, or the variance of ratings. In calculating these statistics, ratings are implicitly assumed to be equidistant. We test whether star ratings are equidistant using reviews from two large-scale online review platforms: Amazon.com and Yelp.com. We develop a deep learning framework to analyze the text of the reviews in order to assess their overall valuation. We find that 4 and 5-star ratings, as well as 1 and 2-star ratings, are closer to each other than 3-star ratings are to 2 and 4-star ratings. An additional online experiment corroborates this pattern. Using simulations, we show that the distortion by non-equidistant ratings is especially harmful in cases when organizations receive only a few reviews and when researchers are interested in estimating variance effects. We discuss potential solutions to solve the issue with rating non-equidistance.

Based on an analysis of qualitative research papers published between 2019 and 2021 in four top-tier management journals, we outline three interrelated silences that play a role in the objectification of qualitative research: silencing of noninterview data, silencing the researcher, and silencing context. Our analysis unpacks six silencing moves: creating a hierarchy of data, marginalizing noninterview data, downplaying researcher subjectivity, weakening the value of researcher interpretation, thin description, and backgrounding context. We suggest how researchers might resist the objectification of qualitative research and regain its original promise in developing more impactful and interesting theories: noninterview data can be unsilenced by democratizing data sources and utilizing nonverbal data, the researcher can be unsilenced by leveraging engagement and crafting interpretations, and finally, context can be unsilenced by foregrounding context as an interpretative lens and contextualizing the researcher, the researched, and the research project. Overall, we contribute to current understandings of the objectification of qualitative research by both unpacking particular moves that play a role in it and delineating specific practices that help researchers embrace subjectivity and engage in inspired theorizing.

In this article, we develop a methodological approach for organizational research regarding the construction of multidimensional and relational similarity measures by using the vector space model in natural language processing (NLP). Our vector space approach draws on the well-established premise in organizational research that texts provide a window into social reality and allow measuring theory-based constructs ( e.g., organizations’ self-representations). Using a vector space approach allows capturing the multidimensionality of these theory-based constructs and computing relational similarities between organizational entities ( e.g., organizations, their members, and subunits) in social spaces and with their environments, such as the organization itself, industries, or countries. Thus, our methodological approach contributes to the recent trend in organizational research to use the potential inherent in big (textual) data by using NLP. In an example, we provide guidance for organizational scholars by illustrating how they can ensure validity when applying our methodological contribution in concrete research practice.

While there has been a great deal of guidance on qualitative research methodology, such scholarship has focused almost exclusively on the first three parts of the qualitative process: study design, data gathering, and coding/analysis. We suggest that writing findings is a fourth stage that involves pre-writing and composing. Our intent is to provide practices for this phase for those who are using qualitative data as the evidentiary basis for their claims. The pre-writing phase is strengthened by structuring claims and storyboarding findings, while the composing phase is improved by critically evaluating how to insert the author's voice. Practices surrounding qualitative writing are discussed, such as which quotes to include, where to place quotes, and how to edit quotes. Annotated examples are also provided that show both recommended and nonrecommended ways of inserting the author's voice into a findings section. A sample structure for writing a claim—a claim table—and a sample storyboard are provided.

Organizational research increasingly tests moderated relationships using multiple regression with interaction terms. Most research does so with little concern regarding curvilinear relationships. But methodologists have established that omitting quadratic terms of correlated primary variables may create false interaction positives (type 1 errors). If dependent variables are generated by the canonical process where fully specified regressions satisfy the Gauss-Markov assumptions, including quadratics solves the problem. But our empirical analysis of published organizational research suggests that dependent variables are often generated by processes where, even with quadratics included, regression analyses will remain Gauss-Markov non-compliant. In such cases, our linear algebraic analysis demonstrates that including quadratics—even those motivated by compelling theory—may exacerbate rather than mitigate the incidence of false interaction positives. The interaction coefficient may substantially change its magnitude and even flip sign once quadratics are included, and not necessarily for the better. We encourage researchers to present two full sets of results when testing moderating hypotheses—one with, and one without, quadratic terms. Researchers should then answer five questions developed here in order to determine the preferable set of results.

The study of measurement equivalence has important implications for organizational research. Nonequivalence across groups or over time can affect the results of a study and the conclusions that are drawn from it. As a result, the review paper by Vandenberg & Lance (2000) has been highly cited and has played an important role in understanding the measurement of organizational constructs. However, that paper is now 20 years old, and a number of advances have been made in the application and interpretation of measurement equivalence (ME) since its publication. Therefore, the goal of the present paper is to provide an updated review of ME techniques that describes recent advances in testing for ME and proposes a taxonomy of potential sources of nonequivalence. Finally, we articulate recommendations for applying these newer methods and consider future directions for measurement equivalence research in the organizational literature.