POST: Biases and Errors In Our Tools: How Do You Cope? Reflections of a Newcomer to Textual Analysis

Jennifer Vinopal (New York University) asks the question, “how do you cope with the possibility that the tools you are using may be biased (or error prone) and that the “black box” is sending you down the wrong research highway?” Vinopal relates her experience using a text analysis tool that revealed the term “e-books” was apparently used more frequently in 1930s library literature than in the 2000s. The anomaly suggested an interesting question:

Without the specific programming skills to evaluate how a tool was built and look for biases or errors in the code, how can we be sure the tools we’re using aren’t giving us erroneous results?

Vinopal goes on to offer several suggestions to begin evaluating the effectiveness of tools and methods.