Jargon detection in international development

Jeong Hyun Lee is a Strategic Outreach and Communications Intern with the Economics that Really Matters Blog and a senior at Ithaca High School

In response to Chris Blattman’s invitation to compare jargon words in international development documents, I have compiled data on jargon words from 49 publications across 10 different development organizations and/or research institutions (Table 1) using The International Development Jargon Detector developed by Michael Benedict.  

Why do we care about the use of jargon in international development? Jargon words cause confusion and misunderstanding to those not expert in a given field of study; they effectively discriminate against non-experts. Moreover, their usage makes information inaccessable to the public, as well as the beneficiaries of development work, who may not be able to understand the implications of the “transparent” publications of world institutions.

Before I begin discussing my analysis, I will make clear that when I write “instances of jargon words,” I mean the number of distinct jargon words used in a document and when I write “the number or percentage of jargon words used,” I mean the total number or share of jargon words in the document.

For each institution, I chose an annually published flagship report and collected data from the last 5 published documents, thus spanning a period of 5 years (Table 1). There is one exception, from the International Food Policy Research Institute, where only 4 years’ worth of documents were available. Then, for each document, I found the percentage of jargon words compared to the total number of words and averaged that statistic over the 5, or 4 in one case, documents to find the average percentage of jargon per document for that group.

Overall, aggregating data from all documents, jargon words comprised a surprisingly low proportion of the whole document. There was no single document where jargon words took up more than one percent of the total number of words. However, the percentage of jargon words used in these reports has increased each year, for the past five years, by roughly 0.04% (Figure 1).

Of course, this observation may simply reflect the fact that the jargon detector detects a set of words that are considered jargon today but may not have been considered jargon five years ago.

Comparing the average portion of jargon words used in the reports by institution, I found that, among all the institutions, the International Food Policy Research Institute had the highest average with 0.83% jargon words per document and ReSAKSS the lowest with 0.33% per document (Figure 2). Looking at the instances of distinct jargon words used, the International Foundation for Agricultural Development used the largest variety of jargon words, implementing an average of 15.4 distinct jargon words per document (Figure 3). It may be interesting to pursue further analysis considering the sources of funding for the various institutions under analysis and how these correlate with the use of jargon in general and certain buzzwords words in particular.

The most commonly used jargon word was “accountable,” appearing in all 49 documents with an average of 78.9 instances per document. Appearing most commonly after “accountable” were “capacity” and “sustainable,” each also appearing in all 49 documents with a mean of 54.7 and 50.4 words per document, respectively. I found a total of 18 instances of jargon words used in 49 documents, with on average at least one appearance per document. The following graph (Figure 4) shows the average number of appearances per document for those 18 words.

Of these words, “accountable” is also a word that has been used increasingly over time. Figure 5 lays out the average usage of a few selected words over the course of 6 years. The years 2009 and 2016 are omitted because we only have 1 and 2 sample documents for these years, respectively, but these are words that have in fact been recurring in documents from 2009 to 2016. We see in general, the use of jargon words has increased, but the word “accountable” significantly more so than any other. “Capacity” was the only one that displays a decreasing trend.

Overall, although they make up a very small share of the total words in any given document, the use of jargon, at least for the set of words identified by the jargon detector, is on the rise across the institutions analyzed. This brief analysis raises several interesting questions: is the use of particular jargon words due simply to trends in development? And/or are these trends/jargon words driven by funders of international development? Finally, what do particular jargon words signal to various readers? More research is merited, but I’ll leave it for another millennial for now.