Muhammad Zia Mehmood is a PhD Job Market Candidate at UC Berkeley in the Business and Public Policy group at the Haas School of Business.
Click here for Muhammad’s job market paper elevator pitch.
Introduction
Small businesses form the economic backbone of low-income countries and strengthening these enterprises is fundamental to alleviating poverty. Poor management practices is a major factor constraining firm productivity in these contexts, and over $1 billion is spent annually to address this constraint by providing business trainings to entrepreneurs. However, most of these are conventional, in-person, classroom-style trainings, which are expensive and hard to scale, and can exclude those who are unable to participate in person. Due to their low costs, scalability, and reach, phone-based trainings are gaining popularity as a potential solution, however, there is limited evidence on whether remotely provided trainings are effective for micro-entrepreneurs in low-income settings.
In my job market paper, I study the demand for and potential of text message-based business trainings using a field experiment in Kenya, in which access to an SMS-based training was randomized across 4,700 micro-entrepreneurs. I estimate short- and longer-run impacts using phone-based surveys conducted three months (Midline: 307 observations) and twelve months (Endline: 2,780 observations) after the intervention. I also elicit ex ante predictions for 12-month treatment effects from researchers through the Social Science Predictions Platform (SSPP), in order to assess whether the main findings from the study depart from existing priors. Lastly, I measure demand for these trainings through Take-It-Or-Leave-It (TIOLI) offers and the Becker-DeGroot-Marschak (BDM) willingness-to-pay elicitation method for a subset of the study sample.
Context and Intervention
According to a 2016 nationwide survey of small businesses in Kenya, 78.6 percent of micro-entrepreneurs did not advertise any of their products, 70 percent didn’t keep any records of sales and expenses, less than 10% accounted for prices of their competitors when setting prices for their own products, and 90% had never received any type of business training. I partnered with a local firm specializing in digital content development and dissemination to implement an SMS-based training aimed at addressing these management gaps and others. Available in English as well as Swahili, the training content covered best practices, including marketing, advertising, pricing, record-keeping, and stock management. Information was conveyed in a narrative format describing decision-making of hypothetical micro-entrepreneurs in different scenarios. The content was organized in a fixed sequence of bite-sized chunks spanning approximately 150 text messages, and was accessed by users through self-paced engagement with an interactive chat-bot. Users could complete the entire training in one go in five to seven hours if they wanted to, and access to all received content was retained indefinitely on their phones. Weekly text reminders were sent to those who stopped engaging with the chatbot at any point, and these reminders stopped if engagement was resumed or after two consecutive months of inactivity.
The primary sample for the study was sourced from a list of micro-entrepreneurs compiled by my implementation partner and a local microfinance institution. Half of the study sample consists of female micro-entrepreneurs, and roughly 45 percent is based in rural areas. The average individual was about 35 years old, and had completed almost twelve years of education (high school level).
Results
Three months after the intervention, I find that the SMS training increased knowledge and adoption of best practices by 0.20 and 0.33 standard deviations, respectively. I also find large positive, but statistically insignificant, effects on business performance in the overall sample, and significant positive effects for younger (below-median) micro-entrepreneurs on sales (109 percent increase), profits (38 percent increase), and business survival (11.6 percentage points increase). These positive effects for younger entrepreneurs are driven by higher engagement with the content, and larger effects on time spent on business, and loan amounts applied for and received.
However, these positive results dissipate in the longer run; twelve months after the intervention, I see no effects on knowledge and adoption of best practices, as well as business sales, profits and survival. Additionally, the positive effects on business outcomes observed for younger entrepreneurs after three months, also disappear after twelve months. Data on engagement time-trends reveals that the lack of longer-run effects was likely driven by micro-entrepreneurs abandoning all interactions with the content within the first few months of the intervention. The survival curve in Figure 2 shows how all cumulative aggregate engagement with the platform ended by May 2022 – five months into the intervention.
Figure 3 illustrates how these results compare with predictions for 12-month treatment effects elicited ex ante through the SSPP. I find that SSPP researchers overestimated the engagement levels, both in terms of the proportion of the treatment group that would start engaging with the content (50% vs 30%), as well as in terms of the percentage of content that the average individual in the treatment group would be able to cover after twelve months (40% vs 7%). Furthermore, predictions for the 12-month treatment effects on knowledge and adoption of best practices are somewhat similar to observed effects at three months, but significantly overestimated in light of observed 12-month treatment effects. Effects on business performance offer a similar story; SSPP predictions for the 12-month treatment effects on sales and profits are similar in magnitude to effects observed at three months (albeit statistically insignificant), but they grossly overestimate the effects at twelve months.
Additionally, notwithstanding the low engagement and lack of longer-run effects, I find positive demand for SMS-based trainings amongst micro-entrepreneurs; both methods of elicitations – the TIOLI offers as well as the BDM exercise – reveal that individuals are willing to pay a small amount for an additional SMS-based training, suggesting that they value access to the content.
Policy Implications
These findings show that, despite their growing popularity in low-income and less accessible settings, SMS-based trainings are unlikely to improve outcomes for micro-entrepreneurs in the longer-run. These results highlight the importance of lack of engagement as a major challenge that limits the potential of remotely-provided information-based support.
Further, the forecasting exercise reveals that social science researchers overestimate the potential of SMS-based trainings to improve outcomes for micro-entrepreneurs, and the findings from this study are thus contrary to priors. Updating these priors is important since policy makers and practitioners often rely on beliefs about impacts held by social scientists, for making decisions about investment of resources into such remote information-based support programs
Lastly, the results on willingness to pay suggest that engagement with the content might not reflect the actual demand for SMS-based trainings, pointing towards possible behavioral drivers constraining engagement. Further research is needed in this direction to shed light on how to get people to engage with remotely provided content to improve learning, in order to capitalize on the full potential of digital content delivery in low-income settings.
Feature photo – Janeffer Wacheke Photographer: Luis Tato/Bloomberg 2018
