Germán Reyes is a Ph.D. candidate in the Department of Economics at Cornell University. In June 2023, he will join the briq Institute on Behavior & Inequality as a Postdoctoral Fellow. In January 2024, he will join the Department of Economics at Middlebury College as an Assistant Professor.
Introduction
Just like factories require physical capital—machinery, buildings, tools—to produce output, workers rely on their human capital—abilities, skills, knowledge—to accomplish job tasks. An important question—key for designing more effective education and labor-market policies—is which skills matter the most for worker productivity.
Casual observation and some introspection suggest that skills other than intelligence, often called “noncognitive skills,” such as industriousness, perseverance, and self-control, matter for success in life. Non-cognitive skills are, unfortunately, challenging to measure. To economists, measures based on observed behavior are the gold standard, but these are hard to come about. Most existing work uses self-reported measures of these skills, which are subject to well-known biases.
In my job market paper, I study one noncognitive skill that may be particularly important for knowledge workers: the ability to sustain performance on a cognitively demanding task for an extended duration, or “cognitive endurance,” for short. Psychologists and self-help books have long hypothesized that cognitive endurance is an important productivity determinant. Despite this popular perception, empirical economists have had little to say about the role of endurance in the labor market due to measurement challenges.
Using standardized exams to measure endurance
I address this problem by using data from the college admission exam in Brazil (called “ENEM”) and create an individual-level measure of endurance that is based on performance declines throughout the exam. This endurance relies on an empirical regularity documented in a wide variety of settings, namely, that the performance of professionals tends to deteriorate over relatively short time spans. For example, studies show that over the course of a day: financial analysts make less accurate forecasts, nurses are less likely to wash their hands, and umpires make more incorrect calls during baseball games. The key idea is that how fast the performance of individuals deteriorates is an indicator of their cognitive endurance.
To make this idea concrete, Figure 1 shows the link between student performance and question position in the ENEM exam. The figure illustrates a notable connection between how well students do on an exam and the order of the questions. Students tend to do better at the start of the exam, with an average of 45% correct answers, but this number drops to around 24% by the end.
Figure 1. Student performance tends to decline over the course of each testing day
Notes: This figure shows student performance over the course of each testing day in the ENEM. The y-axis displays the fraction of students who correctly responded to each question, averaged across all years in my sample. The x-axis displays the position of each question in the exam. The dashed lines are predicted values from a linear regression estimated separately for each testing day. The horizontal red dashed line shows the expected performance if students randomly guessed the answer to each question.
How should we interpret Figure 1? One explanation is that students have limited cognitive endurance. As they progress through the exam, they become mentally fatigued, which affects their performance by impairing cognitive functions like attention, memory, or reasoning. Worse cognitive functions can manifest as forgetting an essential formula or making computation mistakes. This explanation predicts that students are more likely to make errors as time goes on. However, it’s important to consider that the negative association between performance and question order could also be influenced by other factors. For example, the material tested later in the exam may be more challenging.
In the paper, I follow two strategies to distinguish between competing explanations. First, I control for question fixed effects. With this, I can assess how students do on a given question in booklets where it appears early on relative to booklets where it appears later on. Second, I control for measures of question difficulty. Luckily, both strategies deliver a similar answer: over the course of each day, the performance of the average student declines by about seven percentage points.
Crucially, this analysis can also be done separately for each student. Intuitively, one can use a given student’s exam responses to measure the magnitude of their performance decline. By doing this, I generate individual-level estimates of cognitive endurance (i.e., how the performance of the student changes throughout the exam) and fatigue-adjusted ability (i.e., what the student’s performance would be if their performance didn’t decline).
Cognitive endurance predicts long-run outcomes
I use these estimates to investigate the relationship between endurance and long-term outcomes. I find that students with greater cognitive endurance are more likely to attend college, enroll in better-quality institutions, graduate, earn higher salaries, and work for higher-paying companies. Importantly, the value of endurance varies across college majors, occupations, and industries. For example, high-paying occupations tend to offer higher wage returns to endurance, suggesting a novel type of assortative matching between high-endurance workers and high-paying jobs. Additionally, occupations with high wage returns for endurance often have high returns for ability as well, suggesting these skills complement each other in the workplace.
A New Perspective on Test Scores and the Design of Standardized Tests
This finding provides a new perspective on the well-documented relationship between test scores and long-run outcomes. Specifically, my findings suggest that test scores bundle information from two distinct skills—endurance and ability—that hold varying importance for different college degrees and careers. This is good news: we have more information about applicants than we previously realized! And we could use this information to better match applicants to college programs (or workers to firms).
More generally, the extent to which the cognitive assessments measure ability vis-à-vis endurance depends on their length. This is pretty intuitive: you don’t need much endurance to do well on a very short exam. Endurance becomes more important as cognitive assessments become longer and longer. An important question is which type of student is most affected by longer assessments. In the paper I also ask how an exam design that focuses less on endurance affects the test-score gap across socioeconomic backgrounds and the predictive validity of the exam. If this is of interest to you, please make sure to check it out!
Implications for Human Capital Investments and Open Questions
My findings suggest that policymakers should consider investing in the development of cognitive endurance, which is typically not addressed in a standard school curriculum. Research on building cognitive endurance is still in its infancy, but examples of protocols that show promise include mindfulness meditation, engaging in cognitively effortful activities, and restricting smartphone usage in learning environments. However, it’s essential to note that this study doesn’t provide causal evidence of the link between endurance and earnings. Future research should establish a causal relationship to guide more effective policies.
Finally, the decomposition of test scores into fatigue-adjusted ability and endurance opens new avenues for future research. Test scores are commonly used in research, for example, to assess the impact of interventions on long-run outcomes or to evaluate the effectiveness of educational inputs, like teachers. By decomposing test scores, researchers can explore the role of endurance in various settings, such as identifying teachers who might be particularly effective at building cognitive endurance through value-added . Relatedly, the decomposition allows researchers to generate measures of endurance that can be used in the subsequent analysis, for example, to understand what variables affect cognitive endurance, to use endurance as a control variable, or to test economic theories. These are some promising areas for future research.
___________________________________________________
Interview with the author
Q: How do you navigate the journey from ideas to research projects?
A: After passing the qualifying exams, I felt the need to get involved in as many projects as possible. I pitched ideas to everyone and said yes to any invitation. This was partly due to the allure of brainstorming and idea exchanges, which is often more fun than the rigorous work necessary to implement them. But also because I was incredibly (and still am) concerned with a “fear of missing out.” I was so hungry to show I was worthy (as a researcher, and I felt that, by extension, as a person) that I didn’t want to say no to any opportunity.
Just like me, I know many people who, at some point, get involved in more projects than they can possibly think deeply about. But saying yes to every opportunity is not a sustainable project-selection criterium. As a rule of thumb, one cannot do serious thinking in more than two to three projects at a time.
So, how should one choose which projects to commit to? One naive approach is to choose projects that seem exciting. The problem is that the initial excitement often fades away, and predicting which project will sustain interest over the long term is challenging given the evolving focus of projects and of your own interests.
Still, there are a few practical heuristics that I have found valuable to navigate the selection process.
First, implement a “cool-off” period before making a long-term commitment. This allows your “future self” to evaluate the potential research project with a more detached, objective perspective. A project that initially appears ground-breaking may lose its charm after some weeks—or after a thorough literature review. One piece of advice I received from a committee member is to set a low bar for initially exploring the potential of a project (say, a week to calculate some descriptive statistics and basic regressions) but a high bar to commit to an actual project.
Second, don’t underestimate the amenities of a project. It’s common to choose projects based on their perceived publishing potential (“Is this top-five material?”). But there is much more than that to a project. First, a paper allows you to think deeply and consistently about a problem for years. Thus, it is sensible to choose a subject you’re genuinely interested in. Second, collaborative projects give you a chance to interact with others frequently. This can be extremely rewarding if you admire and get along with your coauthors (the converse is also true, so be careful while choosing whom to collaborate with). Third, projects outside your “range” allow you to learn new skills and subjects. For example, as an empirical economist, collaborating on a signaling paper with a theorist forced me to learn about signaling models and the fundamentals of model building.
Finally, adopt a “long view.” If you consider being involved in one major project per year (an ambitious number by most accounts) over a forty-year career, this amounts to forty main research projects in a lifetime. When considering a new project, ask yourself: “Is this of those forty?” If not, taking on a mediocre project may hinder the opportunity to undertake more promising ones in the future. While not all projects need to be ground-breaking, I do think all projects need to be chosen deliberately.
The journey of generating ideas and turning them into high-quality papers is both complex and multifaceted. It starts with fostering serendipity and seeking diverse sources of inspiration, creating an environment conducive to the birth of novel ideas. These ideas are then curated and filtered through reflection and external feedback, distinguishing the promising from the mediocre. The next step, committing to a project, involves evaluating not only the idea’s potential academic contribution but also personal factors like sustained interest, collaboration potential, and opportunities for professional growth. These principles underscore the importance of strategic thinking in navigating the challenging yet rewarding path of academic research.
German provided us with some more great insights, you can read more of his interview here.
