Academia: Teaching Historical Research in the Age of AI

Friday's Child Is Loving and Giving

Any pedagogy that is built around being the immovable object in the face of an irresistible force abandons teaching in favor of enforcement. It’s also swapping the possibility of professional and ethical success for the inevitability of failure.

At a minimum, when you become aware as a professor that many of your students are cheating on the tests or exercises you assign, when you realize that most of your students aren’t prepared to do the work you had planned for them to do, when you know that the subject you’re teaching or the skills you work with seem irrelevant or incomprehensible to students, you really have to call a time out and think about what’s going on. You need to understand the underlying causes of these kinds of mismatches. You need to decide whether you can do anything about the situation through your own practices, or if it’s a collective action problem that can be approached departmentally or institutionally.

It is always possible that upon reflection, you will conclude that major changes in your pedagogical status quo are being driven by forces you can’t influence or affect, whether by yourself or in sympathetic alignment with your whole discipline or profession. Even in that situation, you have no real choice but to adapt. If your discipline no longer makes sense to an entire generation, you have to consider whether there are alternative forms or evolutions of your discipline that might connect, and what it would take for you to make the leap. It might be that the basic premise of higher education has shifted, or that the sociology of the students you teach has changed fundamentally. Again, you can only adapt, not contest: you can’t fight your students with the goal of making them pretend to be people they aren’t or compelling them to identify with a narrative about higher education that no longer exists anywhere but in the heads of elderly faculty and alumni.

Most of these moments of recalculation, thankfully, are simpler and offer faculty the chance for some real agency, to make choices that are both a result of their professional training and that rise up from their personal values. I honestly think most faculty in my lifetime have made these kinds of adjustments several times, sometimes on a rolling basis and sometimes in abrupt shifts. I think, I hope, that generative AI is going to be the kind of challenge that calls for those kinds of possible adjustments. Faculty in every discipline need to understand what it is and how it works, understand how and why it is being made available to students, understand where the points of vulnerability and contingency are in how our institutional leaders and colleagues react to and think about generative AI.

I’ve written already about adjustments I’ve made using writing in many of my courses—my assignments now exclusively stage writing as expressive and personal, with a mandate to directly reference what is said in class discussion and in class readings. I’m not going to mark up that writing in terms of its formal structure or quality. This is a shift I’ve moved towards for years, so I’m just committing more dramatically to that direction.

But what about research as an activity and the forms of writing that follow from research? It was a course with a research paper this last fall that convinced me that generative AI was no longer something to just experiment with and investigate but that it had actually become a serious pedagogical problem in my own classes. Some students turned in research papers that were technically fine but intellectually inexplicable—that used obscure sources of the kind that generative AI often privileges and that seemed to have no knowledge of the discussions or ideas in the class. I even tried to anticipate this issue by having students meet with me individually to discuss one source they had read for their paper and warned a number of them that I was concerned about their inability to relate what they had allegedly read to what we’d talked about in the course. (It wasn’t an attendance problem: those students had been there regularly and had even participated in some discussions.)

I could decide to simply be much more punitive about submitted work that is turned in that reads that way, but…that’s trying to be the immovable object. I think this calls for a much fuller meditative pause.

What is research in historical studies, and why should anyone care if students (or anyone) learns what it is and how to do it?

The sociologist Michele Lamont once classified history as a “high consensus discipline” whose practitioners, no matter how much they differ on various issues, tend to readily agree on the difference between bad and good craftwork within the discipline. Lamont attributes this to a shared understanding of the importance of evidence and a belief that knowledge claims about the past must derive from evidence in clearly demonstrated ways. I think this is correct.

Historians restlessly think about what constitutes valid evidence and how to find and collect it. In my lifetime, and I think in no small measure thanks to my own field of specialization (African history), historians have embraced strategies of reading evidence that look in between the lines and work from what was not said, written or visualized. They’ve learned to use spoken testimony in new ways, to pay attention to a much wider range of artifacts and texts, and to use conjecture and imagination in more daring ways than previous generations of scholars. The discipline has also continued to collect and work with large bodies of quantifiable data, both from archives built around such data and through estimations and models from archives that are not primarily quantified.

Most arguments between historians concern interpretations from evidence, though these arguments frequently recurve into the character of the evidence itself, both what it is and how it was assembled. Many of us are aware of a few strong ontological objections to historical work—that the past is, by its nature, not subject to replicable inquiry and claims about the past can never be verified directly—but these are problems that apply not just to historical research but also cosmology and paleontology, and we tend not to be too troubled about these issues. More pressingly we are aware of criticisms by other humanists who view historians as epistemologically unsophisticated or rather theoretically plain in their attachment to and obsession with “the archive”, and that critique has left a mark on the discipline in various ways—among them that many of us have taken a much sharper interest in the production and maintenance of archives and to think much more carefully about what aspects of past human life are not—and in some cases never will be—in any archive. Which in turn requires us to think about how to represent what we feel certain must have been true about some human pasts that we cannot document in our accustomed fashion.

I’ve already indicated that this point is my first and most pressing objection to AI enthusiasm among historians like Mark Humphries—it feels as if all of that hard-won epistemological wisdom is just being thrown away and we’re right back to the more naive positivism that early cliometrics scholarship exhibited—punch in the numbers and hey! it turns out slavery wasn’t so bad. However, if I’m talking about teaching historical research, this is a point that has real pedagogical life to it, if it’s staged well.

That’s the first key thing I want to do when I realize that a strong assumption about the norms in my discipline or my scholarship is no longer safely assumed for the next generation. I have to ask myself whether I am confident about those norms, and perhaps whether I’ve had my own doubts. The second key thing is to enlist my students in that dialogue. Which means I have to treat the answers as open, contingent, unfixed, for myself and for my students. There is nothing more unconvincing in teaching than asking a question to which there is only one right or valid answer.

This insight leads me to something more concrete as a design principle for a research-oriented class in the age of AI. For much of my career, I’ve used research papers at the end of a thematic or topical class as another way of exploring the content of the course, as a way to expand the range of coverage while also giving students a sense of how the historiography of that topic developed, to do their own critically self-aware exploration of the history of how historians have written about it. Sometimes that’s worked beautifully, sometimes less so, for any variety of reasons. But it’s a course design that I think I have to completely junk.

A class that is about research now needs to be entirely about research. If it’s about research and about producing a work of research, it’s got to be about nothing but that. The entire class has to be a precisely scaffolded step-by-step creation of a research process that leads to some kind of concrete product.

This proposition runs smack dab into a different kind of design problem. Skill-based courses that are about nothing but the skill are also notoriously dull and uninvolving for most students, not the least because they tend to be foregone conclusions—there is rarely a moment where such a class might dwell on whether the skill is worth learning, to argue about whether there are other ways to do what the skill proposes, or to consider alternative disciplines and fields that approach the same investigation in a very different way.

As a design prospectus, how does this all add up?

• A course that is focused on historical research has to allow for debates about both the methodological and epistemological validity of historical research.

• In the age of AI, that has to showcase the possibilities and problems of generative AI in relationship to historical research.

• The course has to be focused entirely on building up an accumulative approach to doing research in history.

• The professor has to review and discuss every single stage in the development of the research with each individual student—this cannot be the kind of class where you tell students to go away and do the work for a few weeks and then come back. And this engagement has to bundle together guidance about the research as it develops and epistemological debate about what each student is thinking about the act of research. This means in practical terms that this has to be a very small class.

This list leads me to another design principle: the course should not be done with a tightly curated archive where the professor knows in advance all the valid questions and possible conclusions that could come out of that archive. It has to be done in a single real archive that is not bounded or curated where neither the professor nor the students approach it with a fixed sense of what is there and not there and what should or could be said with and through that archive.

As I write this out, this course starts to look on paper like my own department’s senior research seminar, and it’s true that I haven’t seen students in that class producing work that I think is deformed by generative AI. But I think the difference is that students in that class are coming up with their own projects without much constraint (we only stop them when what they’re proposing is either blatantly non-historical or just can’t be done in practical terms by an undergraduate in a single semester who has limits to the resources they can draw upon). For a course that is not a culminating exercise, somehow I’d have to give all the students an experience of being in a real archive, without salting the mine to the point that I feel “safe” that they will find a real problem that can be researched in entirely respectable ways, but not in any archive, and not by themselves.

This is by no means a finished course design, but I think it’s the kind of design thinking that historians who want to teach research to undergraduates are going to have to exclusively prefer. Generative AI and the enshittification of online catalogs and databases, for good or ill—I think mostly ill—has killed the sort of course where students do a “research paper” as a closing exercise to studying a particular theme or topic. Anybody who keeps going with that kind of class is going to have to decide whether they want to keep being a professor or whether they’d prefer being a cop.

Image credit: British Library Reading Room, 1924, photo by Donald Macbeth.

Image credit: "Historical archiving room" by Wordshore is licensed under CC BY-NC-ND 2.0.