In recent years, the social sciences and humanities have experimented with incorporating instruction-tuned large language models (LLMs) into their methods. Though opinions about their capabilities vary, there has been interest in substituting existing approaches in data annotation and analysis with LLM-based approaches. Much of the interest in LLMs thus far has focused on their accuracy in classification tasks and lowering research costs, partly through obviating the need for manually classified training data via zero-shot learning. At the same time, the current application of these models has been compared to an ”academic Wild West” due to a lack of benchmarks or shared best practices for reliable use.
This presentation explores how LLMs could complement existing analytical methods. In particular, I test their ability to enable topic modelling on noisy text corpora where only a subset of the text is relevant to the research agenda. I illustrate the approach with a case study from the project Eyewitness images in the war in Ukraine. This research compares Ukrainian Telegram channels in terms of how they cover the Ukrainian military operations in Kursk. Informed by domain knowledge and theoretical interests, we extract text segments related to military events and ascribed motivations from longer Telegram messages before applying established topic modelling approaches to the segments. Secondly, I test the reliability of this approach for topic modelling by comparing its results against large human-annotated benchmark datasets. The results suggest that one function for LLMs in social research could be in enabling flexible forms of feature selection (such as selecting text segments) to make complex datasets legible for established research methods.
Dr Aleksi Knuutila is a University Researcher at the Department of Sociology at the University of Helsinki and the Helsinki Institute for Social Sciences and Humanities. After his doctorate in the Digital Anthropology programme at University College London, Knuutila’s research has focused on online harms such as misinformation and harassment and how political groups take advantage of contemporary information environments. His current research projects focus on developing tools and infrastructure for journalists working on conflicts and applying generative AI to interpretative research workflows.
–
Helsinki Institute for Social Sciences and Humanities (HSSH) organizes a weekly Brown Bag Seminar to highlight novel methodological approaches in humanities and social sciences. Bring your own lunch, we bring fresh methodological topics!
There will be a 20-minute introduction to the methodological theme, followed by an open discussion of 40 minutes. The seminars are open to everybody. We expect a multidisciplinary and methodologically curious audience from different faculties and units of the central campus. The most important prerequisite for participation is not methodological expertise, but an open mind towards new methodological innovations and discussion across methodological and disciplinary boundaries.
Please join us on Wednesday 26.3. at 12.15 to listen and discuss!
You are welcome to join us at our seminar room 524 Fabianinkatu 24 A (access via door, not courtyard), 5th floor or online via zoom.
Tapahtuma-ajat
- Alkaa:
- 26.3.2025 12:15
- Loppuu:
- 26.3.2025 13:15
- Tapahtumapaikka:
- HSSH Seminar Room & Zoom
- Osoite:
- Fabianinkatu 24 A, 5. krs, huone 524
- Postinumero:
- 00100
- Postitoimipaikka:
- Helsinki
- Etätapahtuma nimi:
- Zoom
- Etätapahtuma osoite:
- Avaa verkkosivut
Tapahtuman muut tiedot
- Tapahtuman tyyppi:
- Esitelmät ja keskustelutilaisuudet
- Verkkosivut:
- Avaa verkkosivut