The Center for Positive/Empirical Analysis of Political Economy with support from Waseda University’s Top Global University project (TGU) held a special seminar on January 29, 2024, entitled “Advancing Social Science Methodologies: Word Embeddings and Large Language Models” by Professor Luigi Curini (University of Milan). We’d like to share a report from a visiting student, Mr. Jannik Hoffman.
“Advancing Social Science Methodologies: Report on Prof. Curini’s Seminar on Word Embeddings and Large Language Models”
Jannik Marian Hoffmann (University of Mannheim)
The Special Seminar by Professor Luigi Curini offered a deep dive into modern quantitative research methods in social sciences. The session effectively covered the shift from traditional text analysis methods, like the Bag-of-Words (BoW) approach, relying on mere word frequencies, towards more nuanced Word Embedding (WE) techniques, with a specific focus on GloVe, and ventured into the current edge territory of Large Language Models (LLMs) such as BERT and Open AI’s GPT family of models. The introduction to transformers and the self-attention mechanism in these models was a highlight, offering insights into recent advancements in text analysis and machine learning, showing how we can utilize them in our own research.
A central theme of the workshop was the transformative potential of LLMs and the mechanism of self-attention in revolutionizing our approach to textual data analysis. This mechanism allows models like BERT to understand the context of words dynamically, based on their surrounding text, enabling a deeper, more nuanced analysis than ever before. Such advancements underscore the significant shift from mere frequency counts to a more profound exploration of textual meanings.
What set the workshop apart was its practical coding session, which provided an opportunity for hands-on learning. This session was especially beneficial, as it allowed us participants to directly apply the discussed concepts, making the workshop accessible and highly relevant to both beginners and experienced researchers. The examples used and the application-focused approach helped combine the theoretical concepts with practical application, showcasing the potential of these methodologies in real-world research settings.
I highly recommend this workshop to students and researchers aiming to stay on top of the latest in social science research techniques. It offers a unique blend of theoretical knowledge and practical skills, presented by an expert practitioner in the field. Whether you’re looking to begin learning about the foundational aspects of advanced text analysis or seeking to enhance your research capabilities with the latest methodologies, the Special Seminar provided an excellent learning opportunity to explore what’s at the frontier of quantitative social science research in 2024.
Prof. Hino & Prof. Curini
