Data in social sciences is more than just numbers—it’s images, speech, and a mix of different formats that don’t always fit neatly into traditional analysis methods. That’s where deep learning comes in. This masterclass is a practical introduction to using transformers for real-world research problems with focus on non-text (multimodal) applications. No advanced coding or math skills required.
Over three days, you’ll get hands-on with Python and Google Colab, learning how to apply pretrained transformer models to tasks like emotion detection in images, speech-to-text conversion, and integrating multiple data types. You’ll work with real-world datasets and powerful pre-trained models, so you can see firsthand how AI can uncover patterns and insights that traditional methods might miss.
Struggling with messy, unstructured data? Wondering how to make AI work for your research without spending months on theory? This course will show you how to fine-tune models, improve results for your specific dataset, and choose the right evaluation methods—all in a way that’s accessible and practical for social scientists and researchers.
Whether you're looking to add AI to your research toolkit or just curious about what deep learning can do, this masterclass will give you the foundation you need to start experimenting with confidence.
This course will be run over 3 days using the following timetable:
Day 1
- 9.30 am - 10.00 am – Introductions and setup check
- 10.00 am - 11.30 am - Session 1
- 12.30 pm - 2.00 pm - Session 2
- 3.00 pm - 5.00pm - Session 3 + exercises
Days 2 and 3
- 9.00 am - 10.30 am - Session 1
- 11.30 pm - 1.00 pm - Session 2
- 2.00 pm - 4.00pm - Session 3 exercises and consultation
Dr. Maria Prokofieva is an Associate Professor in AI/ML at Victoria University and a data lead at the Mitchell Institute, Vic. She works on projects that use AI/ML approaches and Large Language Models to research applications in business and healthcare. As a machine learning engineer with a deep passion for the responsible application of AI, Maria's work deciphers complex online behaviors to inform consumer and business strategies. She also chairs the CPA Australia Business Analytics Group and part of CPA Australia Technology Innovation Committee, driving innovation in data analysis tools. Maria’s contributions to both research and practical applications are shaping the integration of AI in business and policy on a global scale.
Social science research increasingly involves complex data such as images, speech, and mixed data formats that are difficult to analyse using traditional methods. Deep learning provides practical ways to work with this type of data and extract meaningful patterns from it.
This masterclass introduces deep learning through hands-on examples designed for researchers and professionals in the social sciences. Participants will learn how to use existing deep learning models to analyse images, speech, and text-based data, and how these approaches can support real research questions.
Over three days, participants will work in Python using Google Colab and apply pre-trained models to practical tasks such as emotion detection in images, speech-to-text conversion, and analysing datasets that combine multiple data types. The focus is on experimenting with real datasets and understanding how deep learning can be applied in research settings.
The course emphasises practical experimentation rather than theory. Participants will learn how to apply existing models to their own data, evaluate results, and improve outcomes for specific research tasks.
By the end of the course, participants will understand how deep learning approaches can be used to analyse complex data and will have the foundation needed to begin exploring these methods in their own research projects.
Day 1: Foundations of Deep Learning (DL)
- Introduction to working with Python, Google Colab and coding assistants
- What deep learning is and how it is used in research
- DL and neural networks basics
- Using pre-trained models from the Hugging Face Hub
- Understanding results and improving model performance
Day 2: Working with Images and Visual Data
- Deep learning approaches for analysing images
- Image classification and emotion detection
- Object detection and visual question answering
- Image captioning and video analysis
- Evaluating and improving results
Day 3: Working with Audio and Multimodal Data
- Deep learning approaches for multimodal data (text+images+audio)
- Designing a deep learning workflow for research projects
- Challenges and limitations of Deep Learning in Social Science Research
- Data, evaluation and reliability in Deep Learning Research
- Current developments in Deep Learning and research practice
- Planning your DL project
Hands on Examples:
- Participants explore multimodal datasets including
- RAVDESS (speech emotion recognition)
- LibriSpeech (automatic speech recognition)
- MELD dataset (emotion recognition across text, audio and video)
- How2 Dataset (multimodal video-based learning).
This workshop will take place online.
BYO Laptop + Zoom. Both PC and MAC are great
The course uses Google Colab and requires a Google account (please make sure you have one or please register one before the session)
All course materials will be provided
The course requires understanding of a basic of statistical concepts, exposure to machine learning foundations is beneficial as well, such as Machine Learning for Data Science: Surpervised Learning Techniques
The course assumes no prior knowledge of Python, though some programming experience (e.g. using R) is beneficial.
TensorFlow official Getting Started Guide
https://www.tensorflow.org/tutorials/quickstart/beginner
Géron, A. (2022). Hands-on machine learning with Scikit-Learn, Keras, and TensorFlow. O'Reilly Media, Inc.
https://learning.oreilly.com/library/view/hands-on-machine-learning/9781492032632/
Q. Do I need to install anything before the session? What is Google Colab?
A. No, you do not need to install anything. We will work with Google Colab which is a free cloud service hosted by Google. It allows you to write and execute Python code through your browser. Just make sure you have a Google account! You can sign up here:
https://accounts.google.com/signin
Q. I have used R before, but not Python. Will I struggle?
A. Coming from an R background, you'll find that Python has some differences in syntax and data structures, but many of the underlying concepts are similar – you will be fine!
Q. Where can I see resources for the course?
A. All resources will be available after the course in open access, including Jupyter notebooks with practical examples covered throughout the course and additional cases.
I wanted to get behind the scenes of what was happening with artificial neural networks to see how these algorithms worked. Maria provided a very in-depth presentation of the structure of these processes that suited my experience.
I very much hope Maria's course/s on deep learning continue as the material is extremely insightful.