336 lines
13 KiB
TeX
336 lines
13 KiB
TeX
\documentclass[aspectratio=169]{beamer}
|
|
|
|
\usepackage{tabularx}
|
|
\usepackage{booktabs}
|
|
\usepackage{csquotes}
|
|
\usepackage{graphicx}
|
|
|
|
\usepackage{xcolor}
|
|
\newcommand\todo[1]{\textcolor{red}{#1}}
|
|
|
|
% section frame
|
|
\AtBeginSection[]{
|
|
\begin{frame}
|
|
\vfill
|
|
\centering
|
|
\begin{beamercolorbox}[sep=8pt,center,shadow=true,rounded=true]{title}
|
|
\usebeamerfont{title}\insertsectionhead\par%
|
|
\end{beamercolorbox}
|
|
\vfill
|
|
\end{frame}
|
|
}
|
|
|
|
% URL coloring
|
|
\hypersetup {
|
|
colorlinks=true,
|
|
linkcolor=blue,
|
|
filecolor=magenta,
|
|
urlcolor=blue,
|
|
pdftitle={PhD Interview},
|
|
pdfpagemode=FullScreen,
|
|
}
|
|
|
|
% Referencing
|
|
\bibliographystyle{IEEEtran}
|
|
|
|
\title{Can the use of a virtual avatar and scaffolding with AI agents improve
|
|
first-stage programming students' debugging ability?}
|
|
%\subtitle{aa}
|
|
\date{23rd of April 2026}
|
|
\author{Warwick New}
|
|
|
|
\begin{document}
|
|
\maketitle
|
|
|
|
%\section{What is the problem I'm trying to solve?}
|
|
|
|
\begin{frame}
|
|
\frametitle{What is the problem space?}
|
|
Novices tend to struggle with programming fundamentals, finding the basics
|
|
quite difficult to learn, and so introductory courses tend to have poor rates
|
|
of retention.
|
|
\newline
|
|
|
|
Students self perceptions about their programming ability affects student
|
|
retention and increases the rate of students that drop out \cite{lewis2011} in
|
|
their first stage.
|
|
\newline
|
|
|
|
A particular point of frustration for novice programmers is attempting to
|
|
debug without thinking of the problem critically \cite[p.~23]{vickers2008}.
|
|
\newline
|
|
|
|
Previous interventions in this space have shown promise such as Gidget
|
|
\cite{Lee2014} that has used personification of compilers to great effect and
|
|
Metacodenition \cite{Pechorina2023} which helps the student create a mental
|
|
model about actions to take when a bug appears.
|
|
\newline
|
|
|
|
Recent literature has begun exploring the use of LLMs but out of the box,
|
|
found them in most cases to be worse than the alternatives.
|
|
\cite{Pechorina2023}. Though recent literature has found that there is a
|
|
benefit to providing code explanation outside business hours
|
|
\cite{Renzella2025}.
|
|
|
|
%Recent literature has identified this as a problem space that AI
|
|
%can attempt to help resolve. Renzella et al's work \cite{renzella2025}
|
|
%demonstrates that there are aspects of learning that traditionally could only
|
|
%be fulfilled by an instructor outside highly controlled learning
|
|
%environments (like Gidget \cite{Lee2014}) that can be partially automated with
|
|
%generative AI. And therefore can be made more available to students outside
|
|
%timetabled sessions.
|
|
%%in this case reducing the cognitive load \cite{sweller1988} of reading and understanding compiler messages by providing more human-readable descriptions of compiler errors.
|
|
\end{frame}
|
|
|
|
%\begin{frame}
|
|
% \frametitle{What is the gap I've identified}
|
|
% \todo{Cite and Professionalise this slide}
|
|
% \begin{itemize}
|
|
% \item Much research into debugging education for students more recently is
|
|
% focused on generative AI models and some new paradigms that come from that
|
|
% \todo{\cite{}}. This is to be expected with a whole new paradigm in
|
|
% education in this area.
|
|
% \item Before this focus however much research into the area of first stage
|
|
% computer science debbugging was focused on teaching debugging in closed
|
|
% controlled environments \todo{\cite{}}, This allowed for experimentation
|
|
% with making fixing coding problems more accessible, in terms of compiler
|
|
% error messages and scaffolding of the processes a student might take when
|
|
% they ran into a problem.
|
|
% \item I think that we can take these classical educational tools and use
|
|
% them combined with more recent generative AI computing education
|
|
% technologies to.
|
|
% \begin{enumerate}
|
|
% \item Remove the need for locked down highly controlled debugging
|
|
% environments.
|
|
% \item Improve the human element to debugging messages and tools with how
|
|
% much more human the AI can appear to be to students.
|
|
% \end{enumerate}
|
|
% \end{itemize}
|
|
%\end{frame}
|
|
|
|
\begin{frame}
|
|
\frametitle{What is the Gap}
|
|
{\footnotesize
|
|
Two practices in helping students learn to debug have shown great promise when
|
|
it comes to improving self-efficacy and lowering programming anxiety. I
|
|
believe that we could use recent advances in AI to augment aspects of these
|
|
interventions.
|
|
\begin{itemize}
|
|
\item Personification of programming tools as fallible and encouraging, is
|
|
shown to have a positive impact on learning motivation and success as
|
|
shown in Gidget \cite{Lee2011}.
|
|
\item Scaffolding what steps to take when encountering a bug, improves
|
|
self-efficacy by lowering cognitive load as demonstrated Metacodenition
|
|
\cite{Pechorina2023}.
|
|
\end{itemize}
|
|
%Both of these works are restriceted to very specific environments limiting their application on real world coding projects.
|
|
\begin{figure}
|
|
\centering
|
|
\includegraphics[width=0.45\textwidth]{progprac}
|
|
\caption{Figure 2 provided by Scott et al. \cite{Scott2014}}
|
|
\label{fig:question}
|
|
\end{figure}
|
|
This figure shows how programming self-concept influences programming anxiety
|
|
and increasing time on task, retaining more computing students.
|
|
}
|
|
\end{frame}
|
|
|
|
%\begin{frame}
|
|
% \frametitle{The fields this gap interacts with}
|
|
% \begin{itemize}
|
|
% \item Computer science education
|
|
% \begin{itemize}
|
|
% \item Specifically focusing on first stage debugging.
|
|
% \end{itemize}
|
|
% \item Psychology
|
|
% \begin{itemize}
|
|
% \item Student Self-efficacy. \todo{cite}
|
|
% \item Students engagement. (Time Spent working on programming tasks)
|
|
% \todo{Define engagement}
|
|
% \end{itemize}
|
|
% \item Artificial Intelligence
|
|
% \begin{itemize}
|
|
% \item Scaffolding of Generative AI queries to fit a more traditionally
|
|
% productive learning environment in a wide range of situations.
|
|
% \item Encourage personal reflection and action when attempting to solve
|
|
% an issue.
|
|
% \end{itemize}
|
|
% \end{itemize}
|
|
%\end{frame}
|
|
|
|
%\section{How I plan to address this space}
|
|
|
|
%\begin{frame}
|
|
% \frametitle{How am I addressing the gap}
|
|
% \todo{Talk about Gidget and offer scaffolding tools, then talk about how we
|
|
% can create a tool to aid intervention}
|
|
% I plan to create an intervention wherein I give students access to an AI tool
|
|
% that takes elements from previous interventions that align with the principles
|
|
% of improving self-efficacy in students, and combine it with more recent
|
|
% breakthroughs in LLM based technologies to allow the intervention to be used
|
|
% in a wider variety of programming environments and contexts.
|
|
%
|
|
% I then plan to evaluate its impact on self-efficacy and potentially other
|
|
% outcomes. \todo{Figure out what those outcomes are.}
|
|
%\end{frame}
|
|
|
|
%\begin{frame}
|
|
% \frametitle{The Intervention}
|
|
% \todo{Talk about methodology and the type of intervention that's taking place.
|
|
% Why this method over other methods etc}
|
|
%
|
|
% This artefact attempts to replicate the channels of self-efficacy improvement
|
|
% that one on one support from an instructor, by targeting these factors in
|
|
% Bandura's theory \todo{cite}:
|
|
%\begin{itemize}
|
|
% \item Mastery experiences (Experiencing more success in coding tasks).
|
|
% \item Verbal encouragement.
|
|
% \item Guided support.
|
|
%\end{itemize}
|
|
% \todo{Check the previous itemisation isn't mistaken}
|
|
%
|
|
% The agent will use techniques such as personification \todo{cite gidget} and
|
|
% an AI models ability to translate errors into more accessible descriptions
|
|
% replicate an instructors ability to help a student understand knowledge they
|
|
% may be missing in terms of the language of a compiler. The model will also
|
|
% focus on guiding the student towards the solution in a structured manner such
|
|
% as in \todo{cite structured debugging intervention paper}.
|
|
%
|
|
% The goal with this artefact is to as closely as possible replicate the
|
|
% experience and self-efficacy improvements that come with 1 on 1 tuition
|
|
% \todo{cite}, so that these benefits can be scaled up. And help seeking
|
|
% behaviours be made more available.
|
|
%\end{frame}
|
|
|
|
\begin{frame}
|
|
\frametitle{The Intervention}
|
|
I plan to create an artefact that uses scaffolding and personification in an
|
|
attempt to improve students self-efficacy \cite{Bandura1977} and reduce
|
|
cognitive load \cite{Sweller1988}.
|
|
\newline
|
|
|
|
This will be done by walking a student through any issues a real programming
|
|
environment can throw at them with the use of a conversational AI agent based
|
|
avatar, which will break down the process of solving debugging issues in their
|
|
own real world programming environment.
|
|
\newline
|
|
|
|
The key feature of the agent being that it has been created with pedagogic
|
|
strategy in mind, namely scaffolding and personality to see if AI can help
|
|
fill these gaps when structured effectively.
|
|
\end{frame}
|
|
|
|
\begin{frame}
|
|
\frametitle{Methodology}
|
|
Due to the nature of using a software tool to improve outcomes, an
|
|
intervention based experiment becomes necessary \cite[p.~242]{Coe2025}. The
|
|
projects' reliance on AI should also adhere to guidelines such as
|
|
CONSORT-AI \cite{Liu2020} in order to remain reproducible and transparent.
|
|
\newline
|
|
|
|
If ethical issues surrounding the withholding of resources from some students
|
|
that were made available to others even at random wasn't a worry, a random
|
|
selection of student participants would be the best approach to reduce bias
|
|
\cite[p.~245]{Coe2025}.
|
|
\newline
|
|
|
|
To counter this issue will I will be measuring results from a cohort before
|
|
the intervention has been fully developed and comparing results to a cohort
|
|
that has then since gained access to the artefact in a longitudinal study
|
|
\cite[p.~224]{Coe2025}.
|
|
\begin{itemize}
|
|
\item Year 1 -- Pre-intervention Cohort
|
|
\item Year 2 -- intervention Cohort
|
|
\end{itemize}
|
|
\end{frame}
|
|
|
|
\begin{frame}
|
|
\frametitle{Data Collection}
|
|
%TODO come back to this
|
|
Self-efficacy is a psychological self-perception and as such must be collected
|
|
from communicating with the participants.
|
|
\newline
|
|
|
|
There are however several methods we can use to quantify qualitative data
|
|
through tools like Likert scales \cite{likert1932} and thematic analysis
|
|
\cite{Braun2006} of feedback.
|
|
\newline
|
|
|
|
Both of these methods require careful planning to avoid the influence of bias.
|
|
I've interacted with them before in papers I've contributed to in the past
|
|
\cite{Mitchell2021, Mitchell2022}, and I am confident in the GA's ability to
|
|
guide me through the process of making sure I collect this data correctly.
|
|
\end{frame}
|
|
|
|
%\section{Technology stacks and ethical considerations}
|
|
|
|
%\begin{frame}
|
|
% \frametitle{Key challenges in the study}
|
|
%\end{frame}
|
|
|
|
\begin{frame}
|
|
\frametitle{AI Ethical Concerns}
|
|
This research immediately falls into at least medium risk in Falmouth
|
|
University's ethics policy as it involves human participants. Due to the
|
|
nature of interacting with generative AI addition measures will be necessary.
|
|
\begin{itemize}
|
|
\item We should avoid giving AI access to children and vulnerable people.
|
|
\item We should make sure to use ethically trained AI models where possible.
|
|
\item Furthermore, we should follow CONSORT-AI \cite{Liu2020} to keep this
|
|
research transparent and reproducible.
|
|
\end{itemize}
|
|
|
|
\end{frame}
|
|
|
|
%\section{Me, My Skills, And why I want to study at Falmouth University}
|
|
% 1 4 9? 10?
|
|
|
|
\begin{frame}
|
|
\frametitle{Who am I?: Relevant Job Experience}
|
|
\begin{itemize}
|
|
\item Senior Technician at Falmouth Universities Games Academy, specialising
|
|
in Computing.
|
|
\begin{itemize}
|
|
\item Previously I was an Associate Lecturer and an e-Learning Developer
|
|
\end{itemize}
|
|
\item I have worked on streaming interactive 3D Architectural Visualisation
|
|
experiences with \href{https://www.amutri.com/}{Amutri Ltd}
|
|
\cite{AmutriLtd2025}. \item And I have worked with live audio streaming for
|
|
podcasts in a former startup called Ramble that attempted to live stream
|
|
podcasts and call in radio shows.
|
|
\end{itemize}
|
|
\end{frame}
|
|
|
|
\begin{frame}
|
|
\frametitle{Who am I?: Relevant Research}
|
|
I have been contributed to the following papers with academic staff from
|
|
Falmouth University previously:
|
|
\begin{itemize}
|
|
\item Student Perspectives on the Purpose of Peer Evaluation During Group
|
|
Game Development Projects \cite{Mitchell2021}.
|
|
\item An Exploratory Analysis of Student Experiences with Peer Evaluation in
|
|
Group Game Development Projects \cite{Mitchell2022}.
|
|
\end{itemize}
|
|
\end{frame}
|
|
|
|
\begin{frame}
|
|
\frametitle{Why I want to study here}
|
|
\begin{itemize}
|
|
\item I already work here delivering content to the students and feel that
|
|
the computing departments research goals and my work already align really
|
|
well.
|
|
\item The research area I'm applying to perform research within is the work
|
|
I am already performing at this institution.
|
|
\item It will be a good reason to continue to develop new software keeping
|
|
up with novel techniques which can also influence my teaching.
|
|
\end{itemize}
|
|
\end{frame}
|
|
|
|
\begin{frame}[allowframebreaks]
|
|
\frametitle{References}
|
|
{\tiny
|
|
\bibliography{references.bib}
|
|
}
|
|
\end{frame}
|
|
|
|
\end{document}
|