369 lines
14 KiB
TeX
369 lines
14 KiB
TeX
\documentclass[aspectratio=169]{beamer}
|
||
|
||
\usepackage{tabularx}
|
||
\usepackage{booktabs}
|
||
\usepackage{csquotes}
|
||
\usepackage{graphicx}
|
||
|
||
\usepackage{xcolor}
|
||
\newcommand\todo[1]{\textcolor{red}{#1}}
|
||
|
||
% section frame
|
||
\AtBeginSection[]{
|
||
\begin{frame}
|
||
\vfill
|
||
\centering
|
||
\begin{beamercolorbox}[sep=8pt,center,shadow=true,rounded=true]{title}
|
||
\usebeamerfont{title}\insertsectionhead\par%
|
||
\end{beamercolorbox}
|
||
\vfill
|
||
\end{frame}
|
||
}
|
||
|
||
% URL coloring
|
||
\hypersetup {
|
||
colorlinks=true,
|
||
linkcolor=blue,
|
||
filecolor=magenta,
|
||
urlcolor=blue,
|
||
pdftitle={PhD Interview},
|
||
pdfpagemode=FullScreen,
|
||
}
|
||
|
||
% Referencing
|
||
\bibliographystyle{IEEEtran}
|
||
\setbeamertemplate{bibliography item}{\insertbiblabel}
|
||
|
||
\title{Can the use of a virtual avatar and scaffolding with AI agents improve
|
||
first-stage programming students' debugging ability?}
|
||
%\subtitle{aa}
|
||
\date{23rd of April 2026}
|
||
\author{Warwick New}
|
||
|
||
\begin{document}
|
||
\maketitle
|
||
|
||
%\section{What is the problem I'm trying to solve?}
|
||
|
||
\begin{frame}
|
||
\frametitle{What is the problem space?}
|
||
Novices tend to struggle with programming fundamentals, finding the basics
|
||
quite difficult to learn, and so introductory courses tend to have poor rates
|
||
of retention.
|
||
\newline
|
||
|
||
One factor influencing retention is students’ self-perception of their
|
||
programming ability. Negative perceptions can reduce retention and increase
|
||
the likelihood that students drop out in their first stage.
|
||
\cite{lewis2011}.
|
||
\newline
|
||
|
||
A particular point of frustration for novice programmers is attempting to
|
||
debug without thinking of the problem critically \cite[p.~23]{vickers2008}.
|
||
\newline
|
||
|
||
Previous interventions in this space have shown promise such as Gidget
|
||
\cite{Lee2014} that has used personification of compilers to great effect and
|
||
Metacodenition \cite{Pechorina2023} which helps the student create a mental
|
||
model about actions to take when a bug appears.
|
||
\newline
|
||
|
||
Recent literature has begun exploring the use of LLMs but out of the box,
|
||
found them in most cases to be worse than just reading the compiler.
|
||
\cite{Pechorina2023}.
|
||
%Recent literature has identified this as a problem space that AI
|
||
%can attempt to help resolve. Renzella et al's work \cite{renzella2025}
|
||
%demonstrates that there are aspects of learning that traditionally could only
|
||
%be fulfilled by an instructor outside highly controlled learning
|
||
%environments (like Gidget \cite{Lee2014}) that can be partially automated with
|
||
%generative AI. And therefore can be made more available to students outside
|
||
%timetabled sessions.
|
||
%%in this case reducing the cognitive load \cite{sweller1988} of reading and understanding compiler messages by providing more human-readable descriptions of compiler errors.
|
||
\end{frame}
|
||
|
||
%\begin{frame}
|
||
% \frametitle{What is the gap I've identified}
|
||
% \todo{Cite and Professionalise this slide}
|
||
% \begin{itemize}
|
||
% \item Much research into debugging education for students more recently is
|
||
% focused on generative AI models and some new paradigms that come from that
|
||
% \todo{\cite{}}. This is to be expected with a whole new paradigm in
|
||
% education in this area.
|
||
% \item Before this focus however much research into the area of first stage
|
||
% computer science debbugging was focused on teaching debugging in closed
|
||
% controlled environments \todo{\cite{}}, This allowed for experimentation
|
||
% with making fixing coding problems more accessible, in terms of compiler
|
||
% error messages and scaffolding of the processes a student might take when
|
||
% they ran into a problem.
|
||
% \item I think that we can take these classical educational tools and use
|
||
% them combined with more recent generative AI computing education
|
||
% technologies to.
|
||
% \begin{enumerate}
|
||
% \item Remove the need for locked down highly controlled debugging
|
||
% environments.
|
||
% \item Improve the human element to debugging messages and tools with how
|
||
% much more human the AI can appear to be to students.
|
||
% \end{enumerate}
|
||
% \end{itemize}
|
||
%\end{frame}
|
||
|
||
\begin{frame}
|
||
\frametitle{What is the Gap}
|
||
{\footnotesize
|
||
Two practices in helping students learn to debug have shown great promise when
|
||
it comes to improving self-efficacy and lowering programming anxiety. I
|
||
believe that we could use recent advances in AI to augment aspects of these
|
||
interventions.
|
||
\begin{itemize}
|
||
\item Personification of programming tools as fallible and encouraging, is
|
||
shown to have a positive impact on learning motivation and success as
|
||
shown in Gidget \cite{Lee2011}.
|
||
\item Scaffolding what steps to take when encountering a bug, improves
|
||
self-efficacy by lowering cognitive load as demonstrated Metacodenition
|
||
\cite{Pechorina2023}.
|
||
\end{itemize}
|
||
%Both of these works are restriceted to very specific environments limiting their application on real world coding projects.
|
||
\begin{figure}
|
||
\centering
|
||
\includegraphics[width=0.45\textwidth]{progprac}
|
||
\caption{Provided by Scott et al. \cite{Scott2014}}
|
||
\label{fig:question}
|
||
\end{figure}
|
||
This figure shows how programming self-concept influences programming anxiety
|
||
and increasing time on task, retaining more computing students.
|
||
}
|
||
\end{frame}
|
||
|
||
%\begin{frame}
|
||
% \frametitle{The fields this gap interacts with}
|
||
% \begin{itemize}
|
||
% \item Computer science education
|
||
% \begin{itemize}
|
||
% \item Specifically focusing on first stage debugging.
|
||
% \end{itemize}
|
||
% \item Psychology
|
||
% \begin{itemize}
|
||
% \item Student Self-efficacy. \todo{cite}
|
||
% \item Students engagement. (Time Spent working on programming tasks)
|
||
% \todo{Define engagement}
|
||
% \end{itemize}
|
||
% \item Artificial Intelligence
|
||
% \begin{itemize}
|
||
% \item Scaffolding of Generative AI queries to fit a more traditionally
|
||
% productive learning environment in a wide range of situations.
|
||
% \item Encourage personal reflection and action when attempting to solve
|
||
% an issue.
|
||
% \end{itemize}
|
||
% \end{itemize}
|
||
%\end{frame}
|
||
|
||
%\section{How I plan to address this space}
|
||
|
||
%\begin{frame}
|
||
% \frametitle{How am I addressing the gap}
|
||
% \todo{Talk about Gidget and offer scaffolding tools, then talk about how we
|
||
% can create a tool to aid intervention}
|
||
% I plan to create an intervention wherein I give students access to an AI tool
|
||
% that takes elements from previous interventions that align with the principles
|
||
% of improving self-efficacy in students, and combine it with more recent
|
||
% breakthroughs in LLM based technologies to allow the intervention to be used
|
||
% in a wider variety of programming environments and contexts.
|
||
%
|
||
% I then plan to evaluate its impact on self-efficacy and potentially other
|
||
% outcomes. \todo{Figure out what those outcomes are.}
|
||
%\end{frame}
|
||
|
||
%\begin{frame}
|
||
% \frametitle{The Intervention}
|
||
% \todo{Talk about methodology and the type of intervention that's taking place.
|
||
% Why this method over other methods etc}
|
||
%
|
||
% This artefact attempts to replicate the channels of self-efficacy improvement
|
||
% that one on one support from an instructor, by targeting these factors in
|
||
% Bandura's theory \todo{cite}:
|
||
%\begin{itemize}
|
||
% \item Mastery experiences (Experiencing more success in coding tasks).
|
||
% \item Verbal encouragement.
|
||
% \item Guided support.
|
||
%\end{itemize}
|
||
% \todo{Check the previous itemisation isn't mistaken}
|
||
%
|
||
% The agent will use techniques such as personification \todo{cite gidget} and
|
||
% an AI models ability to translate errors into more accessible descriptions
|
||
% replicate an instructors ability to help a student understand knowledge they
|
||
% may be missing in terms of the language of a compiler. The model will also
|
||
% focus on guiding the student towards the solution in a structured manner such
|
||
% as in \todo{cite structured debugging intervention paper}.
|
||
%
|
||
% The goal with this artefact is to as closely as possible replicate the
|
||
% experience and self-efficacy improvements that come with 1 on 1 tuition
|
||
% \todo{cite}, so that these benefits can be scaled up. And help seeking
|
||
% behaviours be made more available.
|
||
%\end{frame}
|
||
|
||
\begin{frame}
|
||
\frametitle{The Intervention}
|
||
I plan to create an artefact that uses scaffolding and personification in an
|
||
attempt to improve students self-efficacy \cite{Bandura1977} and reduce
|
||
cognitive load \cite{Sweller1988}.
|
||
\newline
|
||
|
||
This will be done by walking a student through any issues a real programming
|
||
environment can throw at them with the use of a conversational AI agent based
|
||
avatar, which will break down the process of solving debugging issues in their
|
||
own real world programming environment.
|
||
\newline
|
||
|
||
The key feature of the agent being that it has been created with pedagogic
|
||
strategy in mind, namely scaffolding and personality to see if AI can help
|
||
fill these gaps when structured effectively.
|
||
\end{frame}
|
||
|
||
\begin{frame}
|
||
\frametitle{Aims and objectives}
|
||
{\small
|
||
Question:
|
||
|
||
Can we use traditional pedagogic strategies to improve the
|
||
effectiveness of conversational AI agents in helping first stage
|
||
programming students believe in their ability to debug programs?
|
||
\newline
|
||
|
||
Aims:
|
||
\begin{itemize}
|
||
\item Improve first stage self perception on their ability to solve bugs.
|
||
\item Improve first stage students debugging ability. \newline
|
||
\end{itemize}
|
||
|
||
Objectives:
|
||
\begin{itemize}
|
||
\item Evaluate the changes in first stage programming students
|
||
self-efficacy.
|
||
\begin{itemize}
|
||
\item Both before and post intervention
|
||
\end{itemize}
|
||
\item Investigate factors that improve students self perceptions of their
|
||
programming ability.
|
||
\item To develop a tool that helps students improve at a rate that makes
|
||
their self perception of programming ability higher.
|
||
\item To develop a tool that students can fall back to when they can't seek
|
||
help elsewhere.
|
||
\end{itemize}
|
||
}
|
||
\end{frame}
|
||
|
||
\begin{frame}
|
||
\frametitle{Methodology}
|
||
Due to the nature of using a software tool to improve outcomes, an
|
||
intervention based experiment becomes necessary \cite[p.~242]{Coe2025}. The
|
||
projects' reliance on AI should also adhere to guidelines such as
|
||
CONSORT-AI \cite{Liu2020} in order to remain reproducible and transparent.
|
||
\newline
|
||
|
||
If ethical issues surrounding the withholding of resources from some students
|
||
that were made available to others even at random wasn't a worry, a random
|
||
selection of student participants would be the best approach to reduce bias
|
||
\cite[p.~245]{Coe2025}.
|
||
\newline
|
||
|
||
To counter this issue will I will be measuring results from a cohort before
|
||
the intervention has been fully developed and comparing results to a cohort
|
||
that has then since gained access to the artefact in a longitudinal study
|
||
\cite[p.~224]{Coe2025}.
|
||
\begin{itemize}
|
||
\item Year 1 -- Pre-intervention Cohort
|
||
\item Year 2 -- Intervention Cohort
|
||
\end{itemize}
|
||
\end{frame}
|
||
|
||
\begin{frame}
|
||
\frametitle{Data Collection}
|
||
%TODO come back to this
|
||
Self-efficacy is a psychological self-perception and as such must be collected
|
||
from communicating with the participants.
|
||
\newline
|
||
|
||
Several measurement instruments, typically questionnaires, have been developed
|
||
and validated for measuring self-beliefs in introductory programming contexts
|
||
(e.g., \cite{Scott2014}) and these are often complemented through thematic
|
||
analysis of post-intervention interviews \cite{Braun2006}.
|
||
%\newline
|
||
%
|
||
%Both of these methods require careful planning to avoid the influence of bias.
|
||
%I've interacted with them before in papers I've contributed to in the past
|
||
%\cite{Mitchell2021, Mitchell2022}, and I am confident in the GA's ability to
|
||
%guide me through the process of making sure I collect this data correctly.
|
||
\end{frame}
|
||
|
||
%\section{Technology stacks and ethical considerations}
|
||
|
||
%\begin{frame}
|
||
% \frametitle{Key challenges in the study}
|
||
%\end{frame}
|
||
|
||
\begin{frame}
|
||
\frametitle{AI \& Ethical Concerns}
|
||
This research immediately falls into at least medium risk in Falmouth
|
||
University's ethics policy as it involves human participants. Due to the
|
||
nature of interacting with generative AI addition measures will be necessary.
|
||
\begin{itemize}
|
||
\item We should avoid giving AI access to children and vulnerable people.
|
||
\item We should make sure to use ethically trained AI models where possible.
|
||
\item Furthermore, we should follow CONSORT-AI \cite{Liu2020} to keep this
|
||
research transparent and reproducible.
|
||
\end{itemize}
|
||
|
||
\end{frame}
|
||
|
||
%\section{Me, My Skills, And why I want to study at Falmouth University}
|
||
% 1 4 9? 10?
|
||
|
||
\begin{frame}
|
||
\frametitle{Who am I?: Relevant Job Experience}
|
||
\begin{itemize}
|
||
\item Senior Technician at Falmouth Universities Games Academy, specialising
|
||
in Computing.
|
||
\begin{itemize}
|
||
\item Previously I was an Associate Lecturer and an e-Learning Developer
|
||
\end{itemize}
|
||
\item I have worked on streaming interactive 3D Architectural Visualisation
|
||
experiences with \href{https://www.amutri.com/}{Amutri Ltd}
|
||
\cite{AmutriLtd2025}. \item And I have worked with live audio streaming for
|
||
podcasts in a former startup called Ramble that attempted to live stream
|
||
podcasts and call in radio shows.
|
||
\end{itemize}
|
||
\end{frame}
|
||
|
||
\begin{frame}
|
||
\frametitle{Who am I?: Relevant Research}
|
||
I have been contributed to the following papers with academic staff from
|
||
Falmouth University previously:
|
||
\begin{itemize}
|
||
\item Student Perspectives on the Purpose of Peer Evaluation During Group
|
||
Game Development Projects \cite{Mitchell2021}.
|
||
\item An Exploratory Analysis of Student Experiences with Peer Evaluation in
|
||
Group Game Development Projects \cite{Mitchell2022}.
|
||
\end{itemize}
|
||
\end{frame}
|
||
|
||
\begin{frame}
|
||
\frametitle{Why I want to study here}
|
||
\begin{itemize}
|
||
\item I already work here delivering content to the students and feel that
|
||
the computing departments research goals and my work already align really
|
||
well.
|
||
\item The research area I'm applying to perform research within is the work
|
||
I am already performing at this institution.
|
||
\item It will be a good reason to continue to develop new software keeping
|
||
up with novel techniques which can also influence my teaching.
|
||
\end{itemize}
|
||
\end{frame}
|
||
|
||
\begin{frame}[allowframebreaks]
|
||
\frametitle{References}
|
||
{\tiny
|
||
\bibliography{references.bib}
|
||
}
|
||
\end{frame}
|
||
|
||
\end{document}
|