Introducing an Open-source Adaptive Tutoring System to Accelerate Learning Sciences Experimentation

This hands-on tutorial at L@S 2023 will cover rapid experimentation, content authoring, and LMS integration of an open-source adaptive tutor, OATutor. This half-day tutorial will be held on July 20th in-person in Copenhagen, Denmark.

Background

Learning @ Scale has embraced movements that spread access to education through open and free platforms of learning. In this tutorial, we introduce OATutor (presented at CHI’23 [1]), the field’s first free and open-source adaptive tutoring system based on ITS principles and designed for rapid experimentation. The MIT-licensed platform can be configured and deployed to git-pages in only a few clicks and supports BKT mastery-based adaptive problem selection. We demonstrate, with hands-on tutorials, how the system can be used to rapidly run A/B experiments, analyze the data, and publish the entire tutor, content, and analysis scripts to github to facilitate unprecedented ease of replication and transparency, as demonstrated in a recent study comparing ChatGPT generated hints to human-tutor hints [2]. Our four-part tutorial will include how to add lessons to the system and link to them from assignments in a MOOC platform or LMS via LTI. The structured JSON format of the four CC BY courses worth of content released with OATutor opens up avenues for researchers to apply new and existing educational data mining and NLP techniques (e.g., KC tagging) and rapidly evaluate the impact of any subsequent changes on learners with an experiment.

Agenda

Introduction (20 Minutes)

• Introductions, Motivations (5 minutes)
• Overview of the system (15 minutes)

Research Capabilities Tutorial (1 hour 45 Minutes)

• Demonstrate how to conduct A/B testing and analyze experiment data (40 minutes)
• Re-tagging content with different skills using ML (20 minutes)
• Giving a personalized hint using a backend server (45 minutes)

Content and LMS Tutorial (25 Minutes)

• Introduce content tools with short demonstration (10 minutes)
• Introduce LTI tool and demonstrate integration with Canvas (10 minutes)
• Teacher dashboard integration showcase (5 minutes)

Hands-on (30 Minutes)

• OPTION 1: Content spreadsheet editing + pipeline (seeing it in the tutor)
• OPTION 2: Data analysis of log data from OAT experiment
• OPTION 3: Creating a new skill model

Content

Tutorial on OATutor Research Capabilities

• A/B testing demonstration
• ML content tagging
• Personalized hints through backend server

Tutorial on OATutor Content Creation

• Creation of content
• How to get content onto the system
• Activity where participants create content

OATutor LMS Integration

• Canvas integration demonstration
• Teacher dashboard showcase
• Activity where participants create an OATutor lesson on Canvas

Intended Audience

Researchers who wish to conduct A/B testing of computer tutoring and adaptive learning approaches, educators who wish to integrate adaptive content in their courses, HCI researchers, adaptive learning and AI researchers who wish to build upon the open-source tutor and extend its capabilities (e.g., to support programming, AI facilitated supports at scale).

Requirements for Participation:

Participants should bring a laptop to participate in the hands-on exercises. Some exercises will require no background knowledge, such as in the Content Creation Tutorial, while others will benefit from minimal coding experience.

Organizers