Insights from teaching artificial intelligence to medical students in Canada

Clinical artificial intelligence (AI) applications are rapidly developing but existing medical school curricula provide limited teaching covering this area. Here we describe an AI training curriculum we developed and delivered to Canadian medical undergraduates and provide recommendations for future training.


Curriculum
The most recent curriculum, from April 2021, is summarized in Table 1 and includes the targeted learning objectives for each topic. The workshop was designed for a novice level of technical proficiency, with no mathematics beyond a first-year undergraduate medical course. The curriculum was designed by 6 medical students and 3 instructors with engineering graduate degrees. Engineers proposed AI theory for teaching and medical students filtered for clinically relevant material.
The workshop consisted of lectures, case studies, and guided programming. In the first lecture, we reviewed select data analytics concepts from biostatistics including data visualization, logistic regression, and comparing descriptive versus inferential statistics. Although data analytics is fundamental to AI, we excluded topics

Box 1 | Glossary
Data Analytics: A field of study in statistics where patterns in data are analyzed, processed, and communicated to identify meaningful patterns in data. Data Mining: The process of identifying and extracting data. In the context of artificial intelligence, this is commonly in large quantities with multiple variables for each sample.
Debugging: The processing of finding and resolving unintentional errors in programs. Dimensionality Reduction: The process of transforming data with many individual features to a lesser number of features while retaining significant properties of the original dataset. Feature (in the context of artificial intelligence): A measurable property of a sample. Commonly used interchangeably with "attribute" or "variable". Fourier Transformation: A technique to convert a periodic signal to individual weighted sinusoids. Gradient Activation Map: A technique for interpreting artificial intelligence models, particularly convolutional neural networks, where the optimization process in final section of the network in analyzed to identify regions of the data or image that have high predictivity. Standard Models: Existing artificial intelligence models that have been previously trained to perform a similar task.
Testing (in the context of artificial intelligence): Observing a model performing a task with data it has not been previously exposed to.
Training (in the context of artificial intelligence): Exposing a model to data and resulting outcomes for the model to adjust its internal parameters to optimize its ability to perform the task with new data.
Vector: An array of data. In machine learning, each element in the array is commonly an unique feature for the sample.
such as data mining, significance tests, or interactive visualizations. This was due to time constraints and because several senior students had previous biostatistics training and were keen to cover more unique machine learning topics. The subsequent lectures presented current state-of-the-art methods and discussed AI problem formulation, strengths and limitations of AI models and model validation. Lectures were reinforced with case studies from the literature and from existing AI devices. We emphasized the skills needed to assess model performance and feasibility for a clinical problem, including understanding limitations of current AI devices. For example, we guided students in interpreting a pediatric head trauma guideline by Kupperman et al. 5 , where an AI decision tree algorithm was implemented to determine if computed tomography scanning was beneficial based on a physician's examination. We highlighted that this is a common example of AI providing predictive analytics for physicians to interpret, rather than a physician replacement.
In guided programming examples, available open-source (https://github.com/ubcaimed/ubcaimed.github.io/tree/master/ programming_examples), we demonstrated how to conduct exploratory data analysis, dimensionality reduction, loading a standard model, training, and testing. We used Google Colaboratory notebooks (Google LLC, Mountain View, California), which allowed execution of Python code from web browsers. An example of a programming exercise is summarized in Fig. 2. The exercise involved predicting malignant tumors using the Wisconsin Breast Imaging Open Dataset 6 with a decision tree algorithm.

Challenges
We identified four main challenges during the training: 1. Heterogeneity of Prior Knowledge: Our participants varied in mathematical proficiency. For instance, students with advanced technical backgrounds sought in-depth content such as how to perform Fourier feature transformations. However, it was not feasible to discuss Fourier algorithms to the class as this required advanced signal processing knowledge. 2. Attendance Attrition: There was reduced attendance in subsequent sessions, particularly with the online format. A solution could be to track attendance and provide a certificate of completion. Medical schools have been known to provide recognition on student transcripts for extracurricular academic activities, which may incentivize completion. 3. Curricular Design: As AI spans numerous subfields, selecting core concepts at an appropriate depth and breadth was challenging. For instance, an important topic is the bench-to-bedside continuum for AI tools. Though we introduced data preprocessing, model construction, and validation, we did not include topics such as mining big data, interactive visualizations, or running an AI clinical trial 7 in favor of focusing on concepts most unique to AI. Our guiding principle was to train literacy over proficiency. For instance, understanding how a model processes input features is important for interpretability and one method is • Problem definition, data collection, model development (1,2) • Descriptive, inferential, predictive statistics and its limitations (1) • Regulatory approval (2,3) • Feature spaces, class balancing, normalization, continuous or discrete data (1) • Roles of physician, engineers, allied healthcare workers (2,3) The Machine Learning Pipeline Programming: Genoa Wine Data • Feature spaces, data cleaning (1) • Exploratory data analysis: looking for outliers, class balance, feature data types (1,2) • Feature selection methods, feature importances, data augmentation (1) • with gradient activation maps, which visualize which region of data is predictive. However, this requires multivariate calculus and was not feasible to introduce 8 . Developing a shared terminology proved challenging as we struggled to explain how to manipulate data as vectors without mathematical formalism. We noticed different terms shared meanings, such as describing a "feature" as a "variable" or "attribute" in epidemiology. 4. Knowledge Retention: It remains to be seen how well participants retain knowledge as there are limited opportunities to apply AI. Medical school curriculums frequently rely on spaced repetition where knowledge is consolidated in practical rotations 9 , which may be applicable to AI education as well.

Successes
We observed four main successes: 1. Proficiency was targeted over literacy: The depth of material was designed without rigorous mathematics, which has been a perceived challenge in launching clinical AI curricula 10 .
In programming examples, we used template programs to allow participants to fill in blanks and run software without requiring knowledge of setting up a full programming environment. 2. Concerns about AI were addressed: There is a common concern that AI might replace certain clinical duties 3 . To address this, we explained the limitations of AI, including that nearly all AI technologies approved by regulatory bodies require physician supervision 11 . We also emphasized the importance of bias, where algorithms are susceptible to systematic error, especially if the dataset is not diverse 12

Conclusions
AI is highly technical, with foundational concepts involving mathematics and computer science. Training medical personnel to understand AI poses unique challenges relating to content selection, clinical relevance, and method used to teach the material. We hope that our insights gained from carrying out AI education workshops may assist future educators of innovative approaches to integrate AI into medical education.