Oral exam “chalk talk” case studies
Case study #1 (2023)
Two weeks before the meeting
The student sent out a proposal document to their thesis committee. The document began with an NIH F31-style Specific Aims page, followed by a 6 page research strategy split into significance (0.5 pages), experimental approach (1.5 page), preliminary results (1 page), Aim 1 (1 page), Aim 2 (1 page), Aim 3 and timeline (1 page).
In the two weeks leading up to the meeting
The student practiced the chalk talk by first writing out the proposal on a tablet to determine what could be written out and what needed to be in the powerpoint (3 slides), and how much detail to include so that the talk was ~30 minutes long. They reserved a room with a large whiteboard so that they would not need to erase during the talk. They practiced once per day for about a week in the exam room so that the organization on the whiteboard would be consistent and could serve as a visual reminder of the discussion points. The talk gradually became more polished until the last couple of practice talks were smooth with details ironed out. They primarily practiced alone, once with their labmate to practice being interrupted with questions.
Right before the meeting
The student writes out main title and aim titles on the whiteboard. The whiteboard in this case is very wide (four panels), and so the student writes at the top of each panel (respectively) the titles of their thesis, Aim 1, Aim 2, and Aim 3.
During the meeting (1): interviewing student and advisors
At the start, the advisors leave. The student talks to the rest of the committee about how things are going from their perspective.
Advisors return, student leaves. Advisors talk to the rest of the committee about how things are going from their perspective.
During the meeting (2): the chalk talk
The chalk talk begins. Under the title column, student describes the background and motivation for the study. (In this case, through a series of bullet points: Why study infectious disease? Why study the genetics of infectious disease? Why use mice?)
Referring to diagrams on slide 1, student describes the study population(s) used in the project. (This was a genetics project using several types of mouse population.)
Referring to slide 2, student describes preliminary data motivating the main study (in this case, differential response to viral susceptibility across different mouse strains.) Student sketches the plot showing this differential response on the whiteboard. Student then writes out the follow-up experimental design that forms the main data source for Aims 1 and 2, describing the factors manipulated and phenotypes collected.
Under Aim 1 heading, student writes out the traditional statistical model for analyzing this type of experimental data and explains the shortcomings of this approach for these particular experiments and study question, that is, the student describes the problem to be addressed. Student writes out the proposed model for Aim 1a and shows a picture from slide 2 illustrating preliminary results from running this model in the real data, demonstrating that the targeted phenomenon exists and would be hard to identify using traditional approaches.
Committee interrupts with questions: some questions ask to clarify parts of the plot (ex: questions about the axes – was the data transformed?), data, and model; some are “what if” type of questions.
Still under the Aim 1 heading, the student describes motivation and proposed model for Aim 1b, and then the motivation for Aim 1c. Aim 1c does not yet have a proposed model or solution, so the student describes avenues that will be explored, listing three directions.
Committee asks questions about these and makes several suggestions about variations of the method, plus observations explaining certain relevant data relationships to the student. The student takes notes on these suggestions.
Under the Aim 2 heading, the student describes the problem with the current state of models typically used to analyze this type of data, and opportunities for improvements in analysis. Student describes in some detail an existing method of another lab that they will apply to this problem.
Committee questions whether the problem can be approached a different way. Committee members prompt the student to speak about other features of the data that help motivate the problem, reminding the student of other points that the student then talks about. More discussion between committee members and student follows, getting the student to think about particular analytic principles, helping the student understand minor gaps.
Still under Aim 2, the student motivates Aim 2b and explains the proposed approach, which is the application of an existing method from the lab to this problem. This method is not described in as much detail as 2a because it is more complex and time is limited.
Committee member asks question about limitations of the experimental data. (This question could have been asked earlier but only just occurred to the committed member.)
Under the Aim 3 heading, the student describes experimental data from another study which motivates this aim. This aim is independent of Aims 1 & 2. Student writes tutorial diagrams explaining the phenomenon that motivates a different, connected, experiment. Then describes this other experiment and its advantages, but also how its analysis can be more complex — how standard analyses of this experiment can be suboptimal in the presence of commonly observed biological effects, causing important signals to be missed. Proposes a modeling solution.
Committee asks lots of questions about model, including asking about the pitfalls of the proposed solution. A committee member asks about terminology, suggests more precise language, and discussion ensues.
The committee has no more questions and the student has finished describing the aims, so the chalk talk ends.
During the meeting (3): committee discusses the chalk talk
The student leaves. The committee discusses how the student did and whether they passed. The committee calls the student back in and tells them whether they passed and provides feedback on the student’s chalk talk (all positive in this case).
After the meeting
The meeting chair writes up any to-do items from the meeting as part of the standard oral proposal paperwork.
Case study #2 (2023)
The student entered the room >30 minutes early, wrote organizing text and drawings on the whiteboard, and connected their laptop to the room AV. The private committee conversations with the mentors and then the student took ~15 minutes. There were no concerns that required an extended discussion.
The student had memorized an introduction and used a brief outline on the board to describe the significance. Questions began almost immediately, within the first 5 minutes of the presentation. The student answered some questions by referring to the literature. Some questions about the significance were about suitability of the approach for the questions being addressed, methods available but not being used to accomplish the goals, the biological principle underlying the study, and examples of the principle not brought up in the written proposal. Conversations about the significance took ~35 minutes.
The student kept the discussion on track by referring to outlines on the board. Some questions about aims were about the rationale, study samples, preliminary data, study design choices and benefits of alternatives not selected, methods, and interpretation and impact of expected results, some of which were specific. Conversations about Aim 1 took 35 minutes, stopping only because the committee was concerned about time. Conversations about Aim 2 took 20 minutes, stopping to allow ~15 minutes for committee to discuss and report back to the student, who passed.
Slides showed 1) aim 1 design and preliminary data; 2) aim 2 preliminary data; and 3) more aim 2 preliminary data.
Case study #3 (2023)
The student entered the room ~45 minutes early, wrote text and some outlines of figures on the whiteboard, and connected their laptop to check microphone and speaker. The private committee conversations with the mentors and then the student took ~15 minutes. Some of the committee members joined by zoom and so the student pointed their laptop camera at the whiteboard so all members could see the writing.
The student began with a high-level description of the motivation of the work. The student directed the attention of the committee to a central conceptual diagram of their proposal (the outlines of which had been sketched out beforehand) at the outset of their talk, filling in some example lines into plots to diagram different types of genetic effects that their method would aim to detect. The student then sketched out (with simple formula) some existing methods to detect these genetic effects. As they went through the description of existing methods, they described some limitations of the current approaches, which they would attempt to improve with their proposed methods. There was a parallel structure between the limitations of existing methods and the bullets describing the aspects of the proposed work (mostly in Aim 1). The student then outlined some particular genetics acronyms/terms they would use in the rest of the talk. This part of the prelim took ~10-15 minutes.
The student then walked through their 3 aims. Each Aim had pre-written details on the approach, including some simple formula and simple diagrams. In Aim 1, the student completed some of the formula with pertinent details as they spoke. The student also completed the key steps of the approach with numbered items, e.g. step 1, step 2, step 3 as they spoke. Committee members asked some questions during the description of Aim 1 approach, in particular about the interpretation of the model parameters, and how it relates to practical aspects of analysis of genetic datasets. The student spent a few minutes talking about computational aspects and how their model could be fit efficiently. Finally, the student showed an example on the whiteboard (completing a sketch of a plot that had been pre-drawn) of how their new method provided additional interpretable of results compared to existing methods.
About 25 minutes into description of Aim 1, the student showed the first slide, which had an example of some experimental data (a plot of data from a single gene) where the student’s proposed method would be a better fit to the data. The student then showed the second slide which contained simulation results for Aim 1.
About 40 minutes into the prelim, the student transitioned to description of Aim 2. This part of the prelim was similarly structured and involved similar questions from committee as Aim 1, however, the description was at a higher level. There was more student-committee conversation in the presentation of Aim 2, and this part took about 15 minutes before student transitioned into Aim 3. Aim 2 and Aim 3 were diagrammed on the same section of the white board, taking up much less space than was devoted to Aim 1. The student concluded the presentation after about 1 hour from the end of the committee/advisor/student conversations. Some committee questions lasted about 15-20 minutes, and then there was a discussion of the timeline for completing the aims.
Slides showed 1) example experimental data for a single gene 2) preliminary results for Aim 1 on simulated data.
Case study #4 (2024)
Before the meeting: The room has four whiteboard panels. The student has written at the top of each one:
-
- Panel 1: the proposal title
- Panel 2: Aim 1 title
- Panel 3: Aim 2 title
- Panel 4: Aim 3 title, with 3 subaim titles splitting the board into thirds.
The meeting starts. Committee asks student to step out of the room so they can speak with advisor. Five minutes later, advisor steps out of the room so committee can talk with student.
The exam starts. Student describes the overall theme of the thesis, writing under the title in panel one. Student lays out the three aims in abbreviated fashion to show the connections between them. In doing this, student refers to their first slide, a diagram that shows the composition and relations between three types of genetic populations under study.
Under Aim 1 (panel 2), student draws a tree diagram to describe the design of the experiment that gave rise to the data under study and the existing analysis pipeline. Student illustrates example results from one part of the analysis using their 2nd powerpoint slide.
Committee member (CM) interrupts to ask a question, student responds, member provides an alternative explanation, student agrees to look into it later.
Student describes extending an existing method. CM asks for more details about the data being modeled. Discussion ensues.
Under Aim 2 (panel 3), student describes the statistical model and the problem that the model is intended to solve. Then student gives examples in the data that illustrate a weakness of the model, where the model gives nonsense results; student describes their fix for this weakness, a more robust version of the statistical model.
CM concerned about the publication of Aim 1 paper holding back the submission of Aim 2 paper. Discussion. Committee volunteer their own data that student can use for the Aim 2 paper to obviate publication dependence.
Under Aim 3 (panel 4), student describes Aim 3a experimental design and preliminary work by other lab members, including preliminary results. [Aim 3 has not been started yet; this is work that is planned.] Student goes on to describe an additional, new and larger dataset of the same type, and describes how they will see if the results replicate. Lots of opinions and suggestions from committee.
Student describes Aim 3b experimental design, the population under study and the justification for using it, and what scientific questions are being asked in general and by the student. CM1 provides additional justification for using the study population but also is concerned about the sample size; CM2, who is designed the study answers CM1’s questions about power.
Committee asks student to leave the room. Committee talks about impact, feasibility of the project, comments on the proposal document, comments on the preparedness of the student in the oral exam. Committee decides on a pass. Committee suggests next committee meeting to follow up in 9 months (because previous meeting was more than a year ago).