Intraoperative 2D/3D Image Registration via Differentiable X-ray Rendering

Speaker: Vivek Gopalakrishnan , CSAIL MIT

Date: Friday, December 01, 2023

Time: 4:00 PM to 4:30 PM Note: all times are in the Eastern Time Zone

Public: Yes

Location: 32-G882 (Hewlett Room)

Event Type: Seminar

Room Description:

Host: Thien Le, CSAIL MIT

Contact: Thien Le,

Relevant URL:

Speaker URL: None

Speaker Photo:

Reminders to:,,

Reminder Subject: TALK: Intraoperative 2D/3D Image Registration via Differentiable X-ray Rendering

Abstract: We investigate the camera pose estimation problem in the context of 2D/3D medical image registration. The application is to align 2D intraoperative images (e.g., X-ray) to a patient's 3D preoperative volume (e.g., CT), helping provide 3D image guidance during minimally invasive surgeries. We present a patient-specific self-supervised approach that uses differentiable rendering to achieve the sub-millimeter accuracy required in this context. Some of aspects of our work that may be of interest to the broader ML community include
- How do you exactly compute the rendering equation for differentiable ray tracing through a voxel grid?
- What is the optimal representation of rotations and translations when using gradient descent to optimize poses?
- What is the optimal image loss function that achieves robust image registration while still being fast enough to use in real time?

Speaker bio: Vivek is a 3rd year PhD student in Polina Golland's group broadly interested in 3D computer vision problems across science and medicine.

Research Areas:
Graphics & Vision

Impact Areas:
Health Care

See other events that are part of the ML Tea.

Created by Thien Le Email at Monday, November 27, 2023 at 4:24 PM.