Gaurav Jain's profile picture.

Hi! I am a Ph.D. student in the computer science department at Columbia University, advised by Brian A. Smith in the Computer-Enabled Abilities Lab (CEAL).


AI-driven Interactive Systems for Access to Visual Experiences

I design, build, and study AI-driven systems that apply existing AI techniques in creative ways to grant people new abilities. My research revolves around developing systems that address accessibility problems, with a focus on enabling access to visual experiences for people with visual impairments.

Using my systems, blind and low-vision (BLV) people can visualize the action in sports broadcasts for themselves rather than relying on other people’s descriptions (CHI 2023, UIST 2023). My systems embed AI within street intersections to give BLV people the ability to interact with street cameras for safe outdoor navigation (ASSETS 2023, UIST 2024).

To inform future systems, I also conduct qualitative studies aimed at understanding people's needs. In my recent work, I introduced the concept of "exploration assistance systems," that give BLV people the freedom to explore unfamiliar environments on their own, rather than being guided by turn-by-turn instructions (CSCW 2023). This paper received an Impact Recognition Award 🏆.


Human-Computer Interaction (HCI) 🧑‍💻, Accessibility👩‍🦯, Computer Vision & Deep Learning 🤖



News


Read More

Read Less

Selected Publications


For an updated list of articles, please visit my Google Scholar profile.


Street intersection as seen from a second floor camera view. A blind pedestrian is detected by the system and is crossing the street. A car is blocking the blind pedestrian’s path, and a mobile app provides them with a warning saying: Caution! Car 3 ft ahead.
StreetNav: Leveraging Street Cameras to Support Precise Outdoor Navigation for Blind Pedestrians

ACM UIST 2024 [Acceptance Rate: 24%]

Gaurav Jain, Basel Hindi, Zihao Zhang, Koushik Srinivasula, Mingyu Xie, Mahshid Ghasemi, Daniel Weiner, Sophie Ana Paris, Xinyi Xu, Michael Malcolm, Mehmet Turkcan, Javad Ghaderi, Zoran Kostic, Gil Zussman, Brian A. Smith

Paper Project Page Talk 30s Video Preview

Blind man sitting in a chair wearing headphones, with a monitor behind him playing a tennis broadcast video.
Front Row: Automatically Generating Immersive Audio Representations of Tennis Broadcasts for Blind Viewers

ACM UIST 2023 [Acceptance Rate: 25.1%]

Gaurav Jain, Basel Hindi, Connor Courtien, Xin Yi Therese Xu, Conrad Wyrick, Michael Malcolm, Brian A. Smith

Paper Project Page Talk 30s Video Preview

A two-part figure showing overview of street camera-based navigation system. On the left is a screenshot of the smartphone iOS app. On the right is the second floor camera view of a street intersection.
Towards Street Camera-based Outdoor Navigation for Blind Pedestrians

ACM ASSETS 2023 (Posters)

Gaurav Jain, Basel Hindi, Mingyu Xie, Zihao Zhang, Koushik Srinivasula, Mahshid Ghasemi, Daniel Weiner, Xinyi Xu, Sophie Ana Paris, Chloe Tedjo, Josh Bassin, Michael Malcolm, Mehmet Turkcan, Javad Ghaderi, Zoran Kostic, Gil Zussman, Brian A. Smith

Paper Demo Video

A blind person watching tennis broadcast on TV with a sighted friend.
Towards Accessible Sports Broadcasts for Blind and Low-Vision Viewers

ACM CHI 2023 (Extended Abstracts) [Acceptance Rate: 34%]

Gaurav Jain, Basel Hindi, Connor Courtien, Xin Yi Therese Xu, Conrad Wyrick, Michael Malcolm, Brian A. Smith

Paper Project Page 30s Video Preview

Sketch of a large shopping complex with three things highlighted: area shape, layout, and collaboration between individuals.
“I Want to Figure Things Out”: Supporting Exploration in Navigation for People with Visual Impairments

ACM CSCW 2023

Trophy icon Impact Recognition Award

Gaurav Jain, Yuanyang Teng, Dong Heon Cho, Yunhao Xing, Maryam Aziz, Brian A. Smith

Paper Project Page Blog

Overview of the proposed deep learning architecture.
Attention-Net: An Ensemble Sketch Recognition Approach Using Vector Images

IEEE Transactions on Cognitive and Developmental Systems 2022

Gaurav Jain*, Shivang Chopra*, Suransh Chopra*, A. S. Parihar (*equal contribution)

Paper

A 2 by 3 grid of target numerical phantoms used for quantitative evaluation of the proposed method.
Deep Neural Network Based Sinogram Super-resolution and Bandwidth Enhancement for Limited-data Photoacoustic Tomography

IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control 2020

Gaurav Jain*, Navchetan Awasthi*, S. K. Kalva, Manojit Pramanik, P. K. Yalavarthy (*equal contribution)

Paper Code

Overview of the proposed architecture showing key-based feature extraction and adaptive weighted graph fusion.
Adaptive Weighted Graph Approach to Generate Multimodal Cancelable Biometric Templates

IEEE Transactions on Information Forensics and Security 2020

G. S. Walia, Gaurav Jain, Nipun Bansal, Kuldeep Singh

Paper

A 3 by 4 grid of human-made sketches with a color bar on the right. The first, second, and third row show bat (animal), turtle, and ant sketches, respectively.
TransSketchNet: Attention-based Sketch Recognition Using Transformers

ECAI 2020 (Short Paper)

Gaurav Jain*, Shivang Chopra*, Suransh Chopra*, A. S. Parihar (*equal contribution)

Paper Poster

Contact


If you’re interested in my work and wish to discuss anything, feel free to email me (gaurav [at] cs [dot] columbia [dot] edu)!