Jaylin Herskovitz on a sailboat on a lake. She is looking to the right and smiling.

I'm a fourth year PhD student at the University of Michigan. I'm a part of the Human-AI (HAIL) Lab in the Computer Science & Engineering department advised by Anhong Guo.

I'm interested in creating augmented reality and AI-based tools for accessibility and collaboration. Specifically, the goal of my research is to help people create customized AI and sensing technologies to suit their unique needs. In my current work, I am developing tools for people with disabilities to create their own AI-based applications to improve the accessibility of tasks in their day-to-day lives.

News

May 2023:  Starting internship at Apple in Seattle!
June 2022:  XSpace, a toolkit for enabling spatially-aware distributed collaboration, accepted to ISS 2022
March 2022:  Project CollabAlly awarded Best Paper Honorable Mention at CHI 2022
December 2021:  Giving a talk on mobile AR accessibility at Cornell Tech's XR Access Initiative Seminar.
November 2021:  Serving on the organizing committee for ASSETS 2022 as a web design co-chair
October 2021:  Two demos CollabAlly (video) and ImageExplorer (video) appeared at ASSETS 2021
March 2021:  Awarded NSF's Graduate Research Fellowship, read about my research proposal here
October 2020:  Attending ASSETS 2020 to present our work on Making Mobile Augmented Reality Applications Accessible
May 2019:  Starting internship at Apple in Pittsburgh, Pensylvania with Jeff Bigham
May 2019:  Attending CHI 2019 to present our work from last summer on Opportunities for In-Home Augmented Reality Guidance
May 2018:  Starting internship at Microsoft Research in Redmond, Washington with Eyal Ofek and Adam Fourney

Publications

XSpace: An Augmented Reality Toolkit for Enabling Spatially-Aware Distributed Collaboration
Jaylin Herskovitz, Yi Fei Cheng, Anhong Guo, Alanson Sample, Michael Nebeling
ISS 2022
pdf ·  video ·  code
UbiChromics: Enabling Ubiquitously Deployable Interactive Displays with Photochromic Paint
Amani Alkayyali, Yasha Iravantchi, Jaylin Herskovitz, Alanson P. Sample
ISS 2022
An image of a blue bed in a hotel room. There are bounding boxes on the image around the bed and pillows. There are speech bubbles pointing to the bounding boxes that say "A white pillow" and "The bed is blue". There is also a pointing hand icon on the image, indicating that someone is touching the image to hear the speech.
Image Explorer: Multi-Layered Touch Exploration to Make Images Accessible
Jaewook Lee, Jaylin Herskovitz, Yi-Hao Peng, Anhong Guo
CHI 2022 Paper & ASSETS 2021 Demo
paper pdf ·  demo pdf ·  ACM DL ·  full video ·  30s preview ·  talk
A screenshot of Google Docs with various annotations. The annotations describe features of Collab Ally, and say "Visual Cue A: Collaborators", "Visual Cue B: Cursor", "Visual Cue C: Comments", "Visual Cue D: Highlighted Text", and "Audio Feature A: Spatial Audio".
CollabAlly: Accessible Collaboration Awareness in Document Editing
Phipson Lee, Zhuohao Zhang, Jaylin Herskovitz, JooYoung Seo, Anhong Guo
Honorable Mention CHI 2022 Paper & ASSETS 2021 Demo
paper pdf ·  demo pdf ·  ACM DL ·  full video ·  30s preview ·  talk ·  code
A man using a virtual reality headset, holding a controller up. There are lines drawn on top of the image to show parts of his body being tracked for XRStudio configuration
XRStudio: A Virtual Production Technology Probe for Immersive Instructional Experiences
Michael Nebeling, Shwetha Rajaram, Liwei Wu, Yi Fei Cheng, Jaylin Herskovitz
CHI 2021
pdf · ACM DL · video
Two example prototypes for making AR apps accessible. A: Foundational Accessibility. Screenshot of a virtual chair with a voice over target around it, a speech bubble shows the app announcing "Back of chair with blue cushion". B: Scanning. Screenshot of AR grid overlaid on a coffee table. Speech bubbles show the app announcing "Found a new horizontal surface" and "Scanned 2 surfaces totaling 2.3 square meters".
Making Mobile Augmented Reality Applications Accessible
Jaylin Herskovitz, Jason Wu, Samuel White, Amy Pavel, Gabriel Reyes, Anhong Guo, Jeffrey P. Bigham
ASSETS 2020
pdf · ACM DL · video · talk
A diagram of Bashon. From left to right: A screenshot of a file system is broken into three sections by red dashed lines. For each section, to create the crowdworker request, that section is combined with the desired commands. Then, crowdworkers validate each section by voting, resulting in a single final set of validated commands.
A Hybrid Crowd-Machine Workflow for Program Synthesis
Yan Chen, Jaylin Herskovitz, Walter S. Lasecki, Steve Oney
VL/HCC 2020
A screenshot of a code editor in EdCode. There is a hand pointing icon towards a button that says "Link", and a yellow box overlaid on the screenshot says "Select the associated code context below!"
EdCode: Towards Personalized Support at Scale for Remote Assistance in CS Education
Yan Chen, Jaylin Herskovitz, Gabriel Matute, April Wang, Sang Won Lee, Walter S. Lasecki, Steve Oney
Best Short Paper Award VL/HCC 2020
A series of images of someone using an AR headset to show instructions for hanging a shelf. In the images, they are using a controller to measure the wall, placing two supports, and placing a shelf.
Opportunities for In-Home Augmented Reality Guidance.
Jaylin Herskovitz, Eyal Ofek, Walter S. Lasecki, Adam Fourney
CHI 2019 Poster
pdf · ACM DL
A screenshot of a 3D web application used to create an AR prototype of an interactive kitchen application. In the screenshot, there are three models of AR headsets, which represent people walking around and reviewing the application. The location of the models updates in real time as people move around.
Opportunities for In-Home Augmented Reality Guidance.
Jaylin Herskovitz, Janani Chinnam, Isabelle Wong, Mengyao Liu, Junlin Mo, Sang Won Lee, Walter S. Lasecki
CHI 2018 Workshop on Novel Interaction Techniques for Collaboration in VR
PhD Candidate
Computer Science & Engineering
College of Engineering
University of Michigan
Curriculum Vitae
Google Scholar
Email: jayhersk@umich.edu
Twitter: @jayhersk

Attending

Apr 23 - Apr 28 2023
CHI
Apr 30 - May 5 2022
CHI
Oct 18-22, 2021
ASSETS
Oct 10-14, 2021
UIST
May 10-13, 2021
CHI