GazeSummary: Exploring Gaze as an Implicit Prompt for Personalization in text-based LLM Tasks.
Published in hotMobile, 2026
Recommended citation: Jiexin Ding, Yizhuo Zhang, Xinyun Liu, Ke Chen, Yuntao Wang, Shwetak Patel, and Akshay Gadre. 2026. GazeSummary: Exploring Gaze as an Implicit Prompt for Personalization in Text-based LLM Tasks. In The 27th International Workshop on Mobile Computing Systems and Applications (HotMobile 26), February 25–26, 2026, Atlanta, GA, USA. ACM, New York, NY, USA, 9 pages. https://arxiv.org/abs/2601.17676
Smart glasses are accelerating progress toward more seamless and personalized LLM-based assistance by integrating multimodal inputs. Yet, these inputs rely on obtrusive explicit prompts. The advent of gaze tracking on smart devices offers a unique opportunity to extract implicit user intent for personalization. This paper investigates whether LLMs can interpret user gaze for text-based tasks. We evaluate different gaze representations for personalization and validate their effectiveness in realistic reading tasks. Results show that LLMs can leverage gaze to generate high-quality personalized summaries and support users in downstream tasks, highlighting the feasibility and value of gaze-driven personalization for future mobile and wearable LLM applications.