Penquiry: A Pen-based Interactive In-situ Q&A System Leveraging LLMs

Synthesis: Rhee et al. (2026) start from a mismatch: pen-based digital study is fluid and spatial, while LLM prompting is discrete and keyboard-bound. Penquiry closes the gap with an in-situ question-answering system that lets Learners ask questions directly on digital study materials with a pen. The authors name the two obstacles — a Referential Barrier (grounding fine-grained visual elements into the query context) and an Expressive Barrier (translating equations and diagrams into typed sentences) — and answer them with Content Snapping and Question Autocompletion. Two iterative studies of 16 participants each found the system significantly reduced the cognitive and physical overhead of inquiry.

Key Findings

1. Two barriers explain why pen users avoid AI help. The Referential Barrier blocks unambiguous pointing at a diagram region or equation term; the Expressive Barrier forces translation of non-textual intent into rigid prose.
2. Content Snapping resolves reference by attaching ink marks to specific document elements, so a question carries an unambiguous target.
3. Question Autocompletion expands sparse ink keywords into rich semantic queries, lowering the effort of formulating a good prompt.
4. Overhead fell measurably in two iterative user studies (N = 16 each) compared with conventional interfaces.
5. The design preserves the study material as the locus of interaction, rather than moving the learner into a chat window — the interaction stays in situ.

Why interaction design is the binding constraint for AI study help

Access to a capable model is no longer the scarce resource in Self-Directed Learning; the scarce resource is the learner's willingness to interrupt their work to formulate a question. The friction is behavioral as much as technical: every keystroke and every rephrasing is a cost paid at exactly the moment Help-Seeking is most fragile. Penquiry's contribution is to treat that cost as the design problem, which is why its results speak to Student-AI Interaction and to the broader Student Experience literature rather than only to input-modality engineering.

Tensions the paper leaves open

Lowering the cost of asking makes asking easier — including for questions a learner could answer themselves, the classic Cognitive Offloading risk. The authors' proposed direction of temporally adaptive autocompletion, moving from foundational fact-checking early in a session toward higher-level prompts later, is explicitly an attempt to turn scaffolding into fading support rather than a permanent crutch. That remains a design hypothesis rather than a tested outcome, and it is the point where this work intersects the wiki's ongoing questions about when AI help helps.

What this means for practice

• Designers. Keep the question inside the study material instead of routing learners to a chat window: attach ink marks to specific document elements so a query carries an unambiguous target, since 85% of queries contained at least one visible mark and underlining (45%), circles (29%) and boxes (9%) were the dominant referencing strategies observed.
• Designers. Expand sparse ink keywords into full semantic queries to cut the physical and cognitive cost of handwriting prompts — the Penquiry condition scored 80.78 on the System Usability Scale ("Excellent") and lowered NASA-TLX physical demand while raising reported learning flow (4.06 to 5.63, p = .046).
• Edtech designers. Instrument the interaction so scaffolding can fade: the authors' proposed direction is temporally adaptive autocompletion that moves from foundational fact-checking early in a session toward higher-level prompts later, which is still a design hypothesis rather than a tested outcome in this work.
• Learners. Use pen-based referencing where spatial precision is the difficulty — part-scoped questions about a specific axis, table row or term were the case in which direct pen marking beat screenshot-and-paste workarounds most clearly, and the gesture approach is worth learning deliberately.

Limitations

• Both evaluations are small lab studies of 16 participants each (Study 1: 9 male, 7 female, mean age 24.8; Study 2: 7 male, 9 female, mean age 22.69), recruited as university students who had used LLMs during pen-based learning within the previous six months — an experienced, self-selected convenience sample rather than a general learner population, and the sessions were short enough that the authors state participants may not have fully adapted to the pen-based questioning environment.
• The outcomes are self-report and preference measures — SUS, the Technology Acceptance Model, NASA-TLX and 7-point Likert items — collected on a fixed set of study materials, so the studies show reduced interaction overhead and higher preference, not measured learning gains.
• Usefulness was conditioned on prior knowledge: qualitative feedback showed that Question Autocompletion's utility scaled with the learner's domain expertise, because the feature helps most when the underlying concepts are already understood.
• The pen-based question answering ran through a Wizard-of-Oz procedure in which a human operator captured the sketched region and forwarded it to GPT-4o, so the measured experience reflects a facilitated pipeline rather than a fully deployed system.

Connected Concepts

• Large Language Models (LLMs)
• Help-Seeking
• Self-Directed Learning
• Student-AI Interaction
• Multimodal AI
• Conversational AI
• Cognitive Offloading
• Scaffolding
• Usability Research
• Student Experience
• Active Learning

Connected Articles

• AI Advice Suppresses People's Willingness to Say "I Don't Know", Even When the Advice Is Wrong and Accuracy Is Incentivized — AI advice and the suspension of prior knowledge
• Are Students Dependent on AI in Writing Courses? Analyzing Factors Influencing Dependence on Generative AI Through the I-PACE Model — AI dependence in academic writing
• Towards an adaptive AI scaffold for developing student collaborative problem solving — Adaptive AI scaffolds for collaborative problem solving
• PromptDecipher: Supporting AI Tutor Authoring Through Editable Simulated Interactions — Authoring AI tutors from prompts

Citation

Rhee, J., Lee, C., Kim, H., Choe, K., Kim, B., Ko, S., & Seo, J. (2026). Penquiry: A Pen-based Interactive In-situ Q&A System Leveraging LLMs. arXiv:2609.19870.