Keywords

Topic modeling; User Study; Mixed-Initiative Interaction;

Abstract

Topic modeling is a common tool for understanding large bodies of text, but is typically provided as a “take it or leave it” proposition. Incorporating human knowledge in unsupervised learning is a promising approach to create high-quality topic models. Existing interactive systems and modeling algorithms support a wide range of refinement operations to express feedback. However, these systems’ interactions are primarily driven by algorithmic convenience, ignoring users who may lack expertise in topic modeling. To better understand how non-expert users understand, assess, and refine topics, we conducted two user studies—an in-person interview study and an online crowdsourced study. These studies demonstrate a disconnect between what non-expert users want and the complex, low-level operations that current interactive systems support. In particular, our findings include: (1) analysis of how non-expert users perceive topic models; (2) characterization of primary refinement operations expected by non-expert users and ordered by relative preference; (3) further evidence of the benefits of supporting users in directly refining a topic model; (4) design implications for future human-in-the-loop topic modeling interfaces.

Original Publication Citation

Lee, Tak Yeon, et al. "The Human Touch: How Non-expert Users Perceive, Interpret, and Fix Topic Models." International Journal of Human-Computer Studies (2017).

Document Type

Peer-Reviewed Article

Publication Date

2017-9

Permanent URL

http://hdl.lib.byu.edu/1877/3801

Publisher

Elsevier

Language

English

College

Physical and Mathematical Sciences

Department

Computer Science

University Standing at Time of Publication

Full Professor

Download

Share

COinS