EZ-Manipulator: Designing a Mobile, Fast and Ambiguity-Free 3D Manipulation Interface using Smartphones

Po-Huan Tseng
National Tsing Hua University
Shih-Hsuan Hung
National Taiwan University of Science and Technology
Pei-Ying Chiang
National Taipei University of Technology
Chih-Yuan Yao
National Taiwan University of Science and Technology
Hung-Kuo Chu
National Tsing Hua University
Computational Visual Media (Proc. of Computational Visual Media 2018)


Interacting with digital contents in 3D is an essential component to various applications (e.g., modeling packages, gaming, virtual reality, etc.). A traditional interface such as keyboard-mouse or trackball usually demands non-trivial working space as well as a learning process. We present the design of EZ-Manipulator, a new 3D manipulation interface on smartphones that supports mobile, fast, and ambiguity-free interactions with 3D objects. Our system leverages the built-in multitouch input and gyroscope sensor of a smartphone to achieve 9DOF (nine Degrees of Freedom) axis-constrained manipulations and free-form rotation. Thus, using EZ-Manipulator to manipulate objects in 3D is easy. The user merely has to perform intuitive single- or two-finger gestures and rotating the device in hand(s) to achieve manipulations at respectively fine-grained and course level. We further investigate the ambiguous manipulations introduced by the indirect manipulations using multitouch interface and propose a dynamic virtual camera adjustment to effectively resolve the ambiguity. A preliminary study reports that our system has significant lower task completion times in comparison to the conventional keyboard-mouse interface, and receives positive user experience from both novices and experts.


Multitouch gestures for 9DOF axis-constrained manipulations, including (a) translation, (b) non-uniform scaling, (c) uniform scaling, and (d) rotation.

Free-form rotation using reference rotation inferred from gyroscope sensor.


Applications using EZ-Manipulator. (Left) Rapid prototyping for interior designs. (Middle) Multiple users collaboration and interaction with a wall-sized display. (Right) Multiple users gaming.