We present PanoAnnotator, a semi-automatic system that facilitates the annotation of 2D indoor panoramas to obtain high-quality 3D room layouts. Observing that fully-automatic methods are often restricted to a subset of indoor panoramas and generate room layouts with mediocre quality, we instead propose a hybrid method to recover high-quality room layouts by leveraging both automatic estimations and user edits. Specifically, our system first employs state-of-the-art methods to automatically extract 2D/3D features from input panorama, based on which an initial Manhattan world layout is estimated. Then, the user can further edit the layout structure via a set of intuitive operations, while the system will automatically refine the geometry according to the extracted features. The experimental results show that our automatic initialization outperforms a selected fully-automatic state-of-the-art method in producing room layouts with higher accuracy. In addition, our complete system reduces annotation time when comparing with a fully-manual tool for achieving the same high quality results.