Emerging Images

ACM Transactions on Graphics (Proc. of SIGGRAPH ASIA 2009)

Abstract

Emergence refers to the unique human ability to aggregate informationfrom seemingly meaningless pieces, and to perceive a wholethat is meaningful. This special skill of humans can constitute aneffective scheme to tell humans and machines apart. This paperpresents a synthesis technique to generate images of 3D objects thatare detectable by humans, but difficult for an automatic algorithmto recognize. The technique allows generating an infinite numberof images with emerging figures. Our algorithm is designed so thatlocally the synthesized images divulge little useful information orcues to assist any segmentation or recognition procedure. Therefore,as we demonstrate, computer vision algorithms are incapableof effectively processing such images. However, when a human observeris presented with an emergence image, synthesized using anobject she is familiar with, the figure emerges when observed as awhole. We can control the difficulty level of perceiving the emergenceeffect through a limited set of parameters. A procedure thatsynthesizes emergence images can be an effective tool for exploringand understanding the factors affecting computer vision techniques. 


Results

(Top) This image, when stared at for a while, can reveal four instances of a familiar figure. Two of the figures are easier to detect than the others. Locally there is little meaningful information, and we perceive the figures only when observing the whole figures.
(Left) A classic example of an emergence image. Although at first sight the left image looks meaningless, suddenly we perceive the central object as the Dalmatian dog pops out.
(Right) Emergence images, when observed through small windows, look meaningless. Although we perceive the subject in the whole image, the smaller sized segments, in isolation, look like random patches. In contrast, the elephant can be recognized through similar windows of the normal shaded scene.
(Left) We often fail to perceive an emergence image when the subject is in an uncommon pose. Among the users who were shown the above images, the average success rate was only 54% and 4%, respectively. When the inverted versions of these images were shown, the success rates went up to 96% and 91%, respectively.

Mouse over the image to flip it.

(Below) Typical emergence images generated by our synthesis algorithm. We generate a range of examples on various subjects synthesized at different difficulty levels. Each example contains exactly one subject. (Please refer to supplementary material for other examples.)
(Below) In many computer vision recognition or segmentation algorithms, the first stages comprise of multi-scale edge detection or other means of bottom-up region processing. At multiple-scales, we detect edges using standard Canny edge detector, and retain the ones that persists scales. Such curves are then linked together based on spatial proximity and curvature continuity. We observe that while on the original renderings the method successfully extracts the feature curves (right image in each box), on the emerging images the results can mostly be seen as noise. This indicates the difficulty that bottom-up algorithms face when detecting objects in the emergence images.
(Left) Emerging frog at various difficulty levels, increasing from left to right. We control the difficultly by controlling the sampling density, breaking the silhouette continuity, perturbing silhouette patches, and adding clutter using cut-perturb-paste.
(Right) Difficulty level as perceived by users and as predicted by our synthesis parameters. (Right) Perceived difficulty level in each category changes gradually. For example, 98% of the easy images were recognized by at least 80% of the observers.

In News

Video

Acknowledgement

This work is supported in part by the Landmark Program of theNCKU Top University Project (contract B0008), the National ScienceCouncil (contracts NSC-97-2628-E-006-125-MY3 and NSC-96-2628-E-006-200-MY3) Taiwan, the Israeli Ministry of Science,and the Israel Science Foundation. Niloy was supported by a MicrosoftOutstanding Young Faculty Fellowship. We are gratefulto the members of Computer graphics Group/Visual System Lab,National Cheng-Kung University, in particular Shu-Hau Nien, forhelping to conduct the user evaluation, and the various users whoparticipated in the user study. We are grateful to the anonymousreviewers for their comments and suggestions. 

Bibtex

@article{Mitra:2009:EI,
 author = {Mitra, Niloy J. and Chu, Hung-Kuo and Lee, Tong-Yee and Wolf, Lior and Yeshurun, Hezy and Cohen-Or, Daniel},
 title = {Emerging Images},
 journal = {ACM Trans. Graph. (Proc. SIGGRAPH Asia)},
 volume = {28},
 number = {5},
 year = {2009},
 pages = {163:1--163:8},
 articleno = {163},
 numpages = {8}
}

Links