FourEyes

FourEyes is an interactive, power-assisted tool for segmenting and annotating images. Click on some regions, give them a label, and FourEyes extrapolates the label to other regions on the image and in the database. This could be implemented by choosing a single image feature like color or texture and forming, say, a Gaussian model of the label in that feature space. However, that naively assumes that one model will be suitable for all kinds of labels in all kinds of imagery. Instead, FourEyes has a "society of models" among which it selects and combines models to produce a labeling. This selection process is guided solely by the examples given by the user.

The figure above is an example of FourEyes extrapolating the labels "building," "car," and "street" from a few examples of each (highlighted in red). Click for a full screen shot. Also check out a country-scene example and the new Photobook-6 interface.

Examples:

Publications:

An Image Database Browser that Learns from User Interaction
T. P. Minka, Master of Engineering Thesis, 1996. TR #365
Interactive Learning using a "Society of Models"
T. P. Minka and R. W. Picard, Pattern Recognition, 30(4), 1997. TR #349
Vision Texture for Annotation
R. W. Picard and T. P. Minka, ACM/Springer Journal of Multimedia Systems, 1995. TR #302
FourEyes is a tool for annotation, embedded in Photobook. Martin Szummer wrote a tool for browsing based on the same technology.
Thomas P. Minka, tpminka@media.mit.edu
Last modified: Wed Mar 3 20:03:48 EST 1999