Managing Digital Photo Collections
by Lynn Wilcox
The FXPAL photo application automatically organizes digital photo collections based on date, event, person, or place.
The increased use of digital cameras is creating a need to manage large collections of photos. People want to organize their photos, browse their collections, search for a particular photo, and create slide shows to share with friends. However, people are typically unwilling to manually classify their photos for organization, and most often simply upload the images from the camera into folders on their computer.
The FXPAL photo application is a tool that allows users to automatically organize their photo collections on the fly. The user first tells the application where the photos are stored on the computer. He then selects the desired view and the system displays the photos accordingly. For example, if the user wants to see the photos organized by date, he can select the dates view to see photos grouped hierarchically by year, month, and day. Similarly, he can select the people view to see photos grouped according to who is in the picture and the place view to see photos grouped according to location. There is also an event view that automatically groups the photos into meaningful events, such as birthdays or weddings.
We achieve these groupings by automatically analyzing the photos’ content and the metadata associated with the photos. By metadata, we mean information such as the date, time, and location the picture was taken. Grouping by time and date is straightforward — we simply create a hierarchy that allows users to select all photos by year, month, or day. The FXPAL photo application provides a calendar interface that visualizes the photos within the calendar for easier selection. The use of space is optimized in sparse calendar views by expanding the days of the calendar where there are photos and shrinking the days where there are none.
Grouping photos by events such as holiday gatherings or vacations is a common practice among photographers. The FXPAL system automatically detects events by clustering photos according to the time they were taken and their content. The technique is based on similarity analysis, in which the self-similarity matrix of the photos, computed from temporal and content similarity, is partitioned into disjoint events. After photos have been automatically grouped into events, users can attach semantic labels, so that when the photos are displayed in the event view the contents of the collection are easily understood.
Another common method for grouping photos is according to people. Manually assigning names to people in photos is tedious and time consuming. We make use of the FSCA face detection and recognition system to make this task easier. Face detection is highly accurate and is run automatically. Faces are cropped from the photos by orienting an ellipse based on the line segment between the eye positions. Users can view only the cropped faces from the photos by selecting the face view.
To determine the identity of people in the photos, we use the FSCA face recognition system. Unlike face detection, face recognition is not reliable. Variations in lighting, pose, and eyewear cause the system to make errors. Thus we provide a user interface for semi-automatic classification of faces according to person. Using the face view, the user selects one or more faces of a particular person to create a model. The user then asks the system to find similar faces. This results in a display of faces ordered by similarity to the model. The user selects correct faces, usually found near the top of the list, and adds them to the set of faces corresponding to the person. These faces are used to update the model, thus increasing the accuracy when more faces of the person. This process is repeated until all faces for people in the collection have been labeled.
The FXPAL photo application also provides interesting ways to view photos. The Pan‘n Scan animated slide show displays full screen photo images with background music. The slide show is animated by panning and zooming through each image using rules based on location of faces. Another way to view photos is as a Stained Glass image. Selected photos are cropped using detected faces and assembled in a collage with irregular boundaries similar to stained glass windows.
The photo application created by FXPAL and FSCA is available for user testing. Please contact the author for use permission.
Link:
http://www.fxpal.com/?p=PhotoApplication
Please contact:
Lynn Wilcox, FX Palo Alto Laboratory, USA
Tel: +1 650 813 7574
E-mail: Wilcox
fxpal.com
