We developped probabilistic models to encode the appearance of objects, and inference methods to perform detection and pose estimation of those object in novel images of cluttered scene. We started with 3D, CAD-style models [Teney2011], then focused on appearance-based models. Those are trained using 2D example images alone, and are applicable either to specific object instances or to object categories, or classes. The appearance is modeled as a distribution of low-level, fine-grained image features. The strength of the approach is its straightforward formulation, applicable to virtually any type of image feature. Appearance is represented by probability distributions of image features, for example points along image edges, or intensity gradients extracted densely over the image.

Such models of appearance have been applied to the tasks of object detection and localization, object recognition, and pose classification, by matching the test view with one of several trained viewpoints of the object.

A notable advantage of the proposed model is its ability to use dense gradients directly (extracted over the whole images), versus relying on hand-crafted image descriptors. Using gradients extracted at coarse scales allows using shading, as well as homogeneous surfaces to recognize objects, when edges alone are ambiguous.

We also proposed extensions of this generative model to perform continuous pose estimation, by explicitely interpolating appearance between trained viewpoints. This makes it one of the rare methods capable of doing appearance-based continuous pose estimation at category level, this capability being usually reserved to methods based on 3D CAD models of objects, and limited to specific object instances.

We found an interesting application of those models and methods in the recognition (and pose estimation) of a robotic arm in 2D images. This task is very challenging due to the smooth and untextured appearance of the robot arm considered (a Kuka LWR).