Semi-Automated Segmentation of Human Head Portraits
We developed a system for the semi-automated segmentation of frontal human head portraits from arbitrary unknown backgrounds. An initial cutout is computed without any user interaction by exploiting information from a face feature detector, image-deduced color models, and a learned parametric head shape model as prior information. If necessary, precise corrections can be applied in an optional refinement step with minimal user interaction.
Segmentation results. The images show the boundary found in the shape-constrained rough segmentation stage as green contour, and the final cutout after local refinement. Left and center images: results computed without any user interaction. Right: results computed with minimum interactive user refinement. The green dots indicate the user's mouse clicks.
Unsupervised Shape-Constrained Segmentation
Using a standard Adaboost eye detector, the image is transformed into a 2D scale and rotation normalized polar reference frame for all subsequent processing. Besides geometric normalization, this simplifies the head contour search to essentially a 1D problem, as the boundary may be approximated by a one-dimensional vector of polar radii.
A parametric PCA head shape model, learned offline from a large database of manually segmented images, is used to guide the rough boundary fitting stage. The shape model provides statistically safe fore- and background regions in the image, which are used to train Gaussian Mixture color models and compute foreground posterior probabilities in the image. An iterative optimization procedure maximizes the distance of foreground probabilities along both sides of the boundary, while at the same time constraining the contour to a plausible head shape using the PCA shape model.
The rough segmentation stage iteratively maximizes the distance of foreground probabilities along both sides of the contour while constraining the boundary to a plausible head shape.
The rough cutout from the previous stage is locally refined with a boundary based algorithm called "Cluster Cutting". Cluster Cutting uses a cost function derived from clustering pixels along the normal of the initial segmentation path with a tree-building algorithm to fit the initial contour to the local boundaries in the image. An interactive version of the same algorithm is used in an interactive refinement stage, where the user can apply precise corrections with a minimum number of mouse clicks.
The refinement stage uses a hierarchical clustering algorithm to adapt the initial shape-constrained contour (left image) to the local boundaries in the image. An interactive version of same algorithm can be used to apply precise corrections with only few mouse clicks (indicated by the green dot in the right image).
Publications
D. C. Schneider, B. Prestele, P. EisertPrecise Head Segmentation on Arbitrary Backgrounds, IEEE International Conference on Image Processing (ICIP), pp. 2381 - 2384, Cairo, November 2009. [PDF]