Feature Extraction

Visual information contained in images is usually represented by low-level fVisual information contained in images is usually represented by low-level feature descriptors focusing on different types of information, such as color, texture, and shape. An adequate feature descriptor is able to discriminate between regions with different characteristics and it allows similar regions to be grouped together even when captured under noisy conditions. However, it is usually difficult to have a single feature descriptor adequate for many application domains; this has motivated researchers to develop a variety of feature extraction methods.

This research we focuses on the development and  analysis of feature extraction methods so that a better representation may be extracted from the visual information contained in images and videos. We also exploit ways of combining multiple feature channels to provide compact and robust representations.

Feature extraction for a two level shearlet decomposition considering eight orientations. A histogram containing the same number of bins as the number of orientations considered by the shearlet transform is estimated for each level of decomposition (the spatial frequencies histogrammed at each level are marked in blue). These histograms describe the distribution of edge orientations at the given scales. The entry for each histogram bin is obtained by adding up the absolute value of the shearlet coefficients at the corresponding orientation. The histograms of shearlet coefficients (HSC) is obtained by the concatenation of the histograms estimated for each decomposition level, followed by L2-norm normalization.

Software

• Histogram of Shearlet Coefficients (HSC) available for download here.
• Circular Center Symmetric-Pairs of Pixels (CCS-POP) available for download here.

References

• [Online First] F. R. de Siqueira and W. R. Schwartz and H. Pedrini. Multi-Scale Gray Level Co-Occurrence Matrices for Texture Description. Neurocomputing, pp. 1-10, 2013. [pdf]
• E. R. Nascimento and G. L. Oliveira and M. F. M. Campos and A. W. Vieira, W. R. Schwartz. BRAND: A Robust Appearance and Depth Descriptor for RGB-D Images. IEEE/RSJ International Conference on Intelligent Robots and Systems, 2012. (oral presentation) [pdf]
• E. R. Nascimento and W. R. Schwartz and M. F. M. Campos. EDVD - Enhanced Descriptor for Visual and Depth Data. IAPR International Conference on Pattern Recognition, 2012. (oral presentation) [pdf]
• E. R. Nascimento and W. R. Schwartz and G. L. Oliveira and A. W. Vieira and M. F. M. Campos and D. B. Mesquita. Appearance and Geometry Fusion for Enhanced Dense 3D Alignment. Conference on Graphics, Patterns and Images, 2012. (oral presentation) [pdf]
• W. R. Schwartz and H. Pedrini. Evaluation of Feature Descriptors for Texture Classification. Journal of Electronic Imaging, vol. 21, n. 2, pp. 1-17, 2012. [pdf]
• R. D. da Silva and W. R. Schwartz and H. Pedrini. Scalar Image Interest Point Detection and Description Based on Discrete Morse Theory and Geometric Descriptors. IEEE International Conference on Image Processing, 2012. [pdf]
• J. Choi and H. Guo and W. R. Schwartz and L. S. Davis. A Complementary Local Feature Descriptor for Face Identification. IEEE Workshop on Applications of Computer Vision, pp. 121-128, 2012. (full oral presentation) [pdf] [code]
• W. R. Schwartz and H. Pedrini. Evaluation of Feature Descriptors for Texture Classification. Journal of Electronic Imaging, vol. 21, n. 2, pp. 1-17, 2012. [pdf]
• W. R. Schwartz and R. D. da Silva and L. S. Davis and H. Pedrini. A Novel Feature Descriptor Based on the Shearlet Transform. IEEE International Conference on Image Processing, pp. 1033-1036, 2011. [pdf] [poster] [code]
• [pt-BR] R. D. da Silva and W. R. Schwartz and H. Pedrini. Avaliação da Invariância à Rotação de Descritores Texturais Extraídos por Transformadas Wavelets. Simpósio Brasileiro de Sensoriamento Remoto, pp. 7159-7166, 2009.
• [pt-BR] W. R. Schwartz and H. Pedrini. Avaliação de Métodos de Análise de Texturas Aplicadas em Imagens Digitais de Terrenos. Colóquio Brasileiro de Ciências Geodésicas, 2005.

Face Recognition

The three primary face recognition tasks are verification, identification, and watch list. In verification, the task is to accept or deny the identity claimed by a person. In identification, an image of an unknown person is matched to a gallery of known people. In the watch list task, a face recognition system must first detect if an individual is on the watch list. If the individual is on the watch list, the system must then correctly identify the individual. We address the identification task. Due to the availability of large amounts of data acquired in a variety of conditions, techniques that are robust to uncontrolled acquisition conditions, handle small sample sizes, and scalable to large gallery sizes are desirable. 

In this research, we tackle the face identification and verification tasks by using large sets of feature descriptors and Partial Least Squares (PLS) to design robust techniques. In the former task, we employ a one-against-all scheme to learn a PLS discriminatory models and then  for construct a tree-based structure to reduce scalability issues. For the latter task, we combine PLS and the one-shot similarity model to compute similarity score between pairs of facial images.

Tree-based structure used to optimize the search for matches to a probe sample. Each internal node contains a PLS regression model used to guide the search, as shown in details for node n3, which has a PLS model constructed so that the response directs the search either to node n6 or n7. In this example the first path to be traversed is indicated by arrows (in this case, it leads to the correct match for this particular probe sample). Alternative search paths are obtained by adding nodes that have not been visited into a priority queue (in this example nodes n3 and n5 will be the starting nodes for additional search paths). After pursuing a number of search paths leading to different leaf nodes, the best match is chosen to be the one presenting the highest response (in absolute value).

Software

References 

• G. P. Carlos and H. Pedrini and W. R. Schwartz. Fast and Scalable Enrollment for Face Identification based on Partial Least Squares. IEEE International Conference on Automatic Face and Gesture Recognition, 2013. [pdf]
• C. Santos Junior and W. R. Schwartz. Detecting Unenrolled Subjects in Face Galleries. Workshop of Undergraduate Works (WUW) in SIBGRAPI - Conference on Graphics, Patterns and Images, pp. 1-6, 2013. (2nd place award) [pdf]
• G. Chiachia and N. Pinto and W. R. Schwartz and A. Rocha and A. X. Falcao and D. Cox. Person-Specific Subspace Analysis for Unconstrained Familiar Face Identification. British Machine Vision Conference, 2012. [pdf]
• W. R. Schwartz and H. Guo and J. Choi and L. S. Davis. Face Identification Using Large Feature Sets. IEEE Transactions on Image Processing, vol. 21, n. 4, pp. 2245-2255, 2012. [pdf]
• [pt-BR] G. P. Carlos and H. Pedrini and W. R. Schwartz. Uma Abordagem Escalável para Manutenção de Galeria de Faces. Workshop of Undergraduate Works (WUW) in SIBGRAPI 2012 (XXV Conference on Graphics, Patterns and Images), pp. 137-142, 2012. [pdf] [poster]
• H. Guo and W. R. Schwartz and L. S. Davis. Face Verification using Large Feature Sets and One Shot Similarity. International Joint Conference on Biometrics, 2011. [pdf]
• W. R. Schwartz and H. Guo and L. S. Davis. A Robust and Scalable Approach to Face Identification. European Conference on Computer Vision, Lecture Notes in Computer Science, vol. 6316, pp. 476-489, 2010. [pdf] [poster]

Spoofing Detection

Nowadays we are experiencing an increasing demand for highly secure identification and personal verification technologies. This demand becomes even more apparent as we become aware of new security breaches and transaction frauds. In this context, biometrics has played a key role in the last decade providing tools and solutions either to verify or recognize the identity of a person based on physiological or behavioral characteristics. Among the used features are face, fingerprints, hand geometry, handwriting, iris, retinal vein, and voice. Such methods, however, sometimes can be fooled (spoofed) by an identity thief, specially the ones based on face recognition, in which the thief can obtain a photo of an authentic user from a significant distance, or even obtain it from the Internet.

In this research, we study two types of face spoofing: with a photograph of a valid user and with a video of a valid user. For the first type of attack, we present an anti-spoofing solution based on a holistic representation of the face region, through a robust set of low-level feature descriptors, able to capture the differences between live and spoof images. For the second attack, we perform an analysis of the noise generated by the recaptured video to distinguish between both classes.

Devised solution to face spoofing detection. Given a set of examples and counter-examples (videos or images) of face spoofing attack, feature vectors (composed by a combination of feature descriptors extracted from the facial region) representing both classes are used to obtain a weighting based on partial least squares, so that novel samples can be classified during the test stage.

References

• I. Chingovska and J. Yang and Z. Lei and D. Yi and O Kahm and C. Glaser and N. Damer and A. Kuijper and A. Nouak and J. Komulainen and T. Pereira amd S. Gupta and S. Khandelwal and S. Bansal and A. Rai and T. Krishna and D. Goyal and M. A. Waris and H. Zhang and I. Ahmad and S. Kiranyaz and M. Gabbouj an R. Tronci and M. Pili and N. Sirena and F. Roli and J. Galbally and J. Fierrez and A. Pinto and H. Pedrini and W. R. Schwartz and A. Rocha and A. Anjos and S. R. Marcel. The 2nd Competition on Counter Measures to 2D Face Spoofing Attacks. International Conference on Biometrics, pp. 1-6, 2013. [pdf]
• A. S. Pinto and H. Pedrini and W. R. Schwartz and A. Rocha. Video-Based Face Spoofing Detection through Visual Rhythm Analysis. Conference on Graphics, Patterns and Images, 2012. (oral presentation) [pdf]
• W. R. Schwartz and A. Rocha and H. Pedrini. Face Spoofing Detection through Partial Least Squares and Low-Level Descriptors. International Joint Conference on Biometrics, 2011. [pdf] [poster]
• M. M. Chakka and A. Anjos and S. Marcel and R. Tronci and D. Muntoni and G. Fadda and M. Pili and N. Sirena and G. Murgia and M. Ristori and F. Roli and J. Yan and D. Yi and Z. Lei and Z. Zhang and S. Li and W. R. Schwartz and A. Rocha and H. Pedrini and J. Lorenzo-Navarro and M. Castrillon-Santana and J. Maatta. Competition on Counter Measures to 2D Facial Spoofing Attacks. International Joint Conference on Biometrics, 2011. [pdf]

Pedestrian Detection

In this research, we develop methods based on Partial Least Squares (PLS) to perform robust human detection considering high dimensional feature spaces and we also fuse face and body information using first-order-logic and Markov Logic Networks (MLN) to detect pedestrians partially occluded. Using the response map computed with the detector, we also develop a new feature descriptor, the local response context (LRC), is designed to be applied to pedestrian detection. 

The left-hand side shows the detection process performed by a generic pedestrian detector. The right-hand side shows the incremented detection process with the addition of the local response context descriptor. Using the resulting response map, LRC descriptors are extracted for each detection window and projected onto a PLS model, resulting in a more accurate classification between humans and non-humans.

Software

References

• W. R. Schwartz and L. S. Davis and H. Pedrini. Local Response Context Applied to Pedestrian Detection. Iberoamerican Congress on Pattern Recognition, pp. 181-188, 2011. [more info] [pdf]
• W. R. Schwartz. Human Detection Based on Large Feature Sets Using Graphics Processing Units. Informatica, vol. 35, n. 4, pp. 473-479, 2011. [pdf]
• R. Gopalan and W. R. Schwartz. Detecting Humans under Partial Occlusions using Markov Logic Networks. Performance Metrics for Intelligent Systems, 2010. [pdf]
• W. R. Schwartz and A. Kembhavi and D. Harwood and L. S. Davis. Human Detection Using Partial Least Squares Analysis. IEEE International Conference on Computer Vision, pp. 24-31, 2009. (oral presentation) [pdf] [code]
• W. R. Schwartz and R. Gopalan and R. Chellappa and L. S. Davis. Robust Human Detection under Occlusion by Integrating Face and Person Detectors. International Conference on Biometrics, Lecture Notes in Computer Science, vol. 5558, pp. 970-979, 2009. [pdf]

Person Re-Identification

Appearance information is essential for applications such as tracking and people recognition. One of the main problems of using appearance-based discriminative models is the ambiguities among classes when the number of persons being considered increases. To reduce the amount of ambiguity, we propose the use of a rich set of feature descriptors based on color, textures and edges. Another issue regarding appearance modeling is the limited number of training samples available for each appearance. 

The discriminative models are created using a powerful statistical tool called Partial Least Squares (PLS), responsible for weighting the features according to their discriminative power for each different appearance. For each appearance represented by an image window, features are extracted and PLS is applied to reduce dimensionality using a one-againstall scheme. Afterwards, a simple classifier is used to match new samples to models learned.

Spatial distribution of the weights of the first projection vector when PLS is used to combine the three feature channels. High weights are located in regions that better distinguish a specific appearance from the remaining ones. For example, black regions of the homogeneous jackets are not given high weights, since several people wear black jackets. However, the regions where the white and red jackets are located obtain high weights due to their unique appearances.

Dataset

The dataset used to obtain the results shown in the paper can be downloaded from this link. We cropped samples from the ETHZ dataset because the video sequences present challenges for recognition such as partial occlusions, changes in illumination, and large variability of poses.

References

• C. R. S. Dutra and T. Souza and R. Alves and W. R. Schwartz and L. R. Oliveira. Re-identifying People based on Indexing Structure and Manifold Appearance Modeling. SIBGRAPI - Conference on Graphics, Patterns and Images, pp. 1-8, 2013. [pdf]
• W. R. Schwartz. Scalable People Re-Identification Based on a One-Against-Some Classification Scheme. IEEE International Conference on Image Processing, 2012. (oral presentation) [pdf]
• W. R. Schwartz and L. S. Davis. Learning Discriminative Appearance-Based Models Using Partial Least Squares. Brazilian Symposium on Computer Graphics and Image Processing, 2009. [pdf]

Fractal compression is one of the most promising techniques image and video compression due to advantages such as resolution independence and fast decompression. It exploits the fact that natural scenes present self-similarity to remove redundancy and obtain high compression rates with smaller quality degradation compared to traditional compression methods. The main drawback of fractal compression is its computationally intensive encoding process, due to the need for searching regions with high similarity in the image. Several approaches have been developed to reduce the computational cost to locate similar regions. 

In this research, we exploit methods based on robust feature descriptors to speed up the encoding time and develop 3D fractal video encoders based on searchless or limited search approaches to being able to provide real time video encoding achieving better results than x264 video encoder for high motion video sequences. 

Fractal image compression approach. For a given level of the quadtree decomposition where range blocks have BxB pixels are considered, domain blocks of 2Bx2B pixels are sampled from the input image. Using feature vectors as descriptor for each block, a clustering algorithm is applied to partition the domain blocks in k clusters. After extracting the feature vector for a range block, the closest cluster, according to the distance between the cluster centroid and range block feature vector, is estimated, then only domain blocks belonging to the referred cluster are considered to find the best match. Once the best matching domain block is obtained, the reconstruction error is estimated. If it is smaller than the threshold a fractal code is generated, otherwise, the range block is split into four sub-blocks of size B/2 x B/2 pixels to be considered in the next level of the quadtree decomposition.

References

• V. de Lima and W. R. Schwartz and H. Pedrini. 3D Searchless Fractal Video Encoding at Low Bit Rates. Journal of Mathematical Imaging and Vision, vol. 45, n. 3, pp. 239-250, 2013. [pdf]
• V. de Lima and W. R. Schwartz and H. Pedrini. Fast Low Bit-Rate 3D Searchless Fractal Video Encoding. Conference on Graphics, Patterns and Images, pp. 189-196, 2011. [pdf]
• W. R. Schwartz and H. Pedrini. Improved Fractal Image Compression Based on Robust Feature Descriptors. International Journal of Image and Graphics, vol. 11, n. 4, pp. 571-587, 2011. [pdf]
• V. de Lima and W. R. Schwartz and H. Pedrini. Fractal Image Encoding Using a Constant Size Domain Pool. Workshop de Visão Computacional, pp. 137-142, 2011. [pdf]
• W. R. Schwartz and H. Pedrini and L. S. Davis. Video Compression and Retrieval of Moving Object Location Applied to Surveillance. International Conference on Image Analysis and Recognition, Lecture Notes in Computer Science, vol. 5627, pp. 906-916, 2009. [pdf]
• [pt-BR] R. Barriviera and W. R. Schwartz and H. Pedrini. Compressão Fractal de Imagens Baseada em Tabelas de Dispersão. Workshop de Visão Computacional, 2009.

The primary purpose of an image segmentation systemis to extract information from the images to allow the discrimination among different objects of interest. Image segmentation is of great interest in a variety of scientific and industrial fields, with applications in medicine, microscopy, remote sensing, control of quality, retrieval of information in graphic databases, among others. The segmentation process is usually based on gray level intensity, color, shape, or texture.

This research studies the use feature descriptors based on texture and signal processing to segment images in similar regions. In addition, the spatial dependency is also considered by the use of the Markov random fields. Among the main advantages in using segmentation based on random fields are the integration of spatial relationship between adjacent regions of the image, the use of several features for image description by means of the Bayesian formulation, the region labeling for generating the final segmentation obtained directly from the random field, and the incorporation of constraints into the energy function to be minimized.

Diagram shows a segmentation approach based on two stages. First, the wavelet transform is applied to extract feature descriptors and the k-means clustering algorithm is executed. In the second stage, boundary regions are segmented in a pixel-based approach.

References

• R. D. da Silva and W. R. Schwartz and H. Pedrini. Image Segmentation Based on Wavelet Feature Descriptor and Dimensionality Reduction Applied to Remote Sensing. Chilean Journal of Statistics, vol. 2, n. 2, pp. 51-60, 2011.
• R. D. da Silva and W. R. Schwartz and H. Pedrini. Terrain Image Segmentation Based on Wavelet Transform and Partial Least Squares. Simpósio em Estatística Espacial e Modelagem de Imagens, pp. 1-4, 2010.
• R. S. da Silva and R. Minetto and W. R. Schwartz and H. Pedrini. Satellite Image Segmentation Using Wavelet Transforms Based on Color and Texture Features. International Symposium on Advances in Visual Computing, Lecture Notes in Computer Science, vol. 5359, pp. 113-122, 2008. [pdf]
• W. R. Schwartz and H. Pedrini. Color Textured Image Segmentation Based on Spatial Dependence Using 3D Co-occurrence Matrices and Markov Random Fields. International Conference in Central Europe on Computer Graphics, Visualization and Computer Vision, pp. 81-87, 2007. [pdf]
• W. R. Schwartz and H. Pedrini. Textured Image Segmentation Based on Spatial Dependence using a Markov Random Field Model. IEEE International Conference on Image Processing, pp. 2449-2452, 2006. [pdf]
• W. R. Schwartz and R. D. da Silva and R. Minetto and H. Pedrini. An Improved K-means Clustering Algorithm for Image Segmentation. Workshop de Visão Computacional, 2006.
• [pt-BR] W. R. Schwartz and H. Pedrini. Segmentação de Imagens de Terrenos Baseada na Associação de Características de Texturas com Dependência Espacial Modelada por Campo Aleatório de Markov. Simpósio Brasileiro de Sensoriamento Remoto, pp. 4311-4318, 2005.
• [pt-BR] R. Minetto and R. D. Silva and W. R. Schwartz and H. Pedrini. Segmentação de Imagens Utilizando Abordagem Espectral por Transformadas Wavelet e de Fourier. Colóquio Brasileiro de Ciências Geodésicas, 2005.

Robot soccer has been adopted as a standard problem to promote the development of new techniques in the field of Artificial Intelligence. The domain of robot soccer is highly dynamic and complex, consisting of multi-agents that are capable of performing individual and cooperative actions to solve a task. 

This research describes the computer vision module implemented in UFPR robot-soccer team. The software architecture used for the identification and tracking of the objects is presented and discussed. A flexible and efficient algorithm is proposed for real time identification and tracking of objects in the scene.

Diagram shows the general scheme of a robot soccer game. Visual information is captured from cameras placed on the top of the field. Computers evaluate the environment and plan a game, which is transmitted by radio to each robot control.

References

• [pt-BR] W. R. Schwartz and O. M. van Kaick and M. V. G. da Silva and H. Pedrini. Reconhecimento em Tempo Real de Agentes Autônomos em Futebol de Robôs. Simpósio Brasileiro de Automação Inteligente, pp. 94-99, 2003.
• [pt-BR] V. C. Brand and C. W. Paulis and M. V. G. da Silva and O.M. van Kaick and W. R. Schwartz. Um Algoritmo Eficiente para Rastreamento de Objetos em Futebol de Robôs. Evento de Iniciação Científica da UFPR, pp. 237, 2002.
• [pt-BR] O. M. van Kaick and W. R. Schwartz and M. V. G. da Silva and H. Pedrini. Identificação e Rastreamento em Tempo Real de Múltiplos Agentes Autônomos. Seminário de Informática, pp. 59-70, 2001.
• [pt-BR] C. W. Paulis and M. V. G. da Silva and O. M. van Kaick and W. R. Schwartz. Visão Computacional para Futebol de Robôs. Evento de Iniciação Científica da UFPR, pp. 182, 2001.