Few Shot Visual Counting

Background

Visual counting, the automatic enumeration of objects or visual patterns within images and videos, is a critical task with widespread applications in areas like crowd monitoring, inventory tracking, and medical diagnostics. However, existing methods primarily concentrate on specific, predefined object categories such as people, cars, or cells. These techniques, often based on deep neural networks, necessitate large-scale training datasets with thousands or millions of annotated instances. A significant challenge is that these trained networks are inherently limited to counting only the object categories they were specifically trained on, making them inflexible for novel or arbitrary object types without extensive retraining and data acquisition.

Technology

Researchers at Stony Brook University developed a technology utilizing a class-agnostic Few-Shot Visual Counting method leveraging an Exemplar Matching Network (EmNet). EmNet accepts a query image and a few user-provided exemplar objects, which can be annotated using dots, bounding boxes, or polygons. A feature convolution block then processes these features by convolving exemplar feature maps with the image feature map to produce a 3D tensor, which is subsequently reduced via average and max pooling and concatenation into 2D feature maps. Finally, a density prediction block, composed of convolution and upsampling layers, generates a 2D density map indicating the locations of all objects of interest, with the total count derived by summing the pixels in this map.