BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Talks.cam//talks.cam.ac.uk//
X-WR-CALNAME:Talks.cam
BEGIN:VEVENT
SUMMARY:Query-based Hard-Image Retrieval for Object Detection at Test Time
  - Ed Ayers\, Five AI
DTSTART:20221018T120000Z
DTEND:20221018T130000Z
UID:TALK183464@talks.cam.ac.uk
CONTACT:Mateja Jamnik
DESCRIPTION:Hi I'm one of Mateja's recently-graduated PhD students.\nThis 
 is a paper that I worked on during my employment at Five AI\, a self-drivi
 ng car company in Cambridge. \nI worked on verification of object-detector
 s. An object-detector here is a neural network that draws bounding boxes a
 round objects in images (e.g.\, RetinaNet\, FasterRCNN).\nThe problem we f
 aced was finding examples of images which an object-detector does badly on
  within an unannotated dataset. We needed a solution where we could define
  what 'badly' means for our specific task\; in an autonomous vehicle you c
 are much more about misdetections that are close to the vehicle.\nIn this 
 talk I'll walk through our paper containing our simple solution to this tr
 icky problem.\n\nHere is a more formal abstract:\n\nThere is a longstandin
 g interest in capturing the error behaviour of object detectors by finding
  images where their performance is likely to be unsatisfactory. In real-wo
 rld applications such as autonomous driving\, it is also crucial to charac
 terise potential failures beyond simple requirements of detection performa
 nce. For example\, a missed detection of a pedestrian close to an ego vehi
 cle will generally require closer inspection than a missed detection of a 
 car in the distance. The problem of predicting such potential failures at 
 test time has largely been overlooked in the literature and conventional a
 pproaches based on detection uncertainty fall short in that they are agnos
 tic to such fine-grained characterisation of errors. In this work\, we pro
 pose to reformulate the problem of finding “hard” images as a query-ba
 sed hard image retrieval task\, where queries are specific definitions of 
 “hardness”\, and offer a simple and intuitive method that can solve th
 is task for a large family of queries. Our method is entirely post-hoc\, d
 oes not require ground-truth annotations\, is independent of the choice of
  a detector\, and relies on an efficient Monte Carlo estimation that uses 
 a simple stochas- tic model in place of the ground-truth. We show experime
 ntally that it can be applied successfully to a wide variety of queries fo
 r which it can reliably identify hard images for a given detector without 
 any labelled data. We provide results on ranking and classification tasks 
 using the widely used RetinaNet\, Faster-RCNN\, Mask-RCNN\, and Cascade Ma
 sk- RCNN object detectors. The code for this project is available at https
 ://github.com/fiveai/hardest.
LOCATION:Lecture Theatre 2
END:VEVENT
END:VCALENDAR