Workshop on Fine Grained Activity Detection

Summary
The “1st International Workshop on Fine Grained Activity Detection (FGAD’23)” will focus on fine grained activity detection in hand-held video.

Large scale activity recognition has made remarkable progress driven by the curation of large scale labeled video datasets. Evaluation tasks in these datasets include activity classification, activity and object detection and localization, action prediction, episodic memory for object instance retrieval, object interactions with hands/tools/people, speaker prediction and scene diarization in long duration videos.

However, performance on these important tasks remains limited by the scale, quality and applicability of data. While there are many large-scale video datasets for pretraining activity recognition, these datasets are nearly all scraped from social media platforms such as YouTube or Instagram. The activities in these datasets do not reflect true diversity of activities in the real world, and instead represent a biased and unbalanced sample of labels that are interesting enough for social media. Specifically, these datasets lack coverage of:

  • Common activities that we all perform each day, such as dressing or grooming that are not typically captured on video because they are rarely performed in front of a camera or are too boring to share.
  • Fine activities that are closely related and may be easily confused, such as putting on socks vs. putting on shoes or talking on a phone vs. smoothing your hair.
  • Diverse activities that are different ways of performing the same activity in the wild, such as activities viewed from behind or interacting with different objects.

Our goal is to bring together researchers interested in the problem of activity recognition of common, fine and diverse human activities. We will provide a forum for discussion, and provide a new large-scale training/validation/test set of fine-grained activities collected from around the world to focus discussion and characterize progress in this emerging research field.

We will have three invited keynote talks from experts in the field. We will also have two talks from the top performers on the Open Fine-grained Activity Detection (OpenFAD) Activity Classification challenge that uses the Consented Activities of People (CAP) dataset. We will select paper submissions to be published in the WACV’23 proceedings after a double-blind review on topics related to activity classification and detection. The workshop will provide a platform for researchers to share research experiences and foster collaboration.

Call For Papers

A one day workshop on Fine Grained Activity Detection to be held on January 7, 2023 at the 2023 IEEE Winter Conference on Applications of Computer Vision (WACV’23) in Waikoloa, Hawaii.

Every day, humans perform many closely related activities that involve subtle discriminative motions, such as putting on a shirt vs. putting on a jacket, or shaking hands vs. giving a high five. Activity recognition by ethical visual AI could provide insights into our patterns of dailylife, however existing activity recognition datasets do not capture the massive diversity of these human activities around the world. The goal of this workshop is to bring together research the representation and recognition of fine grained activities of daily life around the world, and provide a forum for researchers to share expertise on this emerging problem.

The workshop will host the first Open Fine-grained Activity Detection (OpenFAD) Challenge. OpenFAD is an activity classification and detection evaluation to measure how well systems can automatically classify or temporally detect fine-grained activities collected from handheld devices. This challenge uses the Consented Activities of People (CAP) dataset which is a new dataset of 1.45M videos of 512 fine grained activity classes of consented people. We will host a leaderboard for the Activity Classification (AC) submission track to characterize the state-of-the-art on fine grained activity classification on this new dataset. The leaderboard evaluation will be performed on a video test set available to the challenge participants, with sequestered ground truth.

Accepted papers will be published in the workshop proceedings, and the top two teams on the leaderboard will be invited to give a talk.

Workshop Schedule:

  • September 5, 2022: OpenFAD challenge opens for the Activity Classification (AC) track
  • October 10, 2022 / 5PM EST: Workshop paper submission deadline
  • November 19, 2022 / 5PM EST: Final AC track submission and camera ready deadline
  • December 15, 2022: AC track final report available, top-performers notified
  • January 7, 2023: OpenFAD workshop at WACV’23
Organizers
  • Jeffrey Byrne (Visym Labs)
  • Jon Fiscus (NIST)
  • Yooyoung Lee (NIST)
  • Hilde Kuehne (Goethe University Frankfurt/IBM)
  • Yogesh Singh Rawat (UCF)
  • Mubarak Shah (UCF)
  • Alex Hauptman (CMU)
  • Rama Chellappa (JHU)
Program Committee
  • Lukas Diduch (NIST/Dakota-consulting, inc.)
  • Yijun Qian Liu (CMU)
  • Wenhe Liu (CMU)
  • Chen Chen (UCF)
  • Praveen Tirupattur (UCF)
  • Madeline Schiappa (UCF)
  • Aayushjungbahadur Rana (UCF)
  • Longlong Jing (CUNY)
Invited Speakers
  • TBD
OpenFAD Challenge
In this WACV FGAD’23 workshop we plan to present the research findings of the Open Fine-grained Activity Detection (OpenFAD) Evaluation. OpenFAD is an activity classification and detection evaluation to measure how well systems can automatically classify or temporally detect fine-grained activities collected from Consented Activities of People (CAP) data using handheld devices. The Activity Classification (AC) evaluation challenge task will be discussed at the workshop. The AC task is to assign a single activity class label to each video clip from a set of predefined classes and provide a confidence score. See OpenFAD for details.
Challenge Timeline:
Date Events
September 5, 2022 AC leaderboard opens
December 2, 2022 AC submission deadline
December 15, 2022 AC final report available (top-performers notification)
January 7, 2023 FGAD’23 Workshop at WACV2023
Hawaii Standard Time (HST)
Time Program
9:00 - 9:05 am Workshop introduction
9:05 - 9:45 am OpenFAD Challenge overview
9:45 - 10:30 am Invited talk: Hilde Kuehne (Goethe U. Frankfurt)
10:30 - 10:45 am Coffee break
10:45 - 11:30 am Invited talk: Jeffrey Byrne (Visym Labs)
11:30 - 12:15 pm OpenFAD Challenge results: Jon Fiscus (NIST)
12:00 - 1:30 pm Lunch
1:30 - 2:00 pm Challenge Winner talk (BUPT-MCPRL)
2:00 - 2:45 pm Invited talk: Yogesh S. Rawat (UCF)
2:45 - 3:00 pm Q&A and wrapup
Paper Submissions

Authors are invited to submit original work not currently under consideration elsewhere. The FGAD’23 workshop submissions will be handled via the CMT review site.

Accepted papers will be allocated 8 pages in the proceedings (a paper can be up to 8 pages + the references). The manuscripts should be submitted in PDF format and should follow the requirements of the IEEE WACV paper format. All work submitted to WACV workshop is considered confidential until the papers appear. Accepted papers will be included in the Proceedings of IEEE WACV 2023 & Workshops and will be sent for inclusion into the IEEE Xplore digital library. The author toolkit is available both on Overleaf and Github, and papers will be submitted via CMT. For more details, please refer to the following documents:

Workshop Registration

You will be able to register for the FGAD'23 workshop soon.

Contacts
For any questions about the FGAD'23 workshop, please contact us via the workshop email address: FGAD'23 workshop email: fgad@list.nist.gov