2nd Workshop on Fine Grained Activity Detection

New The leaderboard competition has ended! Congratulations goes to:
  • Our co-leaders on the Activity Classification Task: Cloudwalk Technology and DeepGlint both with a 0.920 mAP.
  • Our leader on the Temporal Activity Detection Task: Waseda University, Meisei University, and SoftBank Corporation with a mAP of 0.212.
Each of the three teams will be given a 20-minute speaking slot at the 2nd International Workshop on Fine Grained Activity Detection (ICCV FGAD’23)
Summary
The “2nd International Workshop on Fine Grained Activity Detection (ICCV FGAD’23)” will focus on fine grained activity detection in hand-held video.

Large scale activity recognition has made remarkable progress driven by the curation of large scale labeled video datasets. Evaluation tasks in these datasets include activity classification, activity and object detection and localization, action prediction, episodic memory for object instance retrieval, object interactions with hands/tools/people, speaker prediction and scene diarization in long duration videos.

However, performance on these important tasks remains limited by the scale, quality and applicability of data. While there are many large-scale video datasets for pretraining activity recognition, these datasets are nearly all scraped from social media platforms such as YouTube or Instagram. The activities in these datasets do not reflect true diversity of activities in the real world, and instead represent a biased and unbalanced sample of labels that are interesting enough for social media. Specifically, these datasets lack coverage of:

  • Common activities that we all perform each day, such as dressing or grooming that are not typically captured on video because they are rarely performed in front of a camera or are too boring to share.
  • Fine activities that are closely related and may be easily confused, such as putting on socks vs. putting on shoes or talking on a phone vs. smoothing your hair.
  • Diverse activities that are different ways of performing the same activity in the wild, such as activities viewed from behind or interacting with different objects.

Our goal is to bring together researchers interested in the problem of activity recognition of common, fine and diverse human activities. We will provide a forum for discussion, and provide a new large-scale training/validation/test set of fine-grained activities collected from around the world to focus discussion and characterize progress in this emerging research field.

We will have invited keynote talks from experts in the field. We will also have two talks from the top performers on the Open Fine-grained Activity Detection (OpenFAD) Activity Classification and Temporal Activity Detection challenges that uses the Consented Activities of People (CAP) dataset.

Call For Papers

A half day workshop on Fine Grained Activity Detection to be held on October 2, 2023 at the 2nd Workshop on Fine Grained Activity Detection to be held on October 2, 2023 at the ICCV’23 in Paris, France.

Every day, humans perform many closely related activities that involve subtle discriminative motions, such as putting on a shirt vs. putting on a jacket, or shaking hands vs. giving a high five. Activity recognition by ethical visual AI could provide insights into our patterns of daily life, however existing activity recognition datasets do not capture the massive diversity of these human activities around the world. The goal of this workshop is to bring together research the representation and recognition of fine grained activities of daily life around the world, and provide a forum for researchers to share expertise on this emerging problem.

The workshop will host the second Open Fine-grained Activity Detection (OpenFAD) Challenge. OpenFAD is an activity classification and detection evaluation to measure how well systems can automatically classify or temporally detect fine-grained activities collected from handheld devices. This challenge uses the Consented Activities of People (CAP) dataset which is a new dataset of 1.45M videos of 512 fine grained activity classes of consented people. We will host a leaderboard for the Activity Classification (AC) and Temporal Activity Detection (TAD) submission tracks to characterize the state-of-the-art on this new dataset. The leaderboard evaluation will be performed on a video test set available to the challenge participants, with sequestered ground truth.

Accepted papers will be published in the workshop proceedings, and the top two teams on the leaderboard will be invited to give a talk.

Workshop Schedule:

  • May 5, 2023: TAD/AC evaluation leaderboard opens
  • August 31, 2023 (23:59 EDT): TAD/AC submission deadline
  • September 5, 2023: Challenge Winner notification
  • September 18, 2023: TAD/AC final report available
  • October 2, 2023: Workshop on 2nd Fine Grained Activity Detection (ICCV FGAD'23) at ICCV23
Organizers
  • Jeffrey Byrne (Visym Labs)
  • Jon Fiscus (NIST)
  • Hilde Kuehne (Goethe University Frankfurt/IBM)
  • Yogesh Singh Rawat (UCF)
  • Lukas Diduch (NIST/Dakota-consulting, inc.)
  • Mubarak Shah (UCF)
  • Alex Hauptman (CMU)
  • Rama Chellappa (JHU)
Invited Speakers
  • Limin Wang (Nanjing University)
  • Joao Carreria (Google DeepMind)
OpenFAD Challenge
In this ICCV FGAD’23 workshop we plan to present the research findings of the Open Fine-grained Activity Detection (OpenFAD) Evaluation. OpenFAD is an activity classification and detection evaluation to measure how well systems can automatically classify or temporally detect fine-grained activities collected from Consented Activities of People (CAP) data using handheld devices. The Temporal Activity Detection (TAD) and Activity Classification (AC) evaluation challenge tasks will be discussed at the workshop. The TAD task is to automatically detect and temporally localize each activity instance and provide a confidence score in untrimmed video. The AC task is to assign a single activity class label to each video clip from a set of predefined classes and provide a confidence score. See OpenFAD for details.
Challenge Timeline:
Date Events
May 5, 2023 TAD/AC evaluation leaderboard opens
September 15 August 31, 2023 (23:59 EDT) TAD/AC submission deadline
September 18 September 5, 2023 Challenge Winner notification
September 21 September 18, 2023 TAD/AC final report available
October 2, 2023 2nd Fine Grained Activity Detection (ICCV FGAD'23) Workshop at ICCV23
ICCV FGAD ‘23 Workshop Agenda
Monday October 2, 2023, 9:00am-1:00pm (CEST)
Time Events
09:00 - 09:05: Welcome and introduction
09:05 - 09:30: Jeffrey Byrne, Visym Labs, Fine Grained Activity Detection (FGAD'23) Challenge Overview
09:30 - 10:15: Limin Wang, Nanjing University, Invited Talk #1 (45 min)
10:15 - 10:45: Jon Fiscus, NIST, Fine Grained Activity Detection (FGAD'23) Challenge Results
10:45 - 11:00: Coffee Break
11:00 - 11:20: Activity Classification Challenge Co-Winner, DeepGlint
11:20 - 11:40: Activity Classification Challenge Co-Winner, Cloudwalk Technology
11:40 - 12:00: Temporal Activity Detection Challenge Winner, WasedaMeiseiSoftbank
12:00 - 12:45: Joao Carreria, Google DeepMind, Invited Talk #2 (45 min)
12:45 - 13:00: Open discussion
Submissions

Participants are encouraged to submit results to the challenge leaderboard. The workshop will feature an in-person poster session for participants to share the design of submitted systems. Our goal is to bring together researchers interested in the problem of activity recognition of common, fine and diverse human activities, and support discussion on this emerging topic.

Workshop Registration

You are able to register for the ICCV FGAD'23 workshop by attending ICCV'23.

Contacts
For any questions about the ICCV FGAD'23 workshop, please contact us via the workshop email address: FGAD workshop email: fgad@list.nist.gov