Leaderboard submissions for the Activity-Classification task are now closed. Thanks to all participating teams !
The AC task is to assign a single activity class label to each video clip from a set of predefined classes and provide a confidence score. The system will be presented nominally 3-second trimmed video clips, each clip containing one of the activity classes and the AC system must output the activity class and a confidence score with higher values indicating the clip is more likely to contain that activity.
The TAD task is to automatically detect and temporally localize each activity instance in untrimmed video. The system will be presented nominally 45-second,untrimmed video clips, where no activity instances overlap during the same duration. For each detected activity instance, the TAD system must output the activity class, a start frame, end frame and confidence score with higher values indicating the instance is more likely to have occured.
Please refer to the evaluation plan below for the detailed tasks and relevant metrics.
|May 2, 2022||OpenFAD evaluation plan available
Call for participation in AC challenge
AC training/validation set announcement
|September 5, 2022||AC leaderboard opens|
|December 2, 2022||AC submission deadline|
|December 15, 2022||AC final report available (top-performers notification)|
|January 7, 2023||Workshop on Fine Grained Activity Detection (FGAD'23) at WACV2023|
CAP contains a new annotated dataset of fine-grained and coarse-grained activity classes of consented people, curated using the Visym Collector platform.
The CAP training and validation dataset are available for public download in the form of a single archive file containing videos and metadata at the CAP dataset distribution site. The CAP dataset site includes a dataset explorer, dataset summary and activity definitions.
In order to participate an user account is required. Signing up for an account is an easy two step process using email-confirmation explained here. The help center additionaly shows how to reset a lost password or unlock the account.
After creating an account and signing into the participation dashboard, please follow the registration workflow on the left side in order to obtain access to the data.
In the next workflow step please select which track to participate in (AC/TAD)
System output submission to the leaderboard must be made through the web-platform using the submission instructions described on the webpage (Submission Management). To prepare your submission, you will first make .tar file of your system output CSV file via the UNIX command ‘tar cvzf [submission-name].tgz [submission-file-name].csv’ and then make your submission as follows:
The OpenFADScorer package (to be public soon) contains an output format checker that validates the submission. To validate your system output locally please use the following command-line:
All clips must be processed independently of each other within a given task and across all tasks, meaning content extracted from the video clip data must not affect the processing of another clip.
While participants may report their own results, participants may not make advertising claims about their standing in the evaluation, regardless of rank, or winning the evaluation, or claim NIST endorsement of their system(s). The following language in the U.S. Code of Federal Regulations (15 C.F.R. § 200.113)14 shall be respected: NIST does not approve, recommend, or endorse any proprietary product or proprietary material. No reference shall be made to NIST, or to reports or results furnished by NIST in any advertising or sales promotion which would indicate or imply that NIST approves, recommends, or endorses any proprietary product or proprietary material, or which has as its purpose an intent to cause directly or indirectly the advertised product to be used or purchased because of NIST test reports or results.
At the conclusion of the evaluation, NIST may generate a report summarizing the system results for conditions of interest. Participants may publish or otherwise disseminate these charts, unaltered and with appropriate reference to their source.