Using AI to help people: AIRO's participation in Google's Isolated Sign Language Recognition Challenge

11 May 2023

AIRO recently participated in the Kaggle competition on isolated ASL sign recognition and received a silver medal, securing the 16^th position out of 1165 teams! Many thanks to my teammates Joni and Maxim.

Participating in this challenge was particularly interesting as a sign language recognition (SLR) researcher, for several reasons:

• The dataset is the largest SLR dataset I have ever worked with, containing over 90k samples for 250 signs, making it a huge dataset in the SLR field.
• The dataset uses keypoints extracted with MediaPipe, which is also the feature extraction pipeline used in SignON.
• The dataset features one-handed signing, which is unique for SLR datasets but necessary for smartphone interaction.

In summary, the dataset is unique and valuable since it is a large dataset recorded in real-life environments with a specific application in mind.

During and after the competition, it was fascinating to see other people's results, the similarities and differences among them. Eventually, we learned several new techniques that we have integrated into SignON as of now.

Check out this Google I/O video on the competition!

Shortly after the completion of this competition, Google launched another ASL competition, this time on continuous fingerspelling recognition. This dataset is unique as it is the largest fingerspelling recognition dataset ever publicly released and features fingerspelling not in isolation ("the MNIST of SLR") but in continuous time.