Projects

Sign Language Translation Using Self-Supervised Models
TTIC’s official submission to WMT-SLT 23. Our approach explores the advantages of using large-scale self-supervised pre-training in the task of sign language translation, over more traditional approaches that rely heavily on supervision, along with costly labels such as gloss annotations. Our model achieves the highest machine-evaluation score in the Swiss-German Sign Language to German track, nearly doubling the second highest BLEU score.
Self-supervised Video Transformers for Isolated Sign Language Recognition
In-depth analysis of various self-supervision methods for isolated sign language recognition (ISLR). We consider four recently introduced transformer-based approaches to self-supervised learning from videos, and four pre-training data regimes, and study all the combinations on the WLASL2000 dataset. We achieve new state of the art in WLASL2000 by leveraging continuous sign language videos.

The role of scientific and pseudo-scientific research in online misinformation
Collaboration with GATech. We investigate the use of medical journals and popular preprint repositories by major vaccine misinformation spreaders, their techniques, and their impact.
Bayesian Media Forensics
Collaboration with NYU Tandon. We develop a benchmark of different Bayesian deep learning approaches in common media forensics problems: scaling factor estimation, manipulation classification, synthetic media detection, etc. The code from this project will be part of a larger forensics framework that can be found here.
Cross-Platform Post-Video Misinformation: Taxonomy, Characteristics, and Detection
Collaboration with GATech. Previous research has found that misinformation social media posts containing misinformation with additional media do not get detected as such by most misinformation detection frameworks (and less than 1% ever get flagged as misinformation in the wild). In this work, we construct a detailed taxonomy based on exploration of post-video pairs extracted from Twitter, Facebook, and Reddit, and develop classifiers for identifying them.

AutoTag: Automated Metadata Tagging for Film Post-Production
MTAP 2023.
Advised by Professors Dennis Shasha (NYU Courant) and Scandar Copti (NYUAD Film). Collection of film post-production tools developed for Adobe Premiere Pro, specifically for screenplay scene-to-media matching, cinematographic shot identification, and transcription. This project includes case studies with filmmakers, cinematographers, and editors, in projects that include short fiction films, experimental projects, and a feature film.

Narratives and Needs: Analysing Experiences of Cyclone Amphan Using Twitter Discourse
NeurIPS 2020 Tackling Climate Change with Machine Learning Workshop and IJCAI 2021 AI for Social Good Workshop.
Collaboration between IWMI and Solve for Good. Exploration of methodologies that leverage Twitter discourse to characterise narratives and identify unmet needs in response to Cyclone Amphan, which affected 18 million people in 2020.
AI for Good COVID-19 Simulator for Refugee Camps
Volunteer project.
Project with AI for Good UK. Network-based SEIRS modelling of COVID-19, using information from their geographic distribution, daily routines, healthcare access, and demographic information. Originally developed for Moria (Greece) and later extended to multiple refugee camps.