Rubber Hits the Ground: Insights from using DeepFake-O-Meter for Deepfake Detection Evaluation

PURPOSE: THE RISE OF SYNTHETIC MEDIA, OR DEEPFAKES, POSES SIGNIFICANT RISKS TO MEDIA INTEGRITY, TRUST IN SOCIAL MEDIA, POLITICS, NATIONAL DEFENSE, AND CYBERSECURITY. THIS PROJECT AIMS TO MITIGATE THESE NEGATIVE IMPACTS BY DEVELOPING AND IMPROVING THE DEEPFAKE-O-METER PLATFORM, WHICH INTEGRATES MULTIPLE STATE-OF-THE-ART DETECTION ALGORITHMS TO IDENTIFY DEEPFAKE IMAGES, VIDEOS, AND AUDIO. BY FOSTERING A COMMUNITY OF USERS, DEVELOPERS, AND RESEARCHERS, THE PLATFORM SEEKS TO ENHANCE DEEPFAKE DETECTION AND PROVIDE VALUABLE INSIGHTS FOR FUTURE IMPROVEMENTS.ACTIVITIES TO BE PERFORMED:1. DATA COLLECTION AND CLEANING (JUN. 2024 - AUG. 2024): COLLECT AND ORGANIZE USAGE DATA FROM THE DEEPFAKE-O-METER PLATFORM, INCLUDING USER REGISTRATION, USAGE PATTERNS, SHARED SAMPLES, FEEDBACK, AND DETECTOR PERFORMANCE.2. COMPREHENSIVE EVALUATION AND ANALYSIS (SEP. 2024 - NOV. 2024): EVALUATE 18 STATE-OF-THE-ART DEEPFAKE DETECTORS COVERING IMAGES, VIDEOS, AND AUDIO USING BOTH PUBLIC DATASETS AND REAL-WORLD DATA COLLECTED FROM THE PLATFORM. ANALYZE APPLICATION SCOPE, DETECTION RELIABILITY, AND EFFICIENCY OF EACH DETECTOR.3. PLATFORM IMPROVEMENT (DEC. 2024 - FEB. 2025): ENHANCE THE DEEPFAKE-O-METER PLATFORM BASED ON ANALYSIS RESULTS. INTEGRATE ADDITIONAL STATE-OF-THE-ART DETECTORS, IMPROVE DETECTION RESULT REPORTING, UPDATE THE USER INTERFACE, AND ENHANCE PLATFORM SCALABILITY AND STABILITY.4. INSIGHT SUMMARY AND FUTURE DIRECTIONS (MAR. 2025 - MAY. 2025): SUMMARIZE KEY INSIGHTS FROM THE APPLICATION OF DETECTORS IN REAL-WORLD SCENARIOS. OUTLINE FUTURE RESEARCH DIRECTIONS TO FURTHER IMPROVE DEEPFAKE DETECTION.EXPECTED OUTCOMES: DEVELOPMENT AND ENHANCEMENT OF DEEPFAKE-O-METER V2.0, INCORPORATING IMPROVED ALGORITHMS FOR DETECTING DEEPFAKE IMAGES, VIDEOS, AND AUDIO. COMPREHENSIVE EVALUATION AND PERFORMANCE INSIGHTS OF CURRENT DEEPFAKE DETECTORS. A MORE USER-FRIENDLY, EFFICIENT, AND SCALABLE DETECTION PLATFORM. VALUABLE DATA AND INSIGHTS FOR THE RESEARCH COMMUNITY TO FURTHER ADVANCE DEEPFAKE DETECTION TECHNOLOGIES.INTENDED BENEFICIARIES: GENERAL PUBLIC: IMPROVED MEDIA INTEGRITY AND REDUCED RISKS OF MISINFORMATION AND IMPERSONATION. RESEARCHERS/DEVELOPERS: ACCESS TO AN OPEN-ACCESS PLATFORM FOR STUDYING AND DEVELOPING DEEPFAKE DETECTION METHODS, SHARING TOOL PERFORMANCES AND RESOURCES WITH EACH OTHER. USERS: AN ACCESSIBLE PLATFORM TO VERIFY MEDIA AUTHENTICITY, ENHANCING TRUST IN DIGITAL CONTENT. APPLICANT: THE PROJECT ENABLES THE RESEARCH TEAM TO ACCESS NECESSARY RESOURCES, TECHNOLOGY, AND EXPERTISE TO ENHANCE THE DEEPFAKE-O-METER PLATFORM. SUCCESSFUL EXECUTION OF THE PROJECT INCREASES THE VISIBILITY OF THE TEAM, LEADING TO POTENTIAL FUTURE COLLABORATIONS AND FUNDING OPPORTUNITIES. NIST: BY FUNDING THIS PROJECT, NIST DEMONSTRATES ITS COMMITMENT TO SUPPORTING CUTTING-EDGE RESEARCH AND INNOVATION IN THE FIELD OF GENERATIVE AI AND DEEPFAKE DETECTION. THE OUTCOMES OF THIS PROJECT CAN INFORM POLICYMAKING BY PROVIDING EVIDENCE-BASED INSIGHTS INTO THE EFFECTIVENESS OF DEEPFAKE DETECTION TECHNOLOGIES. SUPPORTING A PROJECT WITH SIGNIFICANT SOCIAL IMPORTANCE ENHANCES NIST?S REPUTATION AS A LEADER IN AI INNOVATION.SUBRECIPIENT ACTIVITIES: THERE ARE NO PLANNED SUBAWARDS.