Deepfakes in videoconferencing: Fraunhofer develops a real-time warning system

The Fraunhofer Institute for Secure Information Technology (SIT) in Darmstadt has developed a real-time warning system designed to detect deepfakes during video calls. The project was part of the “ATHENE” research center, funded by Germany’s Federal Ministry of Education and Research and the state of Hesse, focusing on cybersecurity. The software, created in collaboration with the Fraunhofer site in Heilbronn, aims to alert users to potential deepfake threats while conversations take place. The rise of deepfake technology has made manipulated audio and video recordings, known as deepfakes, a common tool for cybercriminals. These AI-generated clones can mimic a person's appearance, voice, and mannerisms with increasing accuracy. In a test conducted by heise in partnership with cybersecurity firm Whitelisthackers, researchers were able to deceive a colleague using a live deepfake within seconds, gaining access to confidential information. This highlights how easily such attacks can occur, especially in video conferencing environments where technical imperfections are often overlooked. Detecting deepfakes in video calls presents unique challenges. Users are accustomed to minor transmission issues like lag or pixelation, making it harder to distinguish between genuine human behavior and artificial manipulation. Prof. Martin Steinebach, head of IT forensics at Fraunhofer SIT, notes that developing reliable deepfake detectors for video calls is particularly difficult. Factors such as fluctuating network quality, compression artifacts, and automatic blurring filters can obscure visual anomalies that older detection methods rely on. To address these challenges, the new system combines audio and video analysis using machine learning algorithms. It provides a probability score indicating whether the person on screen is authentic. However, the final decision rests with the user, who may choose to ask follow-up questions or verify through alternative communication channels. Importantly, all processing occurs locally on a device equipped with a modern graphics card and 12 gigabytes of GPU memory. This ensures that image and sound data remain on the user’s machine, avoiding cloud-based transmission. Such a design makes the solution suitable for both private and professional settings where confidentiality is crucial. Fraunhofer has not yet disclosed specific metrics regarding the system’s accuracy or frequency of false alarms. Additionally, it remains unclear whether participants must consent to biometric analysis, given that facial and vocal features are evaluated. Data protection regulations, including the EU’s General Data Protection Regulation (GDPR), would likely apply. Nevertheless, the demonstrator version is still in testing, and legal aspects are yet to be resolved. Moving forward, the researchers plan to engage with companies like Zoom and Microsoft Teams to integrate the technology into existing platforms in a secure and privacy-compliant manner. Despite ongoing improvements, Prof. Steinebach emphasizes that technology alone is insufficient. Even with advanced detection tools, users should always maintain a secondary communication channel for critical tasks, such as verifying financial transactions during a video call. This approach underscores the importance of combining technological safeguards with human vigilance.