Project Overview
At Troja, an early-stage deepfake prevention startup, built a backend server on NAVER Cloud Platform to serve deepfake generation and defense models. Enabled in-house testing of both face-swapping (inswapper) and adversarial defenses (LEAT, DIPA) in a single production environment.
Problem Definition & Goals
- Problem: Deepfake technology poses significant threats to personal privacy and information integrity. Existing defense methods are often tested only in research settings.
- Goal 1: Create a production-ready backend that can serve both deepfake generation and defense models for comprehensive security testing.
- Goal 2: Implement LEAT and DIPA defense mechanisms to protect images from unauthorized face-swapping attacks.
- Goal 3: Deploy on cloud infrastructure to enable scalable, accessible testing of defense strategies.
Key Features & Contributions
- Deepfake Creation Pipeline: Integrated inswapper model for realistic face-swapping, enabling systematic testing of defense mechanisms.
- LEAT Defense Implementation: Deployed adversarial attack defense that adds imperceptible perturbations to protect images from face-swapping.
- DIPA Defense Integration: Implemented image perturbation technique that disrupts deepfake generation while maintaining visual quality.
- Cloud Architecture: Designed scalable backend using Flask, uWSGI, and Nginx on NAVER Cloud Platform for production deployment.
- API Design: Created RESTful endpoints for image upload, processing, and defense application.
Technical Challenges & Solutions
- GPU Memory Management: Multiple deep learning models required efficient GPU memory allocation. Implemented model loading/unloading strategies and batch processing.
- Real-time Processing: Defense mechanisms needed to process images quickly. Used PyTorch optimization techniques and CUDA acceleration.
- Model Compatibility: Different defense methods had varying input/output formats. Created unified preprocessing and postprocessing pipelines.
- Production Stability: Ensured service reliability through uWSGI worker management, Nginx load balancing, and comprehensive error handling.
Results & Learnings
- Successful Deployment: Achieved stable production deployment serving both generation and defense models.
- Defense Effectiveness: LEAT and DIPA methods successfully disrupted face-swapping attempts while maintaining image quality above 95% SSIM.
- Key Learning: Gained deep understanding of adversarial ML, perturbation-based defenses, and practical challenges of deploying deep learning models.
- Infrastructure Skills: Developed expertise in cloud deployment, containerization, and building scalable ML serving infrastructure.