Overview
Modern smart environments often leak raw sensor data to the cloud, and evolving regulations require a constant engineering effort to comply. I developed an LLM-based agentic system that autonomously constructs and executes privacy-preserving pipelines. By translating natural language requirements like "blur all faces" or "mute medical terms" into executable workflows, the system achieved an overall 69% success rate, drastically reducing the manual engineering typically required for GDPR compliance.
Intelligent Orchestration
The core of the system is a Planner Agent powered by Llama-3.3-70B. It interprets user intent and generates a JSON manifest of required tools. If a necessary tool (like a specific text-detector) doesn't exist, a Tool Generator writes and registers new Python code in a sandboxed environment in real-time.
To ensure reliability, I implemented a closed-loop recovery mechanism. If the verification module detects a failure (e.g., a missed face or audible keyword), the system automatically triggers a local retry with substitute tools or escalates to a full re-planning step, rescuing 24% of initially failed pipelines.
Multimodal Processing
For video, I engineered a YuNet-KCF-Kalman hybrid tracking system. This approach uses deep learning for initial detection, correlation filters for speed, and Kalman filtering to predict motion during occlusions, maintaining a miss ratio under 10%.
Audio privacy is handled through a two-stage pipeline: Whisper ASR for word-level timestamps and an LLM-based classifier to identify sensitive categories like addresses or PII. The system then applies temporal redaction via pydub, inserting beeps or silence with sub-second precision.
Automated Verification
Safety is enforced through automated heuristics. Video redaction is verified using Laplacian Variance to measure blur intensity, while audio redaction is checked by running the final output back through ASR—if the sensitive keywords are still detectable, the system fails the verification and prevents data release.
Impact & Results
This project demonstrates that agentic AI can bridge the gap between high-level privacy policies and low-level code execution. The system reduces deployment time from weeks of engineering to seconds of inference. It provides a blueprint for trustworthy, auditable, and user-driven privacy infrastructure in sensitive environments like hospitals and smart homes.