FazeTrak

Smart face-tracking webcam system with gesture control and ESP32-C3 servo integration

---

📋 Table of Contents

• Overview
• Features
• Project Structure
• Hardware Requirements
• Software Requirements
• Installation
• Usage
• System Architecture
• Documentation
• Contributing

---

🎯 Overview

FazeTrak is an intelligent webcam system capable of real-time face detection, gesture-based control, and autonomous tracking. Developed for CS3283 - Embedded Systems Project (Semester 5), the system combines computer vision, embedded systems, and robotics to create a self-centering camera platform.

The system uses:

• Desktop Application (PyQt5) - Face/gesture detection and servo control
• ESP32-C3 Microcontroller - Servo motor driver and command processing
• Pan-Tilt Mechanism - Dual servo motors for 2-axis tracking
• Virtual Camera Integration - Stream tracked video to Zoom, OBS, etc.

---

✨ Features

• ✅ Real-time Face Detection - Powered by MediaPipe
• ✅ Gesture-based Control - Lock/unlock tracking with hand gestures
• ✅ Autonomous Tracking - Servos automatically center face in frame
• ✅ Virtual Camera Support - Stream to OBS, Zoom, Teams
• ✅ Desktop Application - Full PyQt5 GUI with live preview

---

📁 Project Structure

Face-Tracking-WebCam/
├── desktop-app/
│   ├── app/
│   │   ├── main.py                 # PyQt5 GUI entry point
│   │   ├── video_thread.py         # Video capture & face tracking
│   │   ├── servo_controller.py     # Serial communication with ESP32-C3
│   │   ├── gesture.py              # Hand gesture detection
│   │    parameters
│   ├── requirements.txt
│   └── README.md
├── device-firmware/
│   ├── src/
│   │   └── main.cpp                # ESP32-C3 servo control firmware
│   ├── platformio.ini
│   └── README.md
├── assets/
│   ├── esp32_c3_supermini.jpeg
│   ├── circuit.png
│   └── ...
├── README.md
└── .gitignore

---

🔧 Hardware Requirements

• Microcontroller: ESP32-C3 SuperMini
• Servos: 2× SG90 Pan-Tilt Servos (180° range)
• Webcam: Logitech C270 HD (or compatible USB camera)
• Servo Bracket: Pan-Tilt mount
• USB Cable: Type-C (ESP32-C3 connection)

Hardware Connections

ESP32-C3 Pin	Component
GPIO 4	Pan Servo Signal
GPIO 3	Tilt Servo Signal
GND	Servo GND
5V	Servo VCC (via external PSU)

---

💻 Software Requirements

Desktop Application

• Python 3.8+
• PyQt5
• OpenCV (cv2)
• MediaPipe
• pyvirtualcam

ESP32-C3 Firmware

• PlatformIO
• ESP32Servo library

---

📦 Installation

Desktop Application Setup

1. Clone the repository

   git clone https://github.com/AkinduID/FazeTrak.git
   cd FazeTrak/desktop-app

2. Create virtual environment (recommended)

   python -m venv venv
   source venv/bin/activate  # On Windows: venv\Scripts\activate

3. Install dependencies

   pip install -r requirements.txt

ESP32-C3 Firmware Setup

1. Install PlatformIO

   pip install platformio

2. Navigate to firmware directory

   cd device-firmware

3. Build and upload

   platformio run -t upload

---

🚀 Usage

Starting the Desktop Application

cd desktop-app
python -m app.main

Controls:

• Start Tracking - Begin face detection and servo control
• Stop Tracking - Stop the tracking system
• Hand Gestures:
  • ✋ Open Palm → Release face lock
  • ✊ Closed Fist → Lock face and start tracking

Visual Feedback:

• 🔵 Blue circle - Raw face detection (noisy)
• 🟡 Yellow circle - Kalman Filter estimate (smooth)
• 🟢 Green rectangle - Face bounding box
• 🟢 Green line - Servo aim vector

---

🏗️ System Architecture

┌─────────────────────────────────────┐
│      Desktop Application (PyQt5)    │
├─────────────────────────────────────┤
│  ┌───────────────────────────────┐  │
│  │  Video Input (Webcam)         │  │
│  └──────────────┬────────────────┘  │
│                 │                    │
│  ┌──────────────▼────────────────┐  │
│  │  Face Detection (MediaPipe)   │  │
│  └──────────────┬────────────────┘  │
│                 │                    │
│  ┌──────────────▼────────────────┐  │
│  │  Servo Control Algorithm      │  │
│  └──────────────┬────────────────┘  │
│                 │                    │
└─────────────────┼────────────────────┘
                  │ Serial @ 115200 baud
         ┌────────▼─────────┐
         │   ESP32-C3       │
         ├──────────────────┤
         │ Servo Driver     │
         │ (PWM Generator)  │
         └────────┬─────────┘
                  │
         ┌────────┴──────────┐
         │                   │
      ┌──▼──┐          ┌──▼──┐
      │ Pan │          │Tilt │
      │Servo│          │Servo│
      └─────┘          └─────┘

---

📚 Documentation

For detailed progress and technical documentation, see:

• GitHub Wiki - Weekly progress and technical details
• Desktop App README
• Firmware README

---

🤝 Contributing

Contributions are welcome! Please follow these steps:

1. Fork the repository
2. Create a feature branch (git checkout -b feature/AmazingFeature)
3. Commit your changes (git commit -m 'Add AmazingFeature')
4. Push to the branch (git push origin feature/AmazingFeature)
5. Open a Pull Request