Humming Bird: Classical Music Recognition

A full-stack application that identifies a classical music piece from a user's hummed melody.
It performs real-time audio processing, extracts pitch intervals, compares sequences using DTW + k-NN, and validates predictions using a Longest Ascending Chain algorithm.

---

Features

Real-time Pitch Detection

• Uses the Web Audio API to access the microphone.
• Implements Autocorrelation (with FFT fallback) to estimate pitch.
• Extracts pitch every few milliseconds for real-time feedback.

Interval Encoding

• Converts detected pitches into musical intervals modulo 12.
• Produces a short melody "signature."

DTW-kNN Classification (Backend)

• Python/FastAPI backend analyzes the interval sequence.
• Uses Dynamic Time Warping (DTW) for robust time-series comparison.
• Applies k-Nearest Neighbors (k = 3) over pre-indexed melodies.

Melody Indexing (Backend)

• python.py loads MP3 files and extracts short melody fragments.
• Intervals are letter-encoded and stored in a melody hash table.

Chain Validation (Frontend)

• Uses Longest Ascending Chain of splice-order predictions.
• Ensures the detected fragments appear sequentially within the true song.

---

Setup & Installation

This project includes:

• A React frontend
• A FastAPI/Python backend

Run them at the same time.

---

1. Prerequisites

You will need Node.js and Python 3.

Backend Setup (Python)

# Create and activate virtual environment
python -m venv venv
source venv/bin/activate      # Windows: venv\Scripts\activate

# Install required libraries
pip install fastapi uvicorn numpy librosa pandas matplotlib seaborn

For the Python Backend:

# It's recommended to use a virtual environment
python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

# Install required libraries
pip install fastapi uvicorn numpy librosa pandas matplotlib seaborn
For the React Frontend:

# Assuming you are in the project root directory
npm install

2. Prepare the Music Database (Backend)

The python.py script requires a directory of classical music MP3 files to build its internal hash table.

1. Create a directory named pieces inside the backend source folder (e.g., in the same directory as python.py).=
2. Populate this directory with your classical music MP3 files (e.g., Beethoven_Symphony5a.mp3).
3. The python.py script is designed to build the melody hash on startup by calling build_melody_hash().

3. Running the Application

Step A: Start the Backend (FastAPI)

Open a terminal, navigate to the backend directory, and run the API server.

python api.py
# The server should start on http://0.0.0.0:8000

Step B: Start the Frontend (React)

Open a separate terminal, navigate to the frontend directory, and start the React application.

npm start
# The app should open in your browser on http://localhost:3000

Click the "Start Listening" button in the browser, and hum a melody from one of the indexed classical pieces!