IUB RAG LLM

Helping students access information from their university.

Usage

Ubuntu 24.04

Install dependencies:

sudo apt update
sudo apt upgrade
sudo apt satisfy -y "python3 (>= 3.10), python3-pip (>= 3.10), python3-venv (>= 3.10)"
sudo apt install -y libatk1.0-0t64 libatspi2.0-0t64 libxcomposite1 libxdamage1 libxfixes3 libxrandr2 libgbm1 libasound2t64

Download project:

PROJECT_INSTALL_DIR=~/iub-rag-llm  # change this
mkdir -p "$PROJECT_INSTALL_DIR"
curl -L "https://github.com/znnahiyan/iub-rag-llm/archive/refs/heads/main.tar.gz" | tar -xzvf "-" -C "$PROJECT_INSTALL_DIR"

Set up environment:

cd "$PROJECT_INSTALL_DIR"
python3 -m venv .venv
source .venv/bin/activate
pip install --upgrade pip
pip install --editable .
playwright install --only-shell chromium

Run scraper:

PYTHONPATH=src/ python3 -m scraper --source web --spider iub
PYTHONPATH=src/ python3 -m scraper --source pdf --pdf-dir data/pdf/

Set up access to APIs:

cp -v .example.env .env
nano .env

Run indexer:

PYTHONPATH=src/ python3 -m indexer --feed-path data/feeds/items-web-*.jsonl --namespace web
PYTHONPATH=src/ python3 -m indexer --feed-path data/feeds/items-pdf-*.jsonl --namespace pdf

Run server:

PYTHONPATH=src/ streamlit run src/server/__main__.py