Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .env.example
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
PROJECT_NAME="Shelf"
PROJECT_VERSION="0.1.0"

# CORS settings.
CORS_ORIGINS="*"

# PostgreSQL settings.
POSTGRES_DB="shelf"
POSTGRES_USER="postgres"
Expand Down
134 changes: 68 additions & 66 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,99 +1,101 @@
# Shelf
A service for uploading, processing, and managing book files.

## Features
Shelf is a FastAPI-powered service for uploading, processing, and managing book files. It is designed to serve as the backend for mobile or web applications that need digital library capabilities.

- REST API for book management.
- Book metadata parsing (e.g. PDF, EPUB, MOBI) and management..
- Book file and cover storage that supports various storage methods (e.g. MinIO, Google Drive).
- Asynchronous processing for book uploads.
## Features

## Setup
- RESTful API for creating, retrieving, updating, and deleting books and shelves.
- Automatic metadata parsing for common ebook formats (PDF, EPUB, MOBI).
- Pluggable storage backends (local filesystem, MinIO, etc.).
- Asynchronous processing of uploads using Celery.
- Configurable CORS support for frontend integration.

1. **Clone the repository:**
```bash
git clone <repository-url>
cd Shelf
```
## Architecture overview

2. **Create and activate a virtual environment:**
```bash
python -m venv .venv
source .venv/bin/activate
```
```
api/ FastAPI route handlers and Pydantic schemas
core/ application core (auth, configuration, celery, logging)
database/ async SQLAlchemy engine and CRUD utilities
models/ SQLAlchemy ORM models and domain classes
parsers/ format-specific book parsers
services/ business logic and storage backends
```

3. **Install dependencies:**
```bash
pip install -r requirements.txt
```
## Quick start

4. **Setup environment variables:**
Copy `.env.example` to `.env` and update the values as needed.
### Requirements

5. **Ensure your database is running.**
Run PostgreSQL via Docker Compose:
```bash
docker compose up
```
- Python 3.11+
- Docker (for PostgreSQL, Redis, MinIO)

## Running the application
### Installation

```bash
uvicorn main:app --reload
git clone <repository-url>
cd Shelf
python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt # or pip install -e .[dev]
cp .env.example .env
docker compose up -d
```

The API will be available at http://127.0.0.1:8000.
Interactive API documentation (Swagger UI) will be at http://127.0.0.1:8000/docs and an alternative documentation (ReDoc) is available at http://127.0.0.1:8000/redoc.
### Run the API

## Running Celery worker
```bash
uvicorn main:app --reload
```

To process background tasks (such as book uploads and metadata extraction), you need to run a Celery worker. Make sure your Redis and PostgreSQL services are running and your environment variables are set (see `.env.example`).
Visit `http://127.0.0.1:8000/docs` for interactive API documentation.

### Start Celery worker (locally):
### Run background worker

```bash
celery -A core.celery.celery_app worker --loglevel=info
```

- If you want to specify a queue:
## Using Shelf as a backend

```bash
celery -A core.celery.celery_app worker --loglevel=info -Q default
```
After configuring `CORS_ORIGINS` in `.env`, any web or mobile client can interact with the service:

## Database and migrations
```bash
# Upload a book
curl -X POST "http://127.0.0.1:8000/api/v1/books" -F file=@book.epub

This project uses **PostgreSQL** as the main database via SQLAlchemy (async) and Alembic for migrations.
# List books
curl "http://127.0.0.1:8000/api/v1/books"
```

- Ensure PostgreSQL is running (see Docker Compose instructions above).
- Database connection settings are configured via environment variables in `.env` (see `.env.example`).
- To create or update the database schema, use Alembic migrations:
## Configuration

### Creating a new migration after model changes
Key environment variables:

```bash
alembic revision --autogenerate -m "Describe your change"
```
- `POSTGRES_*` – PostgreSQL connection settings.
- `MINIO_*` – object storage configuration.
- `CELERY_*` – Celery broker and backend URLs.
- `CORS_ORIGINS` – comma-separated list of allowed origins for CORS (e.g. `"http://localhost:3000,https://example.com"`; use `"*"` to allow all).

### Applying migrations to the database
## Development and testing

```bash
alembic upgrade head
ruff .
black --check .
mypy .
pytest
```

- Alembic configuration is in `alembic.ini` and migration scripts are in the `alembic/versions/` directory.

## TODO
- Support more file formats (e.g. CBZ, CBR).
- Allow full-text search (PostgreSQL or Elasticsearch).
- Allow creating text highlights and notes.
- Update metadata from external sources (e.g. Google Books, Open Library).
- Reading progress tracking and bookmarks.
- Recommendations based on reading history.
- Search by metadata and fuzzy search.
- Webhooks and event system for real-time updates.
- Statistics tracking (e.g. reading time, books/pages read).
- Bulk uploading and processing.
- Conversion between formats (e.g. PDF to EPUB).
- Bulk file and metadata export.
- Audit logs.
## Roadmap

- Support more file formats (e.g. CBZ, CBR)
- Allow full-text search (PostgreSQL or Elasticsearch)
- Allow creating text highlights and notes
- Update metadata from external sources (e.g. Google Books, Open Library)
- Reading progress tracking and bookmarks
- Recommendations based on reading history
- Search by metadata and fuzzy search
- Webhooks and event system for real-time updates
- Statistics tracking (e.g. reading time, books/pages read)
- Bulk uploading and processing
- Conversion between formats (e.g. PDF to EPUB)
- Bulk file and metadata export
- Audit logs
15 changes: 8 additions & 7 deletions api/v1/routes/shelves.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
from core.auth import get_current_user
from models.user import User
from services.book_service import BookService, get_book_service
from services.shelf_service import ShelfService, get_shelf_service
from services.shelf_service import get_shelf_service, ShelfService

router = APIRouter()

Expand All @@ -30,14 +30,13 @@ async def list_shelves(
user: User = Security(get_current_user),
):
shelves = await shelf_service.list_shelves(user.id)
return [
ShelfRead(id=s.id, name=s.name, book_ids=[b.id for b in s.books])
for s in shelves
]
return [ShelfRead(id=s.id, name=s.name, book_ids=[b.id for b in s.books]) for s in shelves]


@router.get(
"/{shelf_id}", response_model=ShelfRead, summary="Retrieve a shelf",
"/{shelf_id}",
response_model=ShelfRead,
summary="Retrieve a shelf",
)
async def get_shelf(
shelf_id: str,
Expand Down Expand Up @@ -76,7 +75,9 @@ async def add_book_to_shelf(


@router.delete(
"/{shelf_id}/books/{book_id}", status_code=204, summary="Remove a book from a shelf",
"/{shelf_id}/books/{book_id}",
status_code=204,
summary="Remove a book from a shelf",
)
async def remove_book_from_shelf(
shelf_id: str,
Expand Down
8 changes: 4 additions & 4 deletions core/auth.py
Original file line number Diff line number Diff line change
@@ -1,10 +1,9 @@
from datetime import UTC, datetime, timedelta
from datetime import datetime, timedelta, UTC
import hashlib

from fastapi import Depends, HTTPException, status
from fastapi.security import APIKeyHeader, OAuth2PasswordBearer
from jose import JWTError, jwt

from jose import jwt, JWTError
from passlib.context import CryptContext
from sqlalchemy.ext.asyncio import AsyncSession
from sqlalchemy.future import select
Expand All @@ -27,7 +26,8 @@ class SecretKeyNotSetError(ValueError):

password_context = CryptContext(schemes=["bcrypt"], deprecated="auto")
oauth2_scheme = OAuth2PasswordBearer(
tokenUrl="/api/v1/auth/token", auto_error=False,
tokenUrl="/api/v1/auth/token",
auto_error=False,
)
api_key_header = APIKeyHeader(name="X-API-Key", auto_error=False)

Expand Down
3 changes: 3 additions & 0 deletions core/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,9 @@ class Settings(BaseSettings):
SERVER_PORT: int = int(os.getenv("SERVER_PORT", 8000))
SERVER_PROTOCOL: str = os.getenv("SERVER_PROTOCOL", "http")

# CORS.
CORS_ORIGINS: list[str] = os.getenv("CORS_ORIGINS", "*").split(",")

# Database.
POSTGRES_USER: str = os.getenv("POSTGRES_USER", "postgres")
POSTGRES_PASSWORD: str = os.getenv("POSTGRES_PASSWORD", "postgres")
Expand Down
5 changes: 1 addition & 4 deletions database/book_crud.py
Original file line number Diff line number Diff line change
Expand Up @@ -55,10 +55,7 @@ async def get_all_books(
query = query.where(Book.tags.any(tag))

sort_column = getattr(Book, sort_by, Book.title)
if sort_order.lower() == "desc":
sort_column = sort_column.desc()
else:
sort_column = sort_column.asc()
sort_column = sort_column.desc() if sort_order.lower() == "desc" else sort_column.asc()

result = await db.execute(query.order_by(sort_column).offset(skip).limit(limit))
books = result.scalars().all()
Expand Down
8 changes: 4 additions & 4 deletions database/shelf_crud.py
Original file line number Diff line number Diff line change
Expand Up @@ -23,12 +23,12 @@ async def get_shelves(db: AsyncSession, user_id: str) -> list[Shelf]:


async def get_shelf(
db: AsyncSession, shelf_id: str, user_id: str,
db: AsyncSession,
shelf_id: str,
user_id: str,
) -> Shelf | None:
result = await db.execute(
select(Shelf)
.where(Shelf.id == shelf_id, Shelf.user_id == user_id)
.options(selectinload(Shelf.books)),
select(Shelf).where(Shelf.id == shelf_id, Shelf.user_id == user_id).options(selectinload(Shelf.books)),
)
return result.scalars().first()

Expand Down
9 changes: 9 additions & 0 deletions main.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@

from dotenv import load_dotenv
from fastapi import FastAPI
from fastapi.middleware.cors import CORSMiddleware

from api.v1.routes import auth as auth_v1_router
from api.v1.routes import books as books_v1_router
Expand All @@ -25,6 +26,14 @@ async def lifespan(_app: FastAPI):
lifespan=lifespan,
)

app.add_middleware(
CORSMiddleware,
allow_origins=[origin.strip() for origin in settings.CORS_ORIGINS],
allow_credentials=True,
allow_methods=["*"],
allow_headers=["*"],
)

app.include_router(
books_v1_router.router,
prefix="/api/v1/books",
Expand Down
6 changes: 4 additions & 2 deletions models/book.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
from __future__ import annotations

from typing import TYPE_CHECKING, Any
from typing import Any, TYPE_CHECKING

from sqlalchemy import DateTime, Float, ForeignKey, Integer, String, Text
from sqlalchemy.dialects.postgresql import ARRAY, JSON
Expand Down Expand Up @@ -81,5 +81,7 @@ class Book(Base):
)

shelves: Mapped[list[Shelf]] = relationship(
"Shelf", secondary="shelf_books", back_populates="books",
"Shelf",
secondary="shelf_books",
back_populates="books",
)
10 changes: 7 additions & 3 deletions models/shelf.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
from __future__ import annotations

from typing import TYPE_CHECKING, Any
from typing import Any, TYPE_CHECKING

from sqlalchemy import Column, DateTime, ForeignKey, String, Table
from sqlalchemy.orm import Mapped, mapped_column, relationship
Expand All @@ -27,9 +27,13 @@ class Shelf(Base):
created_at: Mapped[Any] = mapped_column(DateTime, server_default=func.now())
updated_at: Mapped[Any] = mapped_column(DateTime, onupdate=func.now(), nullable=True)
user_id: Mapped[str] = mapped_column(
String, ForeignKey("users.id", ondelete="CASCADE"), nullable=False,
String,
ForeignKey("users.id", ondelete="CASCADE"),
nullable=False,
)

books: Mapped[list[Book]] = relationship(
"Book", secondary="shelf_books", back_populates="shelves",
"Book",
secondary="shelf_books",
back_populates="shelves",
)
5 changes: 4 additions & 1 deletion models/user.py
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,10 @@ class User(Base):
)

api_key_hash: Mapped[str | None] = mapped_column(
String, unique=True, index=True, nullable=True,
String,
unique=True,
index=True,
nullable=True,
)

preferences: Mapped[dict[str, Any]] = mapped_column(JSON, default=dict)