Sharder

Sharder is a distributed file storage platform that splits your files into encrypted chunks and spreads them across multiple storage nodes—so your data is safe, private, and always available.

🛠️ Installation

Quick Start (with Docker)

1. Clone the repo:

   git clone https://github.com/hikariatama/sharder

2. Generate a Docker Compose file:

   python3 docker-compose-generator.py \
       --chunks-per-file 2 \
       --replicas 3 \
       --public-port 3333 \
       --dev-shards 7

3. Start everything up:

   sudo docker compose up -d --build

Custom Setup

You can tweak your deployment with these options:

• --chunks-per-file: How many chunks to split each file into (default: 3)
• --replicas: How many copies of each chunk to keep (default: 2)
• --public-port: Which port to use for the app (default: 80)
• --dev-shards: How many local shards to spin up for testing (default: 0)

---

📦 Want to Run a Shard?

1. Build the shard binary:

   mkdir /tmp/shard
   cd /tmp/shard
   wget https://sharder.dgazizullin.dev/bin/shard.tar.gz
   tar xvf shard.tar.gz
   sudo apt install build-essential libcurl4-openssl-dev
   make
   cd bin

2. Get your connection URL:
   Open the "Management" menu (just type "chimera" anywhere in the web UI).

3. Start your shard:
   Use the command provided in the UI. The shard will install itself as a systemd service and move to /opt/shard.
   Tip: The first run should be as root, or use the --dry flag for a test run.

---

🚀 Features

• End-to-End Encryption: Files are encrypted on your device with ChaCha20 before being split and uploaded.
• Distributed Storage: Files are broken into chunks and distributed across independent shards.
• Redundancy: You choose how many copies of each chunk to keep, so your data survives outages.
• REST API: Simple endpoints for uploading, downloading, and managing files.
• Modern Web UI: Built with Next.js and Tailwind CSS for a smooth experience.
• Live Health Monitoring: See the status of all shards in real time on the dashboard.

🏗️ Architecture

Sharder is made up of three main parts:

• Frontend: Next.js app for the user interface
• Server: FastAPI backend that handles file operations
• Shards: C++ services that store encrypted file chunks

🧩 Tech Stack

• Frontend: Next.js, TypeScript, Tailwind CSS, Framer Motion
• Backend: Python, FastAPI, SQLAlchemy, PostgreSQL
• Shard: C++17, POSIX sockets, libcurl (libcurl4-openssl-dev required)
• Infrastructure: Docker, Nginx

📝 License

MIT License. See the LICENSE file for details.