diff --git a/.github/workflows/build.yaml b/.github/workflows/build.yaml new file mode 100644 index 0000000..fcbcec3 --- /dev/null +++ b/.github/workflows/build.yaml @@ -0,0 +1,55 @@ +name: Build Client and Server + +on: + push: + branches: + - master + pull_request: + branches: + - master + +jobs: + build: + runs-on: ubuntu-latest + strategy: + matrix: + node-version: [18.x] + steps: + - name: Checkout repository + uses: actions/checkout@v3 + + - name: Set up Node.js ${{ matrix.node-version }} + uses: actions/setup-node@v3 + with: + node-version: ${{ matrix.node-version }} + cache: "npm" + cache-dependency-path: | + client/package-lock.json + server/package-lock.json + + - name: Install client dependencies + run: | + cd client + npm ci + - name: Build client + run: | + cd client + npm run build + + - name: Log in to GitHub Container Registry + if: github.event_name == 'push' && github.ref == 'refs/heads/master' + uses: docker/login-action@v3 + with: + registry: ghcr.io + username: ${{ github.actor }} + password: ${{ secrets.GITHUB_TOKEN }} + + - name: Build Docker image + if: github.event_name == 'push' && github.ref == 'refs/heads/master' + run: | + docker build -t ghcr.io/${{ github.repository }}/subarr:latest -f docker/Dockerfile . + + - name: Push Docker image + if: github.event_name == 'push' && github.ref == 'refs/heads/master' + run: | + docker push ghcr.io/${{ github.repository }}/subarr:latest diff --git a/.gitignore b/.gitignore index d65c795..d5d7a33 100644 --- a/.gitignore +++ b/.gitignore @@ -49,3 +49,5 @@ Thumbs.db **/*.crt **/*.key **/secrets.json + +data/* \ No newline at end of file diff --git a/README.md b/README.md index c4526b9..8b97b1e 100644 --- a/README.md +++ b/README.md @@ -1,19 +1,49 @@ -# Subarr +
-*After [a lot of feedback](https://reddit.com/r/selfhosted/comments/1myldh3/i_built_youtubarr_the_sonarr_for_youtube/nacw3am/), I've decided to rename this project from "YouTubarr" to "Subarr". The name "Subarr" also helps define the project a little clearer in how it's based on RSS subscriptions (intended to be a "subscribe to playlists/channels & take action on new uploads") rather than a full PVR & media management system. If you're looking for a more true "Sonarr for YouTube", I recommend checking out one of the solutions below.* +# 🎎 Subarr -Subarr is a **lightweight** YouTube channel/playlist/subscription follower that will take action on new uploads (actions can be a webhook - like Discord - or a process - like downloading through yt-dlp). "Lightweight" means it needs few resources and can run on something like a raspberry pi. +**A lightweight YouTube channel/playlist subscription manager** -image +[![Docker](https://img.shields.io/badge/Docker-2496ED?style=for-the-badge&logo=docker&logoColor=white)](https://github.com/dnlrsr/subarr/pkgs/container/subarr%2Fsubarr) +[![Node.js](https://img.shields.io/badge/Node.js-339933?style=for-the-badge&logo=nodedotjs&logoColor=white)](https://nodejs.org/) +[![License](https://img.shields.io/github/license/dnlrsr/subarr?style=for-the-badge)](LICENSE) +[![Stars](https://img.shields.io/github/stars/dnlrsr/subarr?style=for-the-badge)](https://github.com/dnlrsr/subarr/stargazers) +
-### Background +--- -This is an attempt to create a Sonarr-like application for YouTube videos. A lot of inspiration has been taken from Sonarr (mostly the UI), but this has been written entirely from scratch. +> ### 🔗 **Fork Notice** +> +> This is a fork of [derekantrican/subarr](https://github.com/derekantrican/subarr). I will continue to implement features when they fit my needs. -_Why not use one of the existing solutions?_ +--- -Here are all the similar things I could find and how this is different: +## 📖 Overview + +*After [a lot of feedback](https://reddit.com/r/selfhosted/comments/1myldh3/i_built_youtubarr_the_sonarr_for_youtube/nacw3am/), the original project was renamed from "YouTubarr" to "Subarr". The name "Subarr" helps define the project clearer - it's based on RSS subscriptions (intended to be a "subscribe to playlists/channels & take action on new uploads") rather than a full PVR & media management system.* + +Subarr is a **ðŸŠķ lightweight** YouTube channel/playlist/subscription follower that will take action on new uploads (actions can be a webhook - like Discord). + +**"Lightweight"** means it needs few resources and can run on something like a Raspberry Pi. + +
+ +![Subarr Interface](https://github.com/user-attachments/assets/dd9b42d8-08e9-4d9a-a175-acf7219d059a) + +*Clean, Sonarr-inspired interface for managing your YouTube subscriptions* + +
+ +--- + +## ðŸŽŊ Background + +This is an attempt to create a **Sonarr-like application for YouTube videos**. A lot of inspiration has been taken from Sonarr (mostly the UI), but this has been written entirely from scratch. + +### ðŸĪ” Why not use one of the existing solutions? + +Here are all the similar projects and how Subarr is different: | Name | Active? | Stars | Indexer | Comment | |----------------------|---------|-------|---------|---------| | [Tube Archivist](https://github.com/tubearchivist/tubearchivist) | ✅ | ![](https://img.shields.io/github/stars/tubearchivist/tubearchivist?label=&style=flat-square&color=white) | yt-dlp | Based on yt-dlp | @@ -30,69 +60,171 @@ Here are all the similar things I could find and how this is different: | [Tubarr (Capstone Project)](https://vc.bridgew.edu/cgi/viewcontent.cgi?article=1691&context=honors_proj) | ❌ | N/A | yt-dlp | Private (a student's capstone project) | -As you can see, there's a few other solutions, but most are based on [yt-dlp](https://github.com/yt-dlp/yt-dlp) for getting playlist information. This works fine if you want to index an _entire playlist_ (which a user may, very well, want to do) but it can require a LOT of polling activity (particularly for large channels). _Tube Archivist above even calls out 2-4GB of memory dedicated to its functionality._ **This project is lightweight and can easily run on a raspberry pi or similar.** Additionally, none of the above services can automatically keep your actual subscriptions on YouTube in sync with the app. +As you can see, there are several other solutions, but most are based on [yt-dlp](https://github.com/yt-dlp/yt-dlp) for getting playlist information. This works fine if you want to index an _entire playlist_ (which users may want to do) but it can require **a LOT of polling activity** (particularly for large channels). + +> 📊 _Tube Archivist above even calls out 2-4GB of memory dedicated to its functionality._ -Sonarr is based on RSS feeds - explicitly designed for this purpose of getting new updates from subscription-like sources. This is much lighter in processing requirements. I've also tried to make this UI as similar as possible to the other *arr apps for familiarity. +**🚀 This project is lightweight and can easily run on a Raspberry Pi or similar.** Additionally, none of the above services can automatically keep your actual subscriptions on YouTube in sync with the app. -_What are the limitations of the RSS feed approach?_ +Sonarr is based on **RSS feeds** - explicitly designed for this purpose of getting new updates from subscription-like sources. This is much lighter in processing requirements. The UI has been made as similar as possible to the other *arr apps for familiarity. -YouTube already provides RSS feeds for playlists (eg https://www.youtube.com/feeds/videos.xml?playlist_id=PLopY4n17t8RDoFQPcjBKWDEblGH1sOH2h). However, they can be severly limited: +### ⚠ïļ What are the limitations of the RSS feed approach? -- Feeds seem to be limited to only the last 15 items +YouTube already provides RSS feeds for playlists (e.g., `https://www.youtube.com/feeds/videos.xml?playlist_id=PLopY4n17t8RDoFQPcjBKWDEblGH1sOH2h`). However, they can be severely limited: + +- **📊 Limited to 15 items**: Feeds seem to be limited to only the last 15 items - _However, Subarr will list more items as they are found because the internal database will be updated_ - - Since the feed is limited to only the last 15 items, if (for some reason) Subarr is down for an extended period of time (or a large amount of videos are published to the playlist in a short period of time), Subarr may miss some videos entirely. -- YouTube RSS feed items are always in "playlist order", meaning a new video added outside of the first 15 items will not be seen as an update to the RSS feed - - This means that, currently, RSS feeds _**will not work**_ in the following situations: (issue logged with YouTube: https://issuetracker.google.com/issues/429563457) - - YouTube playlists greater than 15 items where items are added outside the "top 15" (regular playlists can be in _any order_ - as determined by the playlist owner - which means that "newly added items" aren't necessarily at the top. Some creators' playlists are ordered oldest -> newest) - - \>=15 items added to a playlist quickly (eg if a playlist has 1 item, then when it updates 15 minutes later it has 20 items, 4 of those items will be missed because only the top 15 will be in the RSS feed) -- RSS feeds may be somewhat slow to update (updates are approximately every 15 minutes). _15 minutes is pretty fast, but other methods (like the direct YouTube API - or perhaps even TubeSync's yt-dl method) could check for new videos even faster_ -- _[May not be a big deal]_ RSS feeds will not include "Members Only" items + - If Subarr is down for an extended period or many videos are published quickly, some videos may be missed entirely -However, this works perfectly fine for mine (and maybe other people's) needs. +- **🔄 Playlist order issues**: YouTube RSS feed items are always in "playlist order", meaning a new video added outside of the first 15 items will not be seen as an update + - This means RSS feeds **will not work** in these situations: ([issue logged with YouTube](https://issuetracker.google.com/issues/429563457)) + - YouTube playlists >15 items where items are added outside the "top 15" + - â‰Ĩ15 items added to a playlist quickly +- **⏰ Update delays**: RSS feeds may be somewhat slow to update (~15 minutes) -### Notes +- **ðŸ‘Ĩ Members-only content**: RSS feeds will not include "Members Only" items -⚠ïļ **Subarr currently does not implement any sort of authetication. It is highly recommended that you do not expose your instance to the internet (or, at least, put it behind a form of authentication like nginx or Cloudflare)** ⚠ïļ +**However, this works perfectly fine for our needs and many others' use cases.** -Subarr is NOT intended to do the following: -- Index an entire channel/playlist or get "older" videos. Subarr's RSS approach is specifically for "subscriptions": new video is posted, take some action -- Media management. Once Subarr kicks off the post-processor (like yt-dlp), its job is done. Use Plex/Jellyfin/etc or another one of the linked solutions above if you require more control over your media +--- +## 📋 Important Notes -### Current features +🔐 **Security Notice**: Subarr currently does not implement any sort of authentication. It is **highly recommended** that you do not expose your instance to the internet (or, at least, put it behind a form of authentication like nginx or Cloudflare). -- Add playlists -- Limit playlist items by regular expression -- Exclude shorts -- [ytsubs.app](https://github.com/derekantrican/ytsubs) integration to import user's YouTube subscriptions and keep them in sync -- Post processors (an action to run when a new video is found) - - Webhook: call a webhook (eg Discord, Raindrop.io, etc) - - Process: execute a process (eg yt-dlp to download the video) +### ðŸšŦ What Subarr is NOT intended for: -### Future features +- **📚 Indexing entire channels/playlists** or getting "older" videos. Subarr's RSS approach is specifically for "subscriptions": new video is posted → take some action +- **🎎 Media management**. Once Subarr triggers the post-processor (webhook), its job is done. Use Plex/Jellyfin/etc or another solution above if you require more control over your media -- API key for authenticating calls to the server -- It would be neat to have some sort of web socket or other real-time communication between the server & client that will do things like updating the "last checked" or video list on the UI -- Native "url base" support like sonarr (for reverse proxies or cloudflare tunnels) -- Backup & Restore functionality (_should_ be pretty easy by just giving a copy of the sqlite db?) -- _Index more than 15 items initially? (this would require significant up-front processing power like TubeSync does - so...unlikely)_ +--- +## âœĻ Current Features -### Installation +- 📝 **Add playlists** - Subscribe to YouTube playlists and channels +- ðŸŽŊ **Regex filtering** - Limit playlist items by regular expression +- ðŸšŦ **Exclude shorts** - Filter out YouTube Shorts automatically +- 🔗 **[ytsubs.app](https://github.com/derekantrican/ytsubs) integration** - Import user's YouTube subscriptions and keep them in sync +- ⚡ **Post processors** - Actions to run when a new video is found: + - 🊝 **Webhook**: Call a webhook (e.g., Discord, Raindrop.io, etc.) -Make sure you have Node >= 18 installed, then run the following: +## 🚀 Future Features -``` -git clone https://github.com/derekantrican/subarr.git +- 🔐 **API key authentication** for secure server calls +- 🔄 **Real-time updates** via WebSocket communication between server & client +- 🌐 **Native URL base support** like Sonarr (for reverse proxies or Cloudflare tunnels) +- ðŸ’ū **Backup & Restore functionality** (should be easy with SQLite database) +- 📊 _Index more than 15 items initially?_ (would require significant processing power like TubeSync - unlikely) + +--- + +## 🛠ïļ Installation + +### Prerequisites +- **Node.js** >= 18 + +### Quick Start + +```bash +git clone https://github.com/dnlrsr/subarr.git cd subarr npm install -# optionally create server/.env with PORT=5000 or whatever +# Optionally create server/.env with PORT=5000 or whatever npm run start-server # On Windows, use 'npm run start-server-win' ``` -And if you'd like to set it to run at startup, put that last command into a .service file (Linux) or your Startup folder (Windows). +### 🔄 Run at Startup +To set it to run at startup, put the last command into: +- **Linux**: A `.service` file +- **Windows**: Your Startup folder + +--- + +## ðŸģ Docker Usage +⚠ïļ **Important**: Do not mount NFS storage! The application uses SQLite. + +### 🏗ïļ Build the Docker Image + +```bash +docker build -t subarr -f docker/Dockerfile . +``` + +### 🚀 Run the Container + +```bash +docker run -p 3001:3001 subarr +``` + +### ðŸ’ū Mount Data Directory for Persistent Storage + +```bash +docker run -p 3001:3001 -v /path/to/host/data:/data/db subarr +``` +This stores all database files in `/path/to/host/data/data` on your host. + +### 🔧 Set Environment Variables + +Using the `-e` flag: +```bash +docker run -p 3001:3001 -v /path/to/host/data:/data -e NODE_ENV=production subarr +``` + +Using a custom `.env` file: +```bash +docker run -p 3001:3001 -v /path/to/host/data:/data --env-file /path/to/.env subarr +``` + +### 🐙 Docker Compose Example + +Create a `docker-compose.yml` file in your project directory: + +```yaml +services: + subarr: + image: ghcr.io/dnlrsr/subarr/subarr:latest # TODO: Change after merge + ports: + - "3001:3001" + volumes: + - db-data:/data/db + env_file: + - .env + +volumes: + db-data: + driver: local +``` + +Start with: +```bash +docker compose up --build +``` + +**This will:** +- 🏗ïļ Build and run the Subarr container +- ðŸ’ū Mount the local `./data` folder to `/data` in the container for persistence +- 🔧 Load environment variables from `.env` +- 🌐 Expose port 3001 + +--- + +## ðŸ”Ū Plans for Future Maintenance + +I am currently building this as a **hobby project** for myself and I already have about 10x the amount of hobby projects that I can handle. I'll probably fix some bugs or maintenance issues as they arise, but I don't plan to work on any major features. + +**ðŸĪ Want to contribute?** Please reach out! + +--- + +
+ +## 💖 Support + +If you find Subarr useful, consider: -### Plans for future maintenance +⭐ **Starring this repository** +🐛 **Reporting bugs** +ðŸ’Ą **Suggesting features** +🔧 **Contributing code** -I am currently just building this as a hobby project for myself and I already have about 10x the amount of hobby projects that I can handle. I'll probably fix some bugs or maintenance issues as they arise, but I don't plan to work on any major features. If you'd like to contribute, please reach out! +
diff --git a/docker/Dockerfile b/docker/Dockerfile new file mode 100644 index 0000000..4abace2 --- /dev/null +++ b/docker/Dockerfile @@ -0,0 +1,35 @@ +FROM node:alpine AS builder + +# Install build dependencies for native modules +RUN apk add --no-cache python3 make g++ py3-pip + +WORKDIR /app +COPY server/package.json ./server/ +WORKDIR /app/server +RUN npm install && npm rebuild + +FROM node:alpine AS runtime + +# Install build dependencies for native modules and yt-dlp +RUN apk add --no-cache python3 make g++ + +ENV NODE_ENV=production +WORKDIR /app + +# Copy built node_modules from builder stage +COPY --from=builder /app/server/node_modules ./server/node_modules +COPY server/ ./server/ +COPY client/build ./client/build + +WORKDIR /app/server + +# Rebuild native modules for the runtime environment +RUN npm rebuild + +RUN mkdir -p /data/db && mkdir -p /data/config + +USER 1000:1000 + +EXPOSE 3001 +CMD ["node", "index.js"] + diff --git a/package.json b/package.json index 9602d7c..dd2c008 100644 --- a/package.json +++ b/package.json @@ -5,6 +5,7 @@ "server" ], "scripts": { + "build": "npm --workspace client run build", "start-server": "npm --workspace client run build && NODE_ENV=production node server/index.js", "start-server-win": "npm --workspace client run build && SET NODE_ENV=production && node server/index.js", "update": "git pull && npm install && npm run start-server" diff --git a/server/db.js b/server/db.js index d4323cb..59a2a07 100644 --- a/server/db.js +++ b/server/db.js @@ -3,9 +3,11 @@ const path = require('path'); const Database = require('better-sqlite3'); // Always use subarr.db from the server folder (and support youtubarr.db if it still exists from before the app rename) -const dbPath = fs.existsSync(path.join(__dirname, 'youtubarr.db')) - ? path.join(__dirname, 'youtubarr.db') - : path.join(__dirname, 'subarr.db'); +const dir = path.resolve("/data/db"); + +const dbPath = fs.existsSync(path.join(dir, 'youtubarr.db')) + ? path.join(dir, 'youtubarr.db') + : path.join(dir, 'subarr.db'); const db = new Database(dbPath);