feat: add Deezer as the default download source
- V9 adds source/external_id to songs, moods, recommendations and jobs - Deezer public catalog for search, related album covers and metadata - streamrip downloader configured with DEEZER_ARL, 128/320 MP3 - per-mood source selector; switching clears and refills recommendations - keep YouTube Music as a full second catalog and downloader - media layout <source>-<externalId>, legacy YouTube folders still read - Docker/streamrip install, chart ARL secret, docs and dual-source smoke test
This commit is contained in:
1 parent
81a47f4cd2
commit
141698282d
43 files changed
+2072
-605
No files matched your search
@@ -1,11 +1,12 @@
|
||||
# AudioManager
|
||||
|
||||
A self-hosted, mood-first music server. You create private moods, add tracks
|
||||
from YouTube Music search or a pasted URL, and the server downloads them into a
|
||||
shared media store. Recommendations are discovered and downloaded automatically
|
||||
so a mood keeps playing without a mounted music library.
|
||||
from Deezer or YouTube Music, and the server downloads them into a shared media
|
||||
store. Recommendations are discovered and downloaded automatically so a mood
|
||||
keeps playing without a mounted music library.
|
||||
|
||||
- Server: Kotlin, Ktor, SQLite (Exposed + Flyway), `yt-dlp`, `ffmpeg`.
|
||||
- Server: Kotlin, Ktor, SQLite (Exposed + Flyway), `yt-dlp`, `streamrip`,
|
||||
`ffmpeg`.
|
||||
- Client: Compose Multiplatform (Android, desktop, Wasm).
|
||||
- No uploads, no mounted library, no likes, no podcasts.
|
||||
|
||||
@@ -25,7 +26,10 @@ so a mood keeps playing without a mounted music library.
|
||||
## How it works
|
||||
|
||||
- **Moods are private.** A user sees only their own moods. Songs are shared
|
||||
server-wide and deduplicated by YouTube video id.
|
||||
server-wide and deduplicated by `(source, externalId)`.
|
||||
- **Two catalogs.** Every mood has a download source: **Deezer** (default) or
|
||||
**YouTube Music**. Search, recommendations and downloads follow that source;
|
||||
permanent songs from either source can live in the same mood.
|
||||
- **Songs are the collection.** A mood needs at least one song before it can
|
||||
play. Removing a song only detaches it; the file is deleted when no mood,
|
||||
recommendation slot or open playback session references it.
|
||||
@@ -40,12 +44,14 @@ so a mood keeps playing without a mounted music library.
|
||||
other one. Small pools replay least-recently-played tracks.
|
||||
- **Downloads are automatic.** Recommendation downloads run in the background
|
||||
up to the configured count even if the mood is never played. Manual adds have
|
||||
higher priority.
|
||||
higher priority. Deezer needs an ARL cookie; YouTube Music needs no account.
|
||||
|
||||
## Quick start
|
||||
|
||||
Requirements: JDK 21, `ffmpeg`, `yt-dlp`, and (for the smoke test) `sqlite3`,
|
||||
`curl`, `python3`. `nix-shell` provides everything.
|
||||
Requirements: JDK 21, `ffmpeg`, `yt-dlp`, and `streamrip` (`pip install
|
||||
streamrip==2.1.0`, which provides `rip`) for Deezer. The smoke test also needs
|
||||
`sqlite3`, `curl` and `python3`. `nix-shell` provides everything except
|
||||
`streamrip`.
|
||||
|
||||
```bash
|
||||
# 1. Build and create local config
|
||||
@@ -61,9 +67,11 @@ Requirements: JDK 21, `ffmpeg`, `yt-dlp`, and (for the smoke test) `sqlite3`,
|
||||
The server creates `./data/media` (the media store), `./data/downloads`
|
||||
(download temp files) and `./data/musicmanager.db`.
|
||||
|
||||
Downloads require `yt-dlp` to be able to reach YouTube Music. If YouTube applies
|
||||
bot checks, search and downloads fail with `502 search_failed` / `failed`
|
||||
recommendations; retry later or update `yt-dlp`.
|
||||
Deezer downloads need `DEEZER_ARL`; YouTube Music downloads need `yt-dlp` to be
|
||||
able to reach YouTube. If YouTube applies bot checks, search and downloads fail
|
||||
with `502 search_failed` / `failed` recommendations; retry later or update
|
||||
`yt-dlp`. A Deezer Free ARL is limited to 128 kbps; 320 kbps needs Deezer
|
||||
Premium (`DEEZER_QUALITY=1`).
|
||||
|
||||
## Configuration
|
||||
|
||||
@@ -77,7 +85,12 @@ Everything is environment variables:
|
||||
| `DOWNLOAD_DIR` | `<DATA_DIR>/downloads` | yt-dlp temp/work directory |
|
||||
| `USERS_FILE` | `./config/users.yaml` | User accounts |
|
||||
| `FFMPEG_PATH` | `ffmpeg` | Transcoder binary |
|
||||
| `YTDLP_PATH` | `yt-dlp` | Downloader / search binary |
|
||||
| `YTDLP_PATH` | `yt-dlp` | YouTube Music downloader / search binary |
|
||||
| `STREAMRIP_PATH` | `rip` | Deezer downloader (`streamrip`) |
|
||||
| `DEFAULT_SOURCE` | `deezer` | Source for new moods: `deezer` or `youtube` |
|
||||
| `DEEZER_ARL` | empty | Deezer account cookie used for downloads |
|
||||
| `DEEZER_QUALITY` | `0` | `0` = 128 kbps MP3, `1` = 320 kbps MP3 |
|
||||
| `DEEZER_API_BASE` | `https://api.deezer.com` | Public Deezer API base |
|
||||
|
||||
### `users.yaml`
|
||||
|
||||
@@ -112,37 +125,41 @@ Media URLs may use `?token=` because HTML audio and images cannot send headers.
|
||||
| Method | Path | Purpose |
|
||||
|---|---|---|
|
||||
| `GET` | `/moods` | List the user's moods |
|
||||
| `POST` | `/moods` | Create `{name,description,query,genres,collectionPercent,recommendationCount,refreshHours}` |
|
||||
| `POST` | `/moods` | Create `{name,description,query,genres,source,collectionPercent,recommendationCount,refreshHours}` |
|
||||
| `GET` | `/moods/{id}` | Mood detail: songs and active recommendations |
|
||||
| `PATCH` | `/moods/{id}` | Update options |
|
||||
| `DELETE` | `/moods/{id}` | Delete mood, detach songs and run GC |
|
||||
| `GET` | `/moods/{id}/preview` | Unpersisted radio preview |
|
||||
| `POST` | `/moods/{id}/songs` | Add `{songId}`, `{videoId}` or `{url}` |
|
||||
| `POST` | `/moods/{id}/songs` | Add `{songId}`, `{source,externalId}` or `{url}` |
|
||||
| `DELETE` | `/moods/{id}/songs/{songId}` | Detach a song |
|
||||
| `GET` | `/moods/{id}/recommendations` | Active recommendation slots |
|
||||
| `POST` | `/moods/{id}/recommendations/refresh` | Force discovery + top-up |
|
||||
| `POST` | `/moods/{id}/recommendations/{videoId}/add` | Add to the collection |
|
||||
| `POST` | `/moods/{id}/recommendations/{videoId}/reject` | Hide permanently |
|
||||
| `POST` | `/moods/{id}/recommendations/{videoId}/retry` | Retry a failed download |
|
||||
| `POST` | `/moods/{id}/recommendations/{externalId}/add` | Add to the collection |
|
||||
| `POST` | `/moods/{id}/recommendations/{externalId}/reject` | Hide permanently |
|
||||
| `POST` | `/moods/{id}/recommendations/{externalId}/retry` | Retry a failed download |
|
||||
|
||||
Adding an already downloaded song attaches instantly (`200`). A new video is
|
||||
Moods accept `source` on create/update; switching it clears the active
|
||||
recommendation pool and refills it from the new catalog.
|
||||
|
||||
Adding an already downloaded song attaches instantly (`200`). A new track is
|
||||
queued for download (`202`, `downloading: true`) and is attached when the job
|
||||
finishes.
|
||||
finishes. `/moods/{id}/songs` accepts `{"songId":...}`, `{"source":"deezer",
|
||||
"externalId":"..."}`, or `{"url":"..."}` (Deezer or YouTube).
|
||||
|
||||
### Search
|
||||
|
||||
| Method | Path | Purpose |
|
||||
|---|---|---|
|
||||
| `GET` | `/search?q=...&limit=...` | YouTube Music search |
|
||||
| `GET` | `/search?q=...&limit=...&source=...&moodId=...` | Deezer or YouTube Music search |
|
||||
|
||||
Results longer than 10 minutes (or shorter than 30 seconds) are hidden. Each
|
||||
result reports `downloaded`, `songId` and `inMood` when applicable.
|
||||
result reports `source`, `externalId`, `downloaded`, `songId` and `inMood`.
|
||||
|
||||
Search uses the YouTube Music catalog (the "Songs" filter), so results are
|
||||
canonical tracks with an artist, album and duration instead of arbitrary
|
||||
uploads. yt-dlp has to resolve every hit to read those fields, which takes
|
||||
roughly 10–15 seconds for ten results. If the Music search fails, the server
|
||||
falls back to the regular YouTube search so the feature keeps working.
|
||||
`source` defaults to the target mood's source, then to `DEFAULT_SOURCE`.
|
||||
Deezer search uses the public API and returns canonical tracks with a square
|
||||
album cover. YouTube Music search uses the "Songs" filter and resolves every
|
||||
hit, which takes roughly 10–15 seconds for ten results; if Music search fails,
|
||||
it falls back to plain YouTube search.
|
||||
|
||||
### Playback
|
||||
|
||||
@@ -169,10 +186,11 @@ falls back to the regular YouTube search so the feature keeps working.
|
||||
```
|
||||
Ktor routes
|
||||
└── MoodRepository ── moods, mood_songs, recommendation lifecycle
|
||||
└── SongRepository ── shared song store keyed by video id
|
||||
└── SongRepository ── shared song store keyed by (source, externalId)
|
||||
└── RadioService ── mixes collection + ready recommendations
|
||||
└── DownloadService── single worker: yt-dlp → media store → index → attach
|
||||
└── DiscoveryService── yt-dlp YouTube Music search → hidden `new` candidates
|
||||
└── MusicCatalogs ── Deezer public API + yt-dlp YouTube Music search
|
||||
└── DownloadService── single worker: yt-dlp/streamrip → media store → index
|
||||
└── DiscoveryService── mood source search → hidden `new` candidates
|
||||
└── TopUpService ── keeps active slots at recommendationCount
|
||||
└── MediaScanner ── rebuilds the index from the media root
|
||||
└── MediaGcService ── refcounted file deletion
|
||||
@@ -186,19 +204,18 @@ media GC (hourly). A scan runs at startup.
|
||||
Layout:
|
||||
|
||||
```
|
||||
<MEDIA_DIR>/<videoId>/<sanitized-title>.mp3
|
||||
<MEDIA_DIR>/<source>-<externalId>/<sanitized-title>.mp3
|
||||
```
|
||||
|
||||
The first path segment is the YouTube video id, so the startup scan never
|
||||
depends on tags. One row and one file exist per video, no matter how many moods
|
||||
or users reference it. Covers are cached under `<DATA_DIR>/covers` keyed by song
|
||||
Examples: `deezer-67238732/Instant Crush.mp3`, `youtube-dQw4w9WgXcQ/...`.
|
||||
Older YouTube downloads in `<MEDIA_DIR>/<videoId>/` are still read and treated
|
||||
as YouTube. One row and one file exist per track, no matter how many moods or
|
||||
users reference it. Covers are cached under `<DATA_DIR>/covers` keyed by song
|
||||
id.
|
||||
|
||||
Downloads are requested from the YouTube Music player
|
||||
(`music.youtube.com/watch?v=<id>`), which makes yt-dlp embed the song's artist,
|
||||
album and release year into the MP3. Videos that are not on the Music host fall
|
||||
back to the regular YouTube URL. Files are MP3 only
|
||||
(`-x --audio-format mp3 --audio-quality 0`).
|
||||
Deezer is downloaded with `streamrip` (MP3 128/320) and YouTube Music with
|
||||
`yt-dlp` (`-x --audio-format mp3 --audio-quality 0`). Both embed the artist,
|
||||
album and cover art, so the indexer can derive the metadata from the file.
|
||||
|
||||
A file is deleted only when it has no references from `mood_songs`, an active
|
||||
recommendation (`downloading`, `ready`, `added`, `failed`), or an open playback
|
||||
@@ -214,6 +231,9 @@ SQLite in WAL mode. Migrations are Flyway scripts under
|
||||
(folders, artists, albums, songs, moods, seeds, sessions, longform progress,
|
||||
downloads) and creates the mood-first schema. `users` and `api_tokens` are
|
||||
kept.
|
||||
- **V9 adds sources.** It renames `video_id` to `external_id` and adds a
|
||||
`source` column to songs, moods, recommendations and download jobs. Existing
|
||||
rows become `youtube`.
|
||||
- There is no import from an old media directory. Back up externally and start
|
||||
with an empty `MEDIA_DIR`.
|
||||
|
||||
@@ -223,7 +243,7 @@ The Helm chart is in `charts/audiomanager`. It renders one PVC at `/data`:
|
||||
|
||||
```
|
||||
/data/musicmanager.db
|
||||
/data/media/<videoId>/<title>.mp3
|
||||
/data/media/<source>-<externalId>/<title>.mp3
|
||||
/data/covers/
|
||||
/data/downloads/
|
||||
```
|
||||
@@ -234,9 +254,15 @@ PVC writable by the non-root container.
|
||||
```bash
|
||||
kubectl create secret generic audiomanager-users \
|
||||
--from-file=users.yaml=config/users.yaml
|
||||
# Only needed for Deezer. Leave it out to use YouTube Music only.
|
||||
kubectl create secret generic audiomanager-deezer \
|
||||
--from-literal=arl='<deezer-arl>'
|
||||
helm upgrade --install audiomanager charts/audiomanager \
|
||||
--namespace audiomanager --create-namespace \
|
||||
--set users.existingSecret=audiomanager-users \
|
||||
--set deezer.existingSecret=audiomanager-deezer \
|
||||
--set deezer.quality=0 \
|
||||
--set defaultSource=deezer \
|
||||
--set image.tag=<sha>
|
||||
```
|
||||
|
||||
@@ -266,6 +292,8 @@ Compose Multiplatform client: `/home/taptap/AudioManagerClient`.
|
||||
|
||||
- Android, desktop and Wasm from `commonMain`.
|
||||
- Screens: Moods, Mood Detail, Search, Downloads, Settings.
|
||||
- Every mood has a **Download source** selector (Deezer or YouTube Music);
|
||||
search, recommendations and downloads follow it.
|
||||
- Android background playback and media notifications are kept.
|
||||
|
||||
## Testing
|
||||
@@ -276,6 +304,7 @@ Compose Multiplatform client: `/home/taptap/AudioManagerClient`.
|
||||
scripts/check-chart.sh # lint + render the Helm chart
|
||||
```
|
||||
|
||||
The smoke test seeds a tiny media store, puts a deterministic fake `yt-dlp` on
|
||||
`PATH`, and exercises auth, search filtering, adding songs, URL resolve,
|
||||
downloads, recommendations, radio sessions, streaming, covers and logout.
|
||||
The smoke test seeds a tiny media store, puts deterministic fake `yt-dlp` and
|
||||
`rip` binaries plus a fake Deezer API on localhost, and exercises both sources:
|
||||
auth, search filtering, adding songs, URL resolve, downloads, recommendations,
|
||||
radio sessions, streaming, covers and logout.
|
||||
Reference in new issue
Block a user