Files
AudioManager/README.md
T
TapTap 141698282d
CI / chart (push) Successful in 3s
CI / test (push) Successful in 2m23s
CI / chart-push (push) Successful in 5s
CI / image (push) Successful in 3m50s
feat: add Deezer as the default download source
- V9 adds source/external_id to songs, moods, recommendations and jobs
- Deezer public catalog for search, related album covers and metadata
- streamrip downloader configured with DEEZER_ARL, 128/320 MP3
- per-mood source selector; switching clears and refills recommendations
- keep YouTube Music as a full second catalog and downloader
- media layout <source>-<externalId>, legacy YouTube folders still read
- Docker/streamrip install, chart ARL secret, docs and dual-source smoke test
2026-11-17 07:32:34 +00:00

311 lines
12 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# AudioManager
A self-hosted, mood-first music server. You create private moods, add tracks
from Deezer or YouTube Music, and the server downloads them into a shared media
store. Recommendations are discovered and downloaded automatically so a mood
keeps playing without a mounted music library.
- Server: Kotlin, Ktor, SQLite (Exposed + Flyway), `yt-dlp`, `streamrip`,
`ffmpeg`.
- Client: Compose Multiplatform (Android, desktop, Wasm).
- No uploads, no mounted library, no likes, no podcasts.
## Contents
- [How it works](#how-it-works)
- [Quick start](#quick-start)
- [Configuration](#configuration)
- [Native API](#native-api)
- [Architecture](#architecture)
- [Media store and GC](#media-store-and-gc)
- [Database](#database)
- [Deployment](#deployment)
- [Client](#client)
- [Testing](#testing)
## How it works
- **Moods are private.** A user sees only their own moods. Songs are shared
server-wide and deduplicated by `(source, externalId)`.
- **Two catalogs.** Every mood has a download source: **Deezer** (default) or
**YouTube Music**. Search, recommendations and downloads follow that source;
permanent songs from either source can live in the same mood.
- **Songs are the collection.** A mood needs at least one song before it can
play. Removing a song only detaches it; the file is deleted when no mood,
recommendation slot or open playback session references it.
- **Recommendations are active slots.** A mood keeps `recommendationCount`
(0–30, default 10) recommendations in `downloading`, `ready` or `failed`
state. Hidden `new` candidates refill the slots automatically.
- **Add / Not interested are permanent.** Adding promotes a recommendation into
the collection. Rejecting is remembered per mood. Both free a slot and the
server refills it.
- **Playback mixes both pools.** Every queue slot rolls `collectionPercent`;
the rest comes from ready recommendations. An empty pool falls back to the
other one. Small pools replay least-recently-played tracks.
- **Downloads are automatic.** Recommendation downloads run in the background
up to the configured count even if the mood is never played. Manual adds have
higher priority. Deezer needs an ARL cookie; YouTube Music needs no account.
## Quick start
Requirements: JDK 21, `ffmpeg`, `yt-dlp`, and `streamrip` (`pip install
streamrip==2.1.0`, which provides `rip`) for Deezer. The smoke test also needs
`sqlite3`, `curl` and `python3`. `nix-shell` provides everything except
`streamrip`.
```bash
# 1. Build and create local config
./gradlew setupLocal
# edit config/users.yaml (a password is required)
# 2. Run
./gradlew run
# health: http://localhost:8080/health
# API: http://localhost:8080/api/v1
```
The server creates `./data/media` (the media store), `./data/downloads`
(download temp files) and `./data/musicmanager.db`.
Deezer downloads need `DEEZER_ARL`; YouTube Music downloads need `yt-dlp` to be
able to reach YouTube. If YouTube applies bot checks, search and downloads fail
with `502 search_failed` / `failed` recommendations; retry later or update
`yt-dlp`. A Deezer Free ARL is limited to 128 kbps; 320 kbps needs Deezer
Premium (`DEEZER_QUALITY=1`).
## Configuration
Everything is environment variables:
| Variable | Default | Meaning |
|---|---|---|
| `PORT` | `8080` | HTTP port |
| `DATA_DIR` | `./data` | Database, cover cache, default media root |
| `MEDIA_DIR` | `<DATA_DIR>/media` | Server-owned song store |
| `DOWNLOAD_DIR` | `<DATA_DIR>/downloads` | yt-dlp temp/work directory |
| `USERS_FILE` | `./config/users.yaml` | User accounts |
| `FFMPEG_PATH` | `ffmpeg` | Transcoder binary |
| `YTDLP_PATH` | `yt-dlp` | YouTube Music downloader / search binary |
| `STREAMRIP_PATH` | `rip` | Deezer downloader (`streamrip`) |
| `DEFAULT_SOURCE` | `deezer` | Source for new moods: `deezer` or `youtube` |
| `DEEZER_ARL` | empty | Deezer account cookie used for downloads |
| `DEEZER_QUALITY` | `0` | `0` = 128 kbps MP3, `1` = 320 kbps MP3 |
| `DEEZER_API_BASE` | `https://api.deezer.com` | Public Deezer API base |
### `users.yaml`
```yaml
users:
- name: admin
password: "change-me"
admin: true
```
Passwords live only in this file (use a Vault-managed Secret in Kubernetes).
`admin` is only needed for the maintenance scan endpoints. Old `folders:` keys
are accepted and ignored.
## Native API
All endpoints are under `/api/v1` and require a bearer token, except login.
Media URLs may use `?token=` because HTML audio and images cannot send headers.
### Authentication
| Method | Path | Purpose |
|---|---|---|
| `POST` | `/auth/login` | `{username,password,deviceName,platform}` → token |
| `POST` | `/auth/logout` | Revoke the current token |
| `GET` | `/me` | Current user |
| `GET` | `/auth/devices` | List device tokens |
| `DELETE` | `/auth/devices/{id}` | Revoke a device |
### Moods
| Method | Path | Purpose |
|---|---|---|
| `GET` | `/moods` | List the user's moods |
| `POST` | `/moods` | Create `{name,description,query,genres,source,collectionPercent,recommendationCount,refreshHours}` |
| `GET` | `/moods/{id}` | Mood detail: songs and active recommendations |
| `PATCH` | `/moods/{id}` | Update options |
| `DELETE` | `/moods/{id}` | Delete mood, detach songs and run GC |
| `GET` | `/moods/{id}/preview` | Unpersisted radio preview |
| `POST` | `/moods/{id}/songs` | Add `{songId}`, `{source,externalId}` or `{url}` |
| `DELETE` | `/moods/{id}/songs/{songId}` | Detach a song |
| `GET` | `/moods/{id}/recommendations` | Active recommendation slots |
| `POST` | `/moods/{id}/recommendations/refresh` | Force discovery + top-up |
| `POST` | `/moods/{id}/recommendations/{externalId}/add` | Add to the collection |
| `POST` | `/moods/{id}/recommendations/{externalId}/reject` | Hide permanently |
| `POST` | `/moods/{id}/recommendations/{externalId}/retry` | Retry a failed download |
Moods accept `source` on create/update; switching it clears the active
recommendation pool and refills it from the new catalog.
Adding an already downloaded song attaches instantly (`200`). A new track is
queued for download (`202`, `downloading: true`) and is attached when the job
finishes. `/moods/{id}/songs` accepts `{"songId":...}`, `{"source":"deezer",
"externalId":"..."}`, or `{"url":"..."}` (Deezer or YouTube).
### Search
| Method | Path | Purpose |
|---|---|---|
| `GET` | `/search?q=...&limit=...&source=...&moodId=...` | Deezer or YouTube Music search |
Results longer than 10 minutes (or shorter than 30 seconds) are hidden. Each
result reports `source`, `externalId`, `downloaded`, `songId` and `inMood`.
`source` defaults to the target mood's source, then to `DEFAULT_SOURCE`.
Deezer search uses the public API and returns canonical tracks with a square
album cover. YouTube Music search uses the "Songs" filter and resolves every
hit, which takes roughly 10–15 seconds for ten results; if Music search fails,
it falls back to plain YouTube search.
### Playback
| Method | Path | Purpose |
|---|---|---|
| `POST` | `/moods/{id}/sessions` | Start a queue (`400 empty_mood` without songs) |
| `GET` | `/sessions/{id}` | Resume a session |
| `POST` | `/sessions/{id}/next` | Fetch more tracks |
| `POST` | `/sessions/{id}/events` | Report `started`, `completed`, `skipped` |
| `POST` | `/sessions/{id}/add-to-mood` | Add the playing track to a mood |
| `DELETE` | `/sessions/{id}` | Close the session |
### Media and maintenance
| Method | Path | Purpose |
|---|---|---|
| `GET` | `/songs/{id}/stream` | Range-aware stream; `maxBitRate`, `timeOffset`, `download` |
| `GET` | `/songs/{id}/cover?size=` | Embedded artwork |
| `GET` | `/scan` | Scan status |
| `POST` | `/scan` | Start a scan (admin only) |
## Architecture
```
Ktor routes
└── MoodRepository ── moods, mood_songs, recommendation lifecycle
└── SongRepository ── shared song store keyed by (source, externalId)
└── RadioService ── mixes collection + ready recommendations
└── MusicCatalogs ── Deezer public API + yt-dlp YouTube Music search
└── DownloadService── single worker: yt-dlp/streamrip → media store → index
└── DiscoveryService── mood source search → hidden `new` candidates
└── TopUpService ── keeps active slots at recommendationCount
└── MediaScanner ── rebuilds the index from the media root
└── MediaGcService ── refcounted file deletion
```
Background loops: download worker (continuous), recommendation top-up (hourly),
media GC (hourly). A scan runs at startup.
## Media store and GC
Layout:
```
<MEDIA_DIR>/<source>-<externalId>/<sanitized-title>.mp3
```
Examples: `deezer-67238732/Instant Crush.mp3`, `youtube-dQw4w9WgXcQ/...`.
Older YouTube downloads in `<MEDIA_DIR>/<videoId>/` are still read and treated
as YouTube. One row and one file exist per track, no matter how many moods or
users reference it. Covers are cached under `<DATA_DIR>/covers` keyed by song
id.
Deezer is downloaded with `streamrip` (MP3 128/320) and YouTube Music with
`yt-dlp` (`-x --audio-format mp3 --audio-quality 0`). Both embed the artist,
album and cover art, so the indexer can derive the metadata from the file.
A file is deleted only when it has no references from `mood_songs`, an active
recommendation (`downloading`, `ready`, `added`, `failed`), or an open playback
session. GC runs after removals, after rejections, after mood deletion, and
hourly.
## Database
SQLite in WAL mode. Migrations are Flyway scripts under
`src/main/resources/db/migration`.
- **V8 is destructive.** It drops every table from the mounted-library design
(folders, artists, albums, songs, moods, seeds, sessions, longform progress,
downloads) and creates the mood-first schema. `users` and `api_tokens` are
kept.
- **V9 adds sources.** It renames `video_id` to `external_id` and adds a
`source` column to songs, moods, recommendations and download jobs. Existing
rows become `youtube`.
- There is no import from an old media directory. Back up externally and start
with an empty `MEDIA_DIR`.
## Deployment
The Helm chart is in `charts/audiomanager`. It renders one PVC at `/data`:
```
/data/musicmanager.db
/data/media/<source>-<externalId>/<title>.mp3
/data/covers/
/data/downloads/
```
No hostPath, no `library.yaml`, no init container. `fsGroup: 10001` makes the
PVC writable by the non-root container.
```bash
kubectl create secret generic audiomanager-users \
--from-file=users.yaml=config/users.yaml
# Only needed for Deezer. Leave it out to use YouTube Music only.
kubectl create secret generic audiomanager-deezer \
--from-literal=arl='<deezer-arl>'
helm upgrade --install audiomanager charts/audiomanager \
--namespace audiomanager --create-namespace \
--set users.existingSecret=audiomanager-users \
--set deezer.existingSecret=audiomanager-deezer \
--set deezer.quality=0 \
--set defaultSource=deezer \
--set image.tag=<sha>
```
`scripts/release.sh --deploy` builds/pushes the image and runs the same Helm
upgrade. CI builds `:latest` and `:sha-<commit>` on every push to `main`, and
publishes the packaged chart.
Before deploying the mood-first version over an old instance, wipe the old
database and media: V8 drops the content, but the old mounted files are no
longer referenced and should be removed by the operator. The old chart used the
same PVC name with a smaller size, and PVCs cannot shrink or be repopulated by
an upgrade, so recreate it:
```bash
kubectl -n audiomanager scale deployment/audiomanager --replicas=0
kubectl -n audiomanager delete pvc audiomanager-data
helm upgrade --install audiomanager charts/audiomanager \
--namespace audiomanager --reset-values \
--set users.existingSecret=audiomanager-users --set image.tag=<sha>
```
Old hostPath music/longform files can be deleted manually. There is no import.
## Client
Compose Multiplatform client: `/home/taptap/AudioManagerClient`.
- Android, desktop and Wasm from `commonMain`.
- Screens: Moods, Mood Detail, Search, Downloads, Settings.
- Every mood has a **Download source** selector (Deezer or YouTube Music);
search, recommendations and downloads follow it.
- Android background playback and media notifications are kept.
## Testing
```bash
./gradlew test # server unit/integration tests
./scripts/smoke.sh # end-to-end with a fake yt-dlp
scripts/check-chart.sh # lint + render the Helm chart
```
The smoke test seeds a tiny media store, puts deterministic fake `yt-dlp` and
`rip` binaries plus a fake Deezer API on localhost, and exercises both sources:
auth, search filtering, adding songs, URL resolve, downloads, recommendations,
radio sessions, streaming, covers and logout.