Files
AudioManager/README.md
T
TapTap de1e95bfc6
CI / chart (push) Successful in 2s
CI / test (push) Successful in 1m59s
CI / chart-push (push) Successful in 5s
CI / image (push) Successful in 3m35s
feat(chart): set the Deezer ARL from values
- new deezer.arl value rendered as DEEZER_ARL
- existingSecret and arl are mutually exclusive
- values.local.yaml is gitignored; README documents both options
- chart 0.4.1 and a chart check for the new value
2026-11-17 16:10:56 +00:00

345 lines
14 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# AudioManager
A self-hosted, mood-first music server. You create private moods, add tracks
from Deezer or YouTube Music, and the server downloads them into a shared media
store. Recommendations are discovered and downloaded automatically so a mood
keeps playing without a mounted music library.
- Server: Kotlin, Ktor, SQLite (Exposed + Flyway), `yt-dlp`, `streamrip`,
`ffmpeg`.
- Client: Compose Multiplatform (Android, desktop, Wasm).
- No uploads, no mounted library, no likes, no podcasts.
## Contents
- [How it works](#how-it-works)
- [Quick start](#quick-start)
- [Configuration](#configuration)
- [Native API](#native-api)
- [Architecture](#architecture)
- [Media store and GC](#media-store-and-gc)
- [Database](#database)
- [Deployment](#deployment)
- [Client](#client)
- [Testing](#testing)
## How it works
- **Moods are private.** A user sees only their own moods. Songs are shared
server-wide and deduplicated by `(source, externalId)`.
- **Two catalogs.** Every mood has a download source: **Deezer** (default) or
**YouTube Music**. Search, recommendations and downloads follow that source;
permanent songs from either source can live in the same mood.
- **Songs are the collection.** A mood needs at least one song before it can
play. Removing a song only detaches it; the file is deleted when no mood,
recommendation slot or open playback session references it.
- **Recommendations are active slots.** A mood keeps `recommendationCount`
(0–30, default 10) recommendations in `downloading`, `ready` or `failed`
state. Hidden `new` candidates refill the slots automatically.
- **Discovery uses the whole mood.** Every artist (weighted by how many songs
the mood has), every album, the hint and the genres feed one pass. On Deezer
the server also asks Deezer for the artists' **radio** — a Daft Punk mood gets
MGMT, Caribou and Jamiroquai — plus related artists and album deep cuts. On
YouTube Music it follows the song's `RDAMVM` **radio queue**, the same mix the
player builds. Adding a song makes the pool refresh instead of waiting for
`refreshHours`.
- **Add / Not interested are permanent.** Adding promotes a recommendation into
the collection. Rejecting is remembered per mood. Both free a slot and the
server refills it.
- **Playback mixes both pools.** Every queue slot rolls `collectionPercent`;
the rest comes from ready recommendations. An empty pool falls back to the
other one. Small pools replay least-recently-played tracks.
- **Downloads are automatic.** Recommendation downloads run in the background
up to the configured count even if the mood is never played. Manual adds have
higher priority. Deezer needs an ARL cookie; YouTube Music needs no account.
## Quick start
Requirements: JDK 21, `ffmpeg`, `yt-dlp`, and `streamrip` (`pip install
streamrip==2.1.0`, which provides `rip`) for Deezer. The smoke test also needs
`sqlite3`, `curl` and `python3`. `nix-shell` provides everything except
`streamrip`.
```bash
# 1. Build and create local config
./gradlew setupLocal
# edit config/users.yaml (a password is required)
# 2. Run
./gradlew run
# health: http://localhost:8080/health
# API: http://localhost:8080/api/v1
```
The server creates `./data/media` (the media store), `./data/downloads`
(download temp files) and `./data/musicmanager.db`.
Deezer downloads need `DEEZER_ARL`; YouTube Music downloads need `yt-dlp` to be
able to reach YouTube. If YouTube applies bot checks, search and downloads fail
with `502 search_failed` / `failed` recommendations; retry later or update
`yt-dlp`. A Deezer Free ARL is limited to 128 kbps; 320 kbps needs Deezer
Premium (`DEEZER_QUALITY=1`).
## Configuration
Everything is environment variables:
| Variable | Default | Meaning |
|---|---|---|
| `PORT` | `8080` | HTTP port |
| `DATA_DIR` | `./data` | Database, cover cache, default media root |
| `MEDIA_DIR` | `<DATA_DIR>/media` | Server-owned song store |
| `DOWNLOAD_DIR` | `<DATA_DIR>/downloads` | yt-dlp temp/work directory |
| `USERS_FILE` | `./config/users.yaml` | User accounts |
| `FFMPEG_PATH` | `ffmpeg` | Transcoder binary |
| `YTDLP_PATH` | `yt-dlp` | YouTube Music downloader / search binary |
| `STREAMRIP_PATH` | `rip` | Deezer downloader (`streamrip`) |
| `DEFAULT_SOURCE` | `deezer` | Source for new moods: `deezer` or `youtube` |
| `DEEZER_ARL` | empty | Deezer account cookie used for downloads (chart: `deezer.arl` or `deezer.existingSecret`) |
| `DEEZER_QUALITY` | `0` | `0` = 128 kbps MP3, `1` = 320 kbps MP3 |
| `DEEZER_API_BASE` | `https://api.deezer.com` | Public Deezer API base |
### `users.yaml`
```yaml
users:
- name: admin
password: "change-me"
admin: true
```
Passwords live only in this file (use a Vault-managed Secret in Kubernetes).
`admin` is only needed for the maintenance scan endpoints. Old `folders:` keys
are accepted and ignored.
## Native API
All endpoints are under `/api/v1` and require a bearer token, except login.
Media URLs may use `?token=` because HTML audio and images cannot send headers.
### Authentication
| Method | Path | Purpose |
|---|---|---|
| `POST` | `/auth/login` | `{username,password,deviceName,platform}` → token |
| `POST` | `/auth/logout` | Revoke the current token |
| `GET` | `/me` | Current user |
| `GET` | `/auth/devices` | List device tokens |
| `DELETE` | `/auth/devices/{id}` | Revoke a device |
### Moods
| Method | Path | Purpose |
|---|---|---|
| `GET` | `/moods` | List the user's moods |
| `POST` | `/moods` | Create `{name,description,query,genres,source,collectionPercent,recommendationCount,refreshHours}` |
| `GET` | `/moods/{id}` | Mood detail: songs and active recommendations |
| `PATCH` | `/moods/{id}` | Update options |
| `DELETE` | `/moods/{id}` | Delete mood, detach songs and run GC |
| `GET` | `/moods/{id}/preview` | Unpersisted radio preview |
| `POST` | `/moods/{id}/songs` | Add `{songId}`, `{source,externalId}` or `{url}` |
| `DELETE` | `/moods/{id}/songs/{songId}` | Detach a song |
| `GET` | `/moods/{id}/recommendations` | Active recommendation slots |
| `POST` | `/moods/{id}/recommendations/refresh` | Force discovery + top-up |
| `POST` | `/moods/{id}/recommendations/{externalId}/add` | Add to the collection |
| `POST` | `/moods/{id}/recommendations/{externalId}/reject` | Hide permanently |
| `POST` | `/moods/{id}/recommendations/{externalId}/retry` | Retry a failed download |
Moods accept `source` on create/update; switching it clears the active
recommendation pool and refills it from the new catalog.
Adding an already downloaded song attaches instantly (`200`). A new track is
queued for download (`202`, `downloading: true`) and is attached when the job
finishes. `/moods/{id}/songs` accepts `{"songId":...}`, `{"source":"deezer",
"externalId":"..."}`, or `{"url":"..."}` (Deezer or YouTube).
### Search
| Method | Path | Purpose |
|---|---|---|
| `GET` | `/search?q=...&limit=...&source=...&moodId=...` | Deezer or YouTube Music search |
Results longer than 10 minutes (or shorter than 30 seconds) are hidden. Each
result reports `source`, `externalId`, `downloaded`, `songId` and `inMood`.
`source` defaults to the target mood's source, then to `DEFAULT_SOURCE`.
Deezer search uses the public API and returns canonical tracks with a square
album cover. YouTube Music search uses the "Songs" filter and resolves every
hit, which takes roughly 10–15 seconds for ten results; if Music search fails,
it falls back to plain YouTube search.
### Playback
| Method | Path | Purpose |
|---|---|---|
| `POST` | `/moods/{id}/sessions` | Start a queue (`400 empty_mood` without songs) |
| `GET` | `/sessions/{id}` | Resume a session |
| `POST` | `/sessions/{id}/next` | Fetch more tracks |
| `POST` | `/sessions/{id}/events` | Report `started`, `completed`, `skipped` |
| `POST` | `/sessions/{id}/add-to-mood` | Add the playing track to a mood |
| `DELETE` | `/sessions/{id}` | Close the session |
### Media and maintenance
| Method | Path | Purpose |
|---|---|---|
| `GET` | `/songs/{id}/stream` | Range-aware stream; `maxBitRate`, `timeOffset`, `download` |
| `GET` | `/songs/{id}/cover?size=` | Embedded artwork |
| `GET` | `/scan` | Scan status |
| `POST` | `/scan` | Start a scan (admin only) |
## Architecture
```
Ktor routes
└── MoodRepository ── moods, mood_songs, recommendation lifecycle
└── SongRepository ── shared song store keyed by (source, externalId)
└── RadioService ── mixes collection + ready recommendations
└── MusicCatalogs ── Deezer public API + yt-dlp YouTube Music search
└── DownloadService── single worker: yt-dlp/streamrip → media store → index
└── DiscoveryService── mood source search → hidden `new` candidates
└── TopUpService ── keeps active slots at recommendationCount
└── MediaScanner ── rebuilds the index from the media root
└── MediaGcService ── refcounted file deletion
```
Background loops: download worker (continuous), recommendation top-up (hourly),
media GC (hourly). A scan runs at startup.
## Media store and GC
Layout:
```
<MEDIA_DIR>/<source>-<externalId>/<sanitized-title>.mp3
```
Examples: `deezer-67238732/Instant Crush.mp3`, `youtube-dQw4w9WgXcQ/...`.
Older YouTube downloads in `<MEDIA_DIR>/<videoId>/` are still read and treated
as YouTube. One row and one file exist per track, no matter how many moods or
users reference it. Covers are cached under `<DATA_DIR>/covers` keyed by song
id.
Deezer is downloaded with `streamrip` (MP3 128/320) and YouTube Music with
`yt-dlp` (`-x --audio-format mp3 --audio-quality 0`). Both embed the artist,
album and cover art, so the indexer can derive the metadata from the file.
A file is deleted only when it has no references from `mood_songs`, an active
recommendation (`downloading`, `ready`, `added`, `failed`), or an open playback
session. GC runs after removals, after rejections, after mood deletion, and
hourly.
## Database
SQLite in WAL mode. Migrations are Flyway scripts under
`src/main/resources/db/migration`.
- **V8 is destructive.** It drops every table from the mounted-library design
(folders, artists, albums, songs, moods, seeds, sessions, longform progress,
downloads) and creates the mood-first schema. `users` and `api_tokens` are
kept.
- **V9 adds sources.** It renames `video_id` to `external_id` and adds a
`source` column to songs, moods, recommendations and download jobs. Existing
rows become `youtube`.
- There is no import from an old media directory. Back up externally and start
with an empty `MEDIA_DIR`.
## Deployment
The Helm chart is in `charts/audiomanager`. It renders one PVC at `/data`:
```
/data/musicmanager.db
/data/media/<source>-<externalId>/<title>.mp3
/data/covers/
/data/downloads/
```
No hostPath, no `library.yaml`, no init container. `fsGroup: 10001` makes the
PVC writable by the non-root container.
```bash
kubectl create secret generic audiomanager-users \
--from-file=users.yaml=config/users.yaml
# Only needed for Deezer. Leave it out to use YouTube Music only.
kubectl create secret generic audiomanager-deezer \
--from-literal=arl='<deezer-arl>'
helm upgrade --install audiomanager charts/audiomanager \
--namespace audiomanager --create-namespace \
--set users.existingSecret=audiomanager-users \
--set deezer.existingSecret=audiomanager-deezer \
--set deezer.quality=0 \
--set defaultSource=deezer \
--set image.tag=<sha>
```
`scripts/release.sh --deploy` builds/pushes the image and runs the same Helm
upgrade. CI builds `:latest` and `:sha-<commit>` on every push to `main`, and
publishes the packaged chart.
### Setting the Deezer ARL in a values file
`deezer.arl` takes the cookie directly, so no Secret is needed:
```yaml
# values.local.yaml (gitignored)
image:
tag: sha-<commit>
users:
existingSecret: audiomanager-users
defaultSource: deezer
deezer:
arl: "<arl>"
quality: 0
```
```bash
helm upgrade --install audiomanager charts/audiomanager \
--namespace audiomanager --create-namespace \
-f values.local.yaml
```
Set only one of `deezer.arl` and `deezer.existingSecret`; the chart fails
otherwise. A plain value is stored in clear text in your values file, in
`helm get values` and in the release Secret, so use `deezer.existingSecret` on a
shared cluster or an untracked `values.local.yaml` here.
Before deploying the mood-first version over an old instance, wipe the old
database and media: V8 drops the content, but the old mounted files are no
longer referenced and should be removed by the operator. The old chart used the
same PVC name with a smaller size, and PVCs cannot shrink or be repopulated by
an upgrade, so recreate it:
```bash
kubectl -n audiomanager scale deployment/audiomanager --replicas=0
kubectl -n audiomanager delete pvc audiomanager-data
helm upgrade --install audiomanager charts/audiomanager \
--namespace audiomanager --reset-values \
--set users.existingSecret=audiomanager-users --set image.tag=<sha>
```
Old hostPath music/longform files can be deleted manually. There is no import.
## Client
Compose Multiplatform client: `/home/taptap/AudioManagerClient`.
- Android, desktop and Wasm from `commonMain`.
- Screens: Moods, Mood Detail, Search, Downloads, Settings.
- Every mood has a **Download source** selector (Deezer or YouTube Music);
search, recommendations and downloads follow it.
- Android background playback and media notifications are kept.
## Testing
```bash
./gradlew test # server unit/integration tests
./scripts/smoke.sh # end-to-end with a fake yt-dlp
scripts/check-chart.sh # lint + render the Helm chart
```
The smoke test seeds a tiny media store, puts deterministic fake `yt-dlp` and
`rip` binaries plus a fake Deezer API on localhost, and exercises both sources:
auth, search filtering, adding songs, URL resolve, downloads, recommendations,
radio sessions, streaming, covers and logout.