- V11 client_logs table with per-user storage and retention/pruning - POST/GET/DELETE /logs with batch validation, rate limit and admin access - redact bearer tokens, Authorization headers and query/JSON secrets twice - add X-Request-Id to responses and include it in the server log pattern - report client error entries immediately in the server log - scripts/logs.sh for source-side retrieval and clearing - chart knobs CLIENT_LOG_RETENTION_DAYS and CLIENT_LOG_MAX_ROWS_PER_USER
16 KiB
AudioManager
A self-hosted, mood-first music server. You create private moods, add tracks from Deezer or YouTube Music, and the server downloads them into a shared media store. Recommendations are discovered and downloaded automatically so a mood keeps playing without a mounted music library.
- Server: Kotlin, Ktor, SQLite (Exposed + Flyway),
yt-dlp,streamrip,ffmpeg. - Client: Compose Multiplatform (Android, desktop, Wasm).
- No uploads, no mounted library, no likes, no podcasts.
Contents
- How it works
- Quick start
- Configuration
- Native API
- Architecture
- Media store and GC
- Database
- Deployment
- Client
- Testing
How it works
- Moods are private. A user sees only their own moods. Songs are shared
server-wide and deduplicated by
(source, externalId). - Two catalogs. Every mood has a download source: Deezer (default) or YouTube Music. Search, recommendations and downloads follow that source; permanent songs from either source can live in the same mood.
- Songs are the collection. A mood needs at least one song before it can play. Removing a song only detaches it; the file is deleted when no mood, recommendation slot or open playback session references it.
- Recommendations are active slots. A mood keeps
recommendationCount(0–30, default 10) recommendations indownloading,readyorfailedstate. Hiddennewcandidates refill the slots automatically. - Discovery uses the whole mood. Every artist (weighted by how many songs
the mood has), every album, the hint and the genres feed one pass. On Deezer
the server also asks Deezer for the artists' radio — a Daft Punk mood gets
MGMT, Caribou and Jamiroquai — plus related artists and album deep cuts. On
YouTube Music it follows the song's
RDAMVMradio queue, the same mix the player builds. Adding a song makes the pool refresh instead of waiting forrefreshHours. - Add / Not interested are permanent. Adding promotes a recommendation into the collection. Rejecting is remembered per mood. Both free a slot and the server refills it.
- Playback mixes both pools. Every queue slot rolls
collectionPercent; the rest comes from ready recommendations. An empty pool falls back to the other one. Small pools replay least-recently-played tracks. - Downloads are automatic. Recommendation downloads run in the background up to the configured count even if the mood is never played. Manual adds have higher priority. Deezer needs an ARL cookie; YouTube Music needs no account.
Quick start
Requirements: JDK 21, ffmpeg, yt-dlp, and streamrip (pip install streamrip==2.1.0, which provides rip) for Deezer. The smoke test also needs
sqlite3, curl and python3. nix-shell provides everything except
streamrip.
# 1. Build and create local config
./gradlew setupLocal
# edit config/users.yaml (a password is required)
# 2. Run
./gradlew run
# health: http://localhost:8080/health
# API: http://localhost:8080/api/v1
The server creates ./data/media (the media store), ./data/downloads
(download temp files) and ./data/musicmanager.db.
Deezer downloads need DEEZER_ARL; YouTube Music downloads need yt-dlp to be
able to reach YouTube. If YouTube applies bot checks, search and downloads fail
with 502 search_failed / failed recommendations; retry later or update
yt-dlp. A Deezer Free ARL is limited to 128 kbps; 320 kbps needs Deezer
Premium (DEEZER_QUALITY=1).
Configuration
Everything is environment variables:
| Variable | Default | Meaning |
|---|---|---|
PORT |
8080 |
HTTP port |
DATA_DIR |
./data |
Database, cover cache, default media root |
MEDIA_DIR |
<DATA_DIR>/media |
Server-owned song store |
DOWNLOAD_DIR |
<DATA_DIR>/downloads |
yt-dlp temp/work directory |
USERS_FILE |
./config/users.yaml |
User accounts |
FFMPEG_PATH |
ffmpeg |
Transcoder binary |
YTDLP_PATH |
yt-dlp |
YouTube Music downloader / search binary |
STREAMRIP_PATH |
rip |
Deezer downloader (streamrip) |
DEFAULT_SOURCE |
deezer |
Source for new moods: deezer or youtube |
DEEZER_ARL |
empty | Deezer account cookie used for downloads (chart: deezer.arl or deezer.existingSecret) |
DEEZER_QUALITY |
0 |
0 = 128 kbps MP3, 1 = 320 kbps MP3 |
DEEZER_API_BASE |
https://api.deezer.com |
Public Deezer API base |
LOUDNESS_ENABLED |
true |
Measure each file and expose a per-track gain |
LOUDNESS_TARGET_LUFS |
-16 |
Integrated loudness target (-30 to -5) |
CLIENT_LOG_RETENTION_DAYS |
14 |
Days uploaded client diagnostics are kept |
CLIENT_LOG_MAX_ROWS_PER_USER |
20000 |
Newest diagnostic rows kept per user |
users.yaml
users:
- name: admin
password: "change-me"
admin: true
Passwords live only in this file (use a Vault-managed Secret in Kubernetes).
admin is only needed for the maintenance scan endpoints. Old folders: keys
are accepted and ignored.
Client diagnostics
The apps keep a small local ring buffer (500 entries) of warnings, errors and
crashes. Warnings and errors are flushed immediately, so the line that explains
a crash survives it; info entries are batched. When signed in, the client uploads
up to 100 entries per minute to POST /api/v1/logs, tagged with a random install
id, platform, app version and per-run session id.
Privacy and safety:
Authorizationheaders,Bearertokens,token=/password=/arl=query parameters and JSON secret fields are replaced with***on the client and again on the server.- The server stores rows per user; a user can only read their own logs. An
administrator can pass
user=<name>. - Retention defaults to 14 days and 20,000 rows per user; both are configurable.
- A rejected batch (HTTP 400/413/422) is dropped so it cannot block later logs.
Inspect them with the helper:
scripts/logs.sh --server https://audiomanager.example.com --user alice \
--level warn --limit 100
# administrators can read another account
scripts/logs.sh --server ... --user admin --target alice
# clear after investigating
scripts/logs.sh --server ... --user alice --clear
The password is read from LOGS_PASSWORD or prompted; it is never stored.
Native API
All endpoints are under /api/v1 and require a bearer token, except login.
Media URLs may use ?token= because HTML audio and images cannot send headers.
Authentication
| Method | Path | Purpose |
|---|---|---|
POST |
/auth/login |
{username,password,deviceName,platform} → token |
POST |
/auth/logout |
Revoke the current token |
GET |
/me |
Current user |
GET |
/info |
Default source, Deezer quality, loudness and log retention |
GET |
/auth/devices |
List device tokens |
DELETE |
/auth/devices/{id} |
Revoke a device |
Diagnostics
| Method | Path | Purpose |
|---|---|---|
POST |
/logs |
Upload a client diagnostics batch |
GET |
/logs |
List diagnostics (level, device, since, before, limit; admins may add user) |
DELETE |
/logs |
Delete the current user's (or, for admins, a named user's) diagnostics |
Moods
| Method | Path | Purpose |
|---|---|---|
GET |
/moods |
List the user's moods |
POST |
/moods |
Create {name,description,query,genres,source,collectionPercent,recommendationCount,refreshHours} |
GET |
/moods/{id} |
Mood detail: songs, active recommendations, download status |
PATCH |
/moods/{id} |
Update options |
DELETE |
/moods/{id} |
Delete mood, detach songs and run GC |
GET |
/moods/{id}/preview |
Unpersisted radio preview |
POST |
/moods/{id}/songs |
Add {songId}, {source,externalId} or {url} |
DELETE |
/moods/{id}/songs/{songId} |
Detach a song |
GET |
/moods/{id}/recommendations |
Active recommendation slots |
POST |
/moods/{id}/recommendations/refresh |
Force discovery + top-up |
POST |
/moods/{id}/recommendations/{externalId}/add |
Add to the collection |
POST |
/moods/{id}/recommendations/{externalId}/reject |
Hide permanently |
POST |
/moods/{id}/recommendations/{externalId}/retry |
Retry a failed download |
Moods accept source on create/update; switching it clears the active
recommendation pool and refills it from the new catalog.
Adding an already downloaded song attaches instantly (200). A new track is
queued for download (202, downloading: true) and is attached when the job
finishes. /moods/{id}/songs accepts {"songId":...}, {"source":"deezer", "externalId":"..."}, or {"url":"..."} (Deezer or YouTube).
Search
| Method | Path | Purpose |
|---|---|---|
GET |
/search?q=...&limit=...&source=...&moodId=... |
Deezer or YouTube Music search |
Results longer than 10 minutes (or shorter than 30 seconds) are hidden. Each
result reports source, externalId, downloaded, songId and inMood.
source defaults to the target mood's source, then to DEFAULT_SOURCE.
Deezer search uses the public API and returns canonical tracks with a square
album cover. YouTube Music search uses the "Songs" filter and resolves every
hit, which takes roughly 10–15 seconds for ten results; if Music search fails,
it falls back to plain YouTube search.
Playback
| Method | Path | Purpose |
|---|---|---|
POST |
/moods/{id}/sessions |
Start a queue (400 empty_mood without songs) |
GET |
/sessions/{id} |
Resume a session |
POST |
/sessions/{id}/next |
Fetch more tracks |
POST |
/sessions/{id}/events |
Report started, completed, skipped |
POST |
/sessions/{id}/add-to-mood |
Add the playing track to a mood |
DELETE |
/sessions/{id} |
Close the session |
Media and maintenance
| Method | Path | Purpose |
|---|---|---|
GET |
/songs/{id}/stream |
Range-aware stream; maxBitRate, timeOffset, download |
GET |
/songs/{id}/cover?size= |
Embedded artwork |
GET |
/scan |
Scan status |
POST |
/scan |
Start a scan (admin only) |
Architecture
Ktor routes
└── MoodRepository ── moods, mood_songs, recommendation lifecycle
└── SongRepository ── shared song store keyed by (source, externalId)
└── RadioService ── mixes collection + ready recommendations
└── MusicCatalogs ── Deezer public API + yt-dlp YouTube Music search
└── DownloadService── single worker: yt-dlp/streamrip → media store → index
└── DiscoveryService── mood source search → hidden `new` candidates
└── TopUpService ── keeps active slots at recommendationCount
└── MediaScanner ── rebuilds the index from the media root
└── MediaGcService ── refcounted file deletion
Background loops: download worker (continuous), recommendation top-up (hourly), media GC (hourly). A scan runs at startup.
Media store and GC
Layout:
<MEDIA_DIR>/<source>-<externalId>/<sanitized-title>.mp3
Examples: deezer-67238732/Instant Crush.mp3, youtube-dQw4w9WgXcQ/....
Older YouTube downloads in <MEDIA_DIR>/<videoId>/ are still read and treated
as YouTube. One row and one file exist per track, no matter how many moods or
users reference it. Covers are cached under <DATA_DIR>/covers keyed by song
id.
Deezer is downloaded with streamrip (MP3 128/320) and YouTube Music with
yt-dlp (-x --audio-format mp3 --audio-quality 0). Both embed the artist,
album and cover art, so the indexer can derive the metadata from the file.
A file is deleted only when it has no references from mood_songs, an active
recommendation (downloading, ready, added, failed), or an open playback
session. GC runs after removals, after rejections, after mood deletion, and
hourly.
Database
SQLite in WAL mode. Migrations are Flyway scripts under
src/main/resources/db/migration.
- V8 is destructive. It drops every table from the mounted-library design
(folders, artists, albums, songs, moods, seeds, sessions, longform progress,
downloads) and creates the mood-first schema.
usersandapi_tokensare kept. - V9 adds sources. It renames
video_idtoexternal_idand adds asourcecolumn to songs, moods, recommendations and download jobs. Existing rows becomeyoutube. - There is no import from an old media directory. Back up externally and start
with an empty
MEDIA_DIR.
Deployment
The Helm chart is in charts/audiomanager. It renders one PVC at /data:
/data/musicmanager.db
/data/media/<source>-<externalId>/<title>.mp3
/data/covers/
/data/downloads/
No hostPath, no library.yaml, no init container. fsGroup: 10001 makes the
PVC writable by the non-root container.
kubectl create secret generic audiomanager-users \
--from-file=users.yaml=config/users.yaml
# Only needed for Deezer. Leave it out to use YouTube Music only.
kubectl create secret generic audiomanager-deezer \
--from-literal=arl='<deezer-arl>'
helm upgrade --install audiomanager charts/audiomanager \
--namespace audiomanager --create-namespace \
--set users.existingSecret=audiomanager-users \
--set deezer.existingSecret=audiomanager-deezer \
--set deezer.quality=0 \
--set defaultSource=deezer \
--set image.tag=<sha>
scripts/release.sh --deploy builds/pushes the image and runs the same Helm
upgrade. CI builds :latest and :sha-<commit> on every push to main, and
publishes the packaged chart.
Setting the Deezer ARL in a values file
deezer.arl takes the cookie directly, so no Secret is needed:
# values.local.yaml (gitignored)
image:
tag: sha-<commit>
users:
existingSecret: audiomanager-users
defaultSource: deezer
deezer:
arl: "<arl>"
quality: 0
helm upgrade --install audiomanager charts/audiomanager \
--namespace audiomanager --create-namespace \
-f values.local.yaml
Set only one of deezer.arl and deezer.existingSecret; the chart fails
otherwise. A plain value is stored in clear text in your values file, in
helm get values and in the release Secret, so use deezer.existingSecret on a
shared cluster or an untracked values.local.yaml here.
Before deploying the mood-first version over an old instance, wipe the old database and media: V8 drops the content, but the old mounted files are no longer referenced and should be removed by the operator. The old chart used the same PVC name with a smaller size, and PVCs cannot shrink or be repopulated by an upgrade, so recreate it:
kubectl -n audiomanager scale deployment/audiomanager --replicas=0
kubectl -n audiomanager delete pvc audiomanager-data
helm upgrade --install audiomanager charts/audiomanager \
--namespace audiomanager --reset-values \
--set users.existingSecret=audiomanager-users --set image.tag=<sha>
Old hostPath music/longform files can be deleted manually. There is no import.
Client
Compose Multiplatform client: /home/taptap/AudioManagerClient.
- Android, desktop and Wasm from
commonMain. - Screens: Moods, Mood Detail, Search, Downloads, Settings.
- Every mood has a Download source selector (Deezer or YouTube Music); search, recommendations and downloads follow it.
- Android background playback and media notifications are kept.
Testing
./gradlew test # server unit/integration tests
./scripts/smoke.sh # end-to-end with a fake yt-dlp
scripts/check-chart.sh # lint + render the Helm chart
The smoke test seeds a tiny media store, puts deterministic fake yt-dlp and
rip binaries plus a fake Deezer API on localhost, and exercises both sources:
auth, search filtering, adding songs, URL resolve, downloads, recommendations,
radio sessions, streaming, covers and logout.