Files
AudioManager/README.md
T
TapTap 28bfdd9d00
CI / chart (push) Successful in 2s
CI / chart-push (push) Successful in 2s
CI / test (push) Successful in 1m41s
CI / image (push) Successful in 3m11s
feat: add per-user moods as generated playlists
Add private server-managed playlist moods with curated seeds and related
library fill. Moods are created or promoted from existing playlists in the
web UI, refresh lazily after their TTL, and are marked stale after a scan.

Symfonium stars pin the active mood: opening a mood restores its context,
and a starred song becomes a permanent seed. Without an active session the
song is pinned to every mood that already contains it.

Includes the V4 migration, custom mood endpoints, management UI, tests and
smoke coverage. Bump the chart to 0.1.4.
2026-11-14 12:52:25 +00:00

935 lines
41 KiB
Markdown

# MusicManager - Self-hosted OpenSubsonic Server (Kotlin / Ktor)
## How to use
This is the short version. The detailed design and endpoint reference follow in the sections below.
### 1. Prerequisites
- JVM 21 (or `nix-shell` from this repository, which provides JDK 21, Gradle, ffmpeg and sqlite)
- `ffmpeg` on `PATH` (used for transcoding; direct play works without it)
- An MP3 library on disk, laid out as `Artist/Album/Track.mp3` if tags are missing
### 2. Configure the library
Create the local config and media directories:
```bash
./gradlew setupLocal
```
This copies `config/library.example.yaml` to `config/library.yaml` (only when it does not exist), copies `config/users.example.yaml` to `config/users.yaml` and creates `media/music` plus `media/longform`. Both local config files are git-ignored, so `git pull` never touches them.
`config/library.yaml` defines every root the server indexes:
```yaml
library:
folders:
- id: 1
name: "Music"
type: music
path: ./media/music
- id: 10
name: "Audiobooks & Podcasts"
type: longform
path: ./media/longform
```
The example file already uses these relative paths, so no edit is needed for local development. Relative paths are resolved from the working directory (`./gradlew run` uses the project root). For deployment, replace them with absolute paths such as `/mnt/media/music/alice` (see Section 6.1).
- `type: music` roots are assigned per user; `type: longform` roots are shared with every user.
- `id` must be stable and unique (integers). It is part of song IDs, so do not renumber it after a scan.
- Paths must exist and be readable, otherwise the server refuses to start with a clear error naming the folder.
### 3. Configure users
`./gradlew setupLocal` already created it. To recreate it manually:
```bash
cp config/users.example.yaml config/users.yaml
```
```yaml
users:
- name: alice
password: "change-me"
admin: true
folders: [1]
```
- `folders` lists the *music* folder IDs the user may see; long-form folders are added automatically.
- At least one admin is needed to trigger scans.
- Passwords live only in this file and in memory. In production the file is provided by Vault (see Section 6.2); `config/users.yaml` is git-ignored.
### 4. Start the server
```bash
nix-shell # optional: provides the toolchain
./gradlew run
```
Or build and run the fat JAR:
```bash
./gradlew shadowJar
java -jar build/libs/musicmanager-0.1.0-all.jar
```
The server listens on `http://localhost:8080`. Open `http://localhost:8080/ui` for the built-in test client (log in with a user from `users.yaml`). Override with `PORT`, `DATA_DIR`, `LIBRARY_CONFIG`, `USERS_FILE`, `FFMPEG_PATH` and `WEB_UI` (see Section 6.3). The SQLite database and cover cache are written to `./data`.
On first start, the server scans automatically because the database is empty. After that, scans are manual.
### 5. Trigger a scan
Use an admin account. Token auth is what real clients use:
```bash
SALT=abc123
TOKEN=$(printf '%s%s' 'change-me' "$SALT" | md5sum | cut -d' ' -f1)
curl -s "http://localhost:8080/rest/startScan?u=alice&t=$TOKEN&s=$SALT&v=1.16.1&c=curl&f=json"
curl -s "http://localhost:8080/rest/getScanStatus?u=alice&t=$TOKEN&s=$SALT&v=1.16.1&c=curl&f=json"
```
For quick manual testing you can pass the password in plaintext instead: `&p=change-me` (use `p=enc:<hex>` if it contains special characters).
### 6. Connect a client (Symfonium)
For a quick browser check, open `http://localhost:8080/ui` and log in with a user from `users.yaml`. It lists folders, albums and songs and can play streams; it is a debugging aid, not a full music client.
For a real client:
1. Add a new **Subsonic** server.
2. URL: `http://<host>:8080` (or `https://subsonic.tap-tap.win` when deployed).
3. Username and password from `users.yaml`.
4. Let Symfonium sync. Music and long-form appear as separate **music folders** - select the one you want so the two content types never mix.
### 7. Verify it works
```bash
# health (no auth)
curl -s http://localhost:8080/health
# list the folders this user can see
curl -s "http://localhost:8080/rest/getMusicFolders?u=alice&t=$TOKEN&s=$SALT&v=1.16.1&c=curl&f=json"
# search and stream the first result
curl -s "http://localhost:8080/rest/search3?u=alice&t=$TOKEN&s=$SALT&v=1.16.1&c=curl&f=json&query=*&songCount=1"
curl -o track.mp3 "http://localhost:8080/rest/stream?u=alice&t=$TOKEN&s=$SALT&v=1.16.1&c=curl&f=json&id=<song-id>"
```
All responses are JSON (no XML). Errors are returned as HTTP 200 with a Subsonic `error.code`; HTTP status codes are reserved for transport-level problems.
### 8. Stop, back up, upgrade
- Stop the process gracefully (`Ctrl+C` or `SIGTERM`); in-flight streams are allowed to finish.
- Back up `DATA_DIR` (SQLite database + `covers/`) while stopped, or use `sqlite3 data/musicmanager.db ".backup backup.db"`.
- The schema is migrated automatically by Flyway on startup.
- Re-running a scan only reads changed files (delta by mtime + size); IDs stay stable across scans.
---
## 1. Goal & Context
Build a lightweight, self-hosted **OpenSubsonic-compatible music server** in **Kotlin** that replaces Plex for audio streaming and is optimised for the Android client **Symfonium**.
### Scope (MVP)
1. **Library indexing:** index MP3 files only for now (other extensions are skipped and logged). Tags come first, with the folder layout `Artist/Album/Track` as fallback. Data is persisted in SQLite.
2. **Symfonium sync:** implement the OpenSubsonic browsing endpoints so Symfonium can sync the full catalog into its offline-first database.
3. **Streaming & offline downloads:** serve audio files with HTTP `Range` (`206 Partial Content`), exact byte sizes and correct metadata so seeking and Symfonium's offline download manager work reliably.
4. **Two content types, never mixed:** *music* and *long-form* (audiobooks, podcasts, lectures) live in **separate music folders**. Clients see and select them independently.
5. **Multi-user:** users are defined outside the database (config + HashiCorp Vault). Music roots are assigned per user, long-form is shared. Stars, ratings, play counts, bookmarks and playlists are per user.
6. **Transcoding:** optional on-the-fly MP3 transcoding with `maxBitRate` and `timeOffset` support.
7. **Moods:** per-user private playlists generated from curated seeds and related library music. Symfonium likes pin songs to the active mood.
8. **Extensibility:** clean separation between scanner, database layer and HTTP API so a downloader and a discovery/recommendation engine can be added later.
### Explicit non-goals (MVP)
- Audio formats other than MP3 (FLAC/OGG/M4A/Opus are future work).
- XML responses (JSON only).
- Podcast RSS ingestion, chapters or audiobook-specific metadata.
- Upload, sharing, social features.
- User-management API (users come from config/Vault).
- TLS inside the app (terminated by the k3s ingress).
- Automatic filesystem watching (scans are manual or on boot when the DB is empty).
- Downloading or streaming YT Music inside the app (future phase; recommendations are metadata-only until then).
- Backups (documented manual procedure only).
### Roles
- **Admin:** may trigger scans.
- **User:** regular library access.
- All users share the same long-form roots; each user has their own music roots.
- Per-user state: play counts, last played, stars, ratings, bookmarks, playlists.
---
## 2. Tech Stack & Dependencies
- **Language & runtime:** Kotlin 2.x on JVM 21 (`eclipse-temurin:21-jre-jammy`)
- **Build:** Gradle Kotlin DSL (`build.gradle.kts`), ShadowJar fat JAR
- **Web:** Ktor Server (Netty)
- `ktor-server-core`, `ktor-server-netty`
- `ktor-server-content-negotiation` + `ktor-serialization-kotlinx-json`
- `ktor-server-partial-content` (HTTP `Range` / `206`)
- `ktor-server-status-pages`, `ktor-server-call-logging`, `ktor-server-auto-head-response`
- **Database:** SQLite in `WAL` mode + JetBrains Exposed (`exposed-core`, `exposed-dao`, `exposed-jdbc`, `exposed-java-time`) + `sqlite-jdbc` + HikariCP
- **Migrations:** Flyway (`flyway-core`) with versioned SQL under `src/main/resources/db/migration`
- **Audio metadata:** `net.jthink:jaudiotagger` (MP3/ID3 only for the MVP)
- **Covers:** `net.coobird:thumbnailator` for on-the-fly resizing
- **Transcoding:** `ffmpeg` binary in the runtime image (libmp3lame), invoked as a subprocess
- **Logging:** `logback-classic` + `kotlin-logging-jvm`
- **Config:** YAML (`library.yaml`, `users.yaml`) with env-var overrides for paths and secrets
- **Deployment:** multi-stage Dockerfile, Helm chart for k3s, Gitea Actions for tests and artifact publishing, local release script for image pushes
---
## 3. Project Structure
```text
src/main/kotlin/win/taptap/musicmanager/
|-- Application.kt # Ktor entry point, plugins, graceful shutdown
|-- config/
| |-- AppConfig.kt # Port, data dir, ffmpeg path, env overrides
| |-- LibraryConfig.kt # Music folders and their type/path
| `-- UsersConfig.kt # Users, admins and folder assignments
|-- db/
| |-- DatabaseFactory.kt # SQLite init, PRAGMA journal_mode=WAL, Flyway
| |-- Tables.kt # Exposed table definitions
| |-- LibraryRepository.kt # Visibility-scoped browsing, feeds, genres, search
| |-- UserStateRepository.kt # Play counts, stars, ratings, bookmarks
| |-- PlaylistRepository.kt # Playlist CRUD and visibility
| |-- MoodRepository.kt # Mood settings, seeds, pinned recommendations
| |-- MediaRepository.kt # Scanner persistence and aggregate rebuilds
| `-- migrations/ # Flyway SQL (V1__init.sql, ...)
|-- scanner/
| |-- MediaScanner.kt # Recursive scan, delta updates, orphan cleanup
| |-- Mp3TagExtractor.kt # jaudiotagger wrapper (tags, path fallback, duration, bitrate)
| `-- CoverArtService.kt # Cover extraction from tags/folders, disk cache, resizing
|-- mood/
| |-- MoodMixService.kt # Seed + related-library mix generation
| |-- MoodService.kt # TTL refresh and per-mood generation locking
| |-- MoodSessionTracker.kt # Active mood for Symfonium star pinning
| `-- MoodPins.kt # star/unstar -> mood seed translation
|-- subsonic/
| |-- SubsonicAuth.kt # Token (u, t, s) and password (p, enc:) auth
| |-- SubsonicResponse.kt # JSON envelope & error codes
| |-- models/ # kotlinx.serialization DTOs
| | |-- SystemModels.kt
| | |-- BrowsingModels.kt
| | |-- MediaModels.kt
| | `-- PlaylistModels.kt
| `-- routes/ # /rest/{endpoint} and /rest/{endpoint}.view
| |-- SystemRoutes.kt # ping, getLicense, extensions, startScan, getScanStatus
| |-- BrowsingRoutes.kt # indexes, artists, album, song, search3, lists, genres
| |-- MediaRoutes.kt # stream, download, getCoverArt, lyrics
| |-- InteractionRoutes.kt # scrobble, star/unstar, setRating, getStarred2
| |-- PlaylistRoutes.kt # playlists and playlist songs
| `-- MoodRoutes.kt # custom mood management endpoints
|-- transcoding/
| `-- FfmpegTranscoder.kt # MP3 transcoding with maxBitRate and timeOffset
`-- util/
`-- IdGenerator.kt # Deterministic SHA-256 IDs
```
---
## 4. Architecture Rules
### 4.1 Deterministic IDs (critical)
IDs must never change on a re-scan; otherwise Symfonium loses offline download state and playlist assignments.
- IDs use **24 hex characters** (96 bit).
- **Artist ID:** `"ar-" + sha256(normalizedAlbumArtist).take(24)`
- **Album ID:** `"al-" + sha256(normalizedAlbumArtist + "|" + normalizedAlbumTitle).take(24)`
- **Song ID:** `"sg-" + sha256(folderId + ":" + relativePathFromFolderRoot).take(24)`
- **Cover ID:** same as the album ID (`"al-..."`). Artist artwork is not served in the MVP.
**Normalization is frozen** and must never change without a migration:
1. Unicode NFC
2. Trim and collapse internal whitespace
3. Lowercase with `Locale.ROOT`
Song IDs are path-based: renaming or moving a file creates a new ID. This is accepted.
### 4.2 Album grouping via album artist
Albums are grouped by the `ALBUM_ARTIST` tag. Fallbacks: `ARTIST` tag, then parent folder name. This prevents albums with featuring artists from fragmenting.
- `albums` have one or more artists (`album_artists`), position 0 is the primary album artist.
- Songs additionally keep a display artist string and their full artist list (`song_artists`).
- `song.artist_id` in API responses means the **album artist**; track credits are exposed via `displayArtist` / `artists[]`.
### 4.3 User-scoped library visibility
- The catalog is global; users see only folders assigned to them.
- Every browsing and media query filters songs by the requesting user's visible folder IDs.
- Because album IDs are deterministic and roots can contain the same album for different users, an album may aggregate songs from several folders. Therefore:
- `getAlbum(id)` returns only songs from folders visible to the caller.
- An album/artist is returned only if it has at least one visible song.
- Album-level names, years and covers are global; per-user state is never global.
- `getMusicFolders` returns the caller's assigned music folders plus the shared long-form folders.
- Songs store `folder_id`; `music_folders` is synced from config on startup and referenced by foreign key.
- Long-form content uses the same artist/album/song model; separation is via folder type, not a separate API.
### 4.4 Exact file metadata for offline downloads
Every song object must include:
- `size: Long` - exact byte size
- `duration: Int` - seconds
- `bitRate: Int` - kbps
- `suffix: String` - `"mp3"`
- `contentType: String` - `"audio/mpeg"`
Symfonium verifies downloaded bytes against `size` after each offline download.
### 4.5 Routing conventions & `formPost`
- Register every endpoint for **both** `/{method}` and `/{method}.view`.
- Read parameters from `call.request.queryParameters` **and** `call.receiveParameters()` (POST `application/x-www-form-urlencoded`).
- Declare the `formPost` OpenSubsonic extension.
- Repeatable parameters (`songId`, `songIdToAdd`, `songIndexToRemove`) must be read as lists.
### 4.6 JSON envelope
Every response, including errors, uses `status: "ok"` or `status: "failed"` and HTTP 200:
```json
{
"subsonic-response": {
"status": "ok",
"version": "1.16.1",
"type": "musicmanager",
"serverVersion": "0.1.0",
"openSubsonic": true
}
}
```
Error codes: `0` generic, `10` missing parameter, `20` incompatible client, `30` incompatible server, `40` wrong credentials, `50` not authorized, `70` not found.
### 4.7 Transcoding rules
- `stream` accepts `maxBitRate` (kbps), `format` and `timeOffset` (seconds).
- Direct play when no transcode is requested or when the source bitrate is already `<= maxBitRate`.
- Transcoding target is always **MP3** (`libmp3lame`). A requested `format` other than `mp3` falls back to MP3.
- `timeOffset` is implemented with `ffmpeg -ss`, declared as the `transcodeOffset` OpenSubsonic extension.
- Transcoded responses are chunked and do **not** support byte `Range`. Seeking uses `timeOffset`.
- `download` always returns the original file, never a transcode.
- Direct-play responses support `Range` / `206` via the `PartialContent` plugin.
---
## 5. Database Schema (SQLite)
Identity and folder tables are synced from config at startup; passwords are never stored in SQLite.
```sql
CREATE TABLE users (
name TEXT PRIMARY KEY,
is_admin BOOLEAN NOT NULL DEFAULT 0,
updated_at TEXT NOT NULL
);
CREATE TABLE music_folders (
id INTEGER PRIMARY KEY,
name TEXT NOT NULL,
type TEXT NOT NULL CHECK (type IN ('music', 'longform')),
root_path TEXT NOT NULL
);
CREATE TABLE artists (
id TEXT PRIMARY KEY,
name TEXT NOT NULL,
sort_name TEXT NOT NULL,
cover_art TEXT,
search_text TEXT NOT NULL
);
CREATE TABLE albums (
id TEXT PRIMARY KEY,
name TEXT NOT NULL,
year INTEGER,
cover_art TEXT,
created_at TEXT NOT NULL,
search_text TEXT NOT NULL
);
CREATE TABLE album_artists (
album_id TEXT NOT NULL REFERENCES albums(id) ON DELETE CASCADE,
artist_id TEXT NOT NULL REFERENCES artists(id) ON DELETE CASCADE,
position INTEGER NOT NULL,
PRIMARY KEY (album_id, position)
);
CREATE TABLE songs (
id TEXT PRIMARY KEY,
album_id TEXT NOT NULL REFERENCES albums(id) ON DELETE CASCADE,
folder_id INTEGER NOT NULL REFERENCES music_folders(id),
title TEXT NOT NULL,
display_artist TEXT NOT NULL,
album_name TEXT NOT NULL,
track_number INTEGER,
disc_number INTEGER DEFAULT 1,
year INTEGER,
duration INTEGER NOT NULL,
bit_rate INTEGER NOT NULL,
size INTEGER NOT NULL,
suffix TEXT NOT NULL,
content_type TEXT NOT NULL,
relative_path TEXT NOT NULL,
file_mtime INTEGER NOT NULL,
created_at TEXT NOT NULL,
search_text TEXT NOT NULL,
UNIQUE (folder_id, relative_path)
);
CREATE TABLE song_artists (
song_id TEXT NOT NULL REFERENCES songs(id) ON DELETE CASCADE,
artist_id TEXT NOT NULL REFERENCES artists(id) ON DELETE CASCADE,
position INTEGER NOT NULL,
PRIMARY KEY (song_id, position)
);
-- Multi-value tags. Text keys keep the schema simple; uniqueness is enforced.
CREATE TABLE song_genres (
song_id TEXT NOT NULL REFERENCES songs(id) ON DELETE CASCADE,
genre TEXT NOT NULL,
PRIMARY KEY (song_id, genre)
);
CREATE TABLE album_genres (
album_id TEXT NOT NULL REFERENCES albums(id) ON DELETE CASCADE,
genre TEXT NOT NULL,
PRIMARY KEY (album_id, genre)
);
CREATE TABLE artist_genres (
artist_id TEXT NOT NULL REFERENCES artists(id) ON DELETE CASCADE,
genre TEXT NOT NULL,
PRIMARY KEY (artist_id, genre)
);
-- Per-user state
CREATE TABLE user_song_state (
user_name TEXT NOT NULL REFERENCES users(name) ON DELETE CASCADE,
song_id TEXT NOT NULL REFERENCES songs(id) ON DELETE CASCADE,
play_count INTEGER NOT NULL DEFAULT 0,
last_played_at TEXT,
starred_at TEXT,
rating INTEGER,
PRIMARY KEY (user_name, song_id)
);
CREATE TABLE user_stars (
user_name TEXT NOT NULL REFERENCES users(name) ON DELETE CASCADE,
item_type TEXT NOT NULL CHECK (item_type IN ('album', 'artist')),
item_id TEXT NOT NULL,
starred_at TEXT NOT NULL,
PRIMARY KEY (user_name, item_type, item_id)
);
CREATE TABLE bookmarks (
user_name TEXT NOT NULL REFERENCES users(name) ON DELETE CASCADE,
song_id TEXT NOT NULL REFERENCES songs(id) ON DELETE CASCADE,
position_ms INTEGER NOT NULL,
updated_at TEXT NOT NULL,
PRIMARY KEY (user_name, song_id)
);
CREATE TABLE playlists (
id TEXT PRIMARY KEY,
owner TEXT NOT NULL REFERENCES users(name) ON DELETE CASCADE,
name TEXT NOT NULL,
comment TEXT,
is_public BOOLEAN NOT NULL DEFAULT 1,
kind TEXT NOT NULL DEFAULT 'user' CHECK (kind IN ('user', 'mood')),
created_at TEXT NOT NULL,
changed_at TEXT NOT NULL
);
CREATE TABLE playlist_songs (
playlist_id TEXT NOT NULL REFERENCES playlists(id) ON DELETE CASCADE,
song_id TEXT NOT NULL REFERENCES songs(id) ON DELETE CASCADE,
position INTEGER NOT NULL,
PRIMARY KEY (playlist_id, position)
);
-- Moods are private playlists with generation settings. The current mix lives
-- in playlist_songs; these tables hold the anchors and future YT candidates.
CREATE TABLE moods (
playlist_id TEXT PRIMARY KEY REFERENCES playlists(id) ON DELETE CASCADE,
query TEXT,
genres TEXT NOT NULL DEFAULT '[]',
target_size INTEGER NOT NULL DEFAULT 50,
yt_slots INTEGER NOT NULL DEFAULT 0,
expand_related BOOLEAN NOT NULL DEFAULT 1,
refresh_hours INTEGER NOT NULL DEFAULT 24,
generated_at TEXT,
generation INTEGER NOT NULL DEFAULT 0,
last_error TEXT
);
CREATE TABLE mood_seeds (
playlist_id TEXT NOT NULL REFERENCES moods(playlist_id) ON DELETE CASCADE,
song_id TEXT NOT NULL REFERENCES songs(id) ON DELETE CASCADE,
origin TEXT NOT NULL DEFAULT 'manual'
CHECK (origin IN ('manual', 'promoted', 'star', 'yt_like')),
added_at TEXT NOT NULL,
PRIMARY KEY (playlist_id, song_id)
);
CREATE TABLE mood_recommendations (
playlist_id TEXT NOT NULL REFERENCES moods(playlist_id) ON DELETE CASCADE,
video_id TEXT NOT NULL,
title TEXT NOT NULL,
artist TEXT NOT NULL,
duration INTEGER,
state TEXT NOT NULL DEFAULT 'new'
CHECK (state IN ('new', 'liked', 'rejected', 'downloaded')),
matched_song_id TEXT REFERENCES songs(id) ON DELETE SET NULL,
times_added INTEGER NOT NULL DEFAULT 0,
first_seen TEXT NOT NULL,
last_seen TEXT NOT NULL,
PRIMARY KEY (playlist_id, video_id)
);
```
Required indexes:
- `albums(search_text)`, `artists(search_text)`, `songs(search_text)`
- `songs(album_id)`, `songs(folder_id)`, `songs(created_at)`
- `album_artists(artist_id)`, `song_artists(artist_id)`
- `user_song_state(user_name, starred_at)`, `user_stars(user_name, item_type)`
- `playlist_songs(song_id)`, `playlists(owner)`
- `mood_seeds(song_id)`, `mood_recommendations(matched_song_id)`
All timestamps are ISO 8601 in UTC.
---
## 6. Configuration
### 6.1 `library.yaml` (ConfigMap)
```yaml
library:
folders:
- id: 1
name: "Alice - Music"
type: music
path: /mnt/media/music/alice
- id: 2
name: "Bob - Music"
type: music
path: /mnt/media/music/bob
- id: 10
name: "Audiobooks & Podcasts"
type: longform
path: /mnt/media/longform
```
### 6.2 `users.yaml` (Vault-managed Secret)
Mounted from a HashiCorp Vault secret and read at startup. Passwords exist only in that secret and in memory - never in SQLite, logs or the Helm chart.
```yaml
users:
- name: alice
password: "<from-vault>"
admin: true
folders: [1]
- name: bob
password: "<from-vault>"
admin: false
folders: [2]
```
- Long-form folders are shared automatically; users only list their music folders.
- Env override: `USERS_FILE` (default `./config/users.yaml`; in the container `/etc/musicmanager/users.yaml`).
- Env override: `LIBRARY_CONFIG` (default `./config/library.yaml`; in the container `/etc/musicmanager/library.yaml`).
- `users` and `music_folders` tables are reconciled on startup; removed users/folders are deleted with their per-user state or orphaned songs.
- If `users.yaml` is missing or empty, the server refuses to start (fail closed).
### 6.3 Environment variables
| Variable | Default | Purpose |
| --- | --- | --- |
| `PORT` | `8080` | HTTP listen port |
| `DATA_DIR` | `./data` | SQLite DB and cover cache (`/data` in Kubernetes) |
| `LIBRARY_CONFIG` | `./config/library.yaml` | Library folder definitions |
| `USERS_FILE` | `./config/users.yaml` | Vault-managed users file |
| `FFMPEG_PATH` | `ffmpeg` | ffmpeg binary used for transcoding |
| `WEB_UI` | `true` | Serve the built-in test client at `/ui` |
### 6.4 Local development
See [How to use](#how-to-use) for the full setup. In short:
```bash
cp config/library.example.yaml config/library.yaml
cp config/users.example.yaml config/users.yaml
mkdir -p media/music media/longform
# edit the paths in config/library.yaml to point at ./media/*
nix-shell # optional: JDK 21, Gradle, ffmpeg and sqlite
./gradlew run
```
The database and covers are written to `./data`. `config/users.yaml` is git-ignored.
---
## 7. MVP Endpoint Specification
All endpoints live under `/rest/{endpoint}` and `/rest/{endpoint}.view`.
### 7.1 Authentication (`SubsonicAuth`)
For every `/rest/*` call (except an optional health check):
- `u` must match a configured user.
- Token auth: `t` and `s` present -> verify `t.equals(md5(password + s), ignoreCase = true)`.
- Password auth: `p` present -> if `p` starts with `enc:`, hex-decode it; compare to the password.
- Comparisons use constant-time equality.
- On failure: HTTP 200 with a Subsonic error (`code = 40`).
- The authenticated user is attached to the call and used for all per-user filtering and state.
### 7.2 System & scan
- `ping`: empty `subsonic-response` with `status: "ok"`.
- `getLicense`: `{"license": {"valid": true}}`.
- `getOpenSubsonicExtensions`: `[{formPost, [1]}, {songLyrics, [1]}, {transcodeOffset, [1]}]`.
- `startScan`: admin only; starts an async coroutine scan.
- `getScanStatus`: `{"scanStatus": {"scanning": Boolean, "count": Long}}`.
- `getUser(username)`: returns roles and visible folder IDs; users may only query themselves, admins may query anyone.
- `getUsers`: admin only; lists all configured users.
### 7.3 Browsing & search
- `getMusicFolders`: the caller's music folders plus shared long-form folders.
- `getIndexes`: alphabetically grouped artists visible to the caller (`index: [{name, artist[]}]`), including `lastModified` and `ignoredArticles`.
- `getArtists`: ID3-based alphabet structure (`artists: {ignoredArticles, index[]}`), filtered by visibility.
- `getArtist(id)`: artist details plus all visible albums.
- `getAlbum(id)`: album details plus visible songs, sorted by `discNumber`, `trackNumber`.
- `getSong(id)`: single song, only if visible to the caller.
- `search3(query, ...Count, ...Offset)`: case-insensitive substring search over normalized `search_text` using `LIKE` with escaped wildcards. Empty `query` returns paginated results; `query="*"` returns everything visible.
- `getGenres`: distinct genres from visible songs.
- `getSongsByGenre(genre, count, offset)`: paginated visible songs in a genre.
- `getStarred2`: the caller's starred artists, albums and songs for the home screen.
### 7.4 Home feeds
- `getAlbumList2(type, size, offset, fromYear, toYear, genre)`: `newest`, `recent`, `frequent`, `random`, `alphabeticalByName`, `alphabeticalByArtist`, `starred`, `byYear`, `byGenre`. All results are visibility- and user-scoped.
- `getRandomSongs(size, genre, fromYear, toYear)`: random visible songs.
### 7.5 Media & artwork
- `stream(id, maxBitRate, format, timeOffset)`:
- direct play via `call.respondFile` (PartialContent handles `Range`);
- otherwise stream ffmpeg MP3 output with chunked transfer;
- set `Content-Type` from the actual response.
- `download(id)`: original file with `Content-Disposition: attachment; filename*=UTF-8''...` (RFC 5987 for non-ASCII names).
- `getCoverArt(id, size)`: serve `DATA_DIR/covers/{albumId}.jpg`; if `size` is given, resize proportionally with Thumbnailator and cache as `{albumId}_{size}.jpg`. Validate `size` (`1..2048`).
- `getLyricsBySongId(id)`: read embedded ID3 lyrics or a sibling `.lrc` file; return the OpenSubsonic `lyricsList` format.
### 7.6 Interaction, bookmarks & playlists
- `scrobble(id, time, submission)`: per user; `submission=true` (default) increments `play_count` and sets `last_played_at`.
- `star(id, albumId, artistId)` / `unstar(...)`: per user.
- `setRating(id, rating)`: per user, 1-5; `0` clears.
- `getBookmarks` / `createBookmark(id, position)` / `deleteBookmark(id)`: per user; `position` is milliseconds.
- `getPlaylists`: the caller's playlists plus all public ones.
- `getPlaylist(id)`: if public or owned by the caller.
- `createPlaylist(playlistId, name, songId)`: owner is the caller.
- `updatePlaylist(playlistId, name, comment, public, songIdToAdd, songIndexToRemove)`: owner only; indices are applied in order.
- `deletePlaylist(playlistId)`: owner only.
- `getMusicDirectory(id)`: folder IDs return their albums as directories; album IDs return their songs. Used by folder-browsing clients.
- `savePlayQueue` / `getPlayQueue` (and the `*ByIndex` variants): per-user queue kept in memory.
- `jukeboxControl(action=status)` returns a stopped status; other jukebox actions report that jukebox is unsupported.
- `getInternetRadioStations` returns an empty list; radio mutation endpoints report that radio is unsupported.
No `createUser` / `updateUser` / `deleteUser` endpoints: config/Vault is the source of truth.
### 7.7 Moods (custom, for Symfonium)
OpenSubsonic has no mood entity, so a mood is a **private server-managed playlist**. The playlist ID and name stay stable, so Symfonium keeps its reference. Management lives in the built-in `/ui` page under **Moods**.
Custom endpoints use the normal Subsonic auth and JSON envelope:
- `getMoods`: list the caller's moods.
- `getMood(id)`: settings plus `seed` and `entry` song lists.
- `createMood(name, query, genre, songId, targetSize, ytSlots, expandRelated, refreshHours)`: creates a private playlist and generates the first mix.
- `updateMood(id, ...)`: updates settings, marks the mix stale and regenerates.
- `deleteMood(id)`: deletes the playlist and all mood data.
- `refreshMood(id)`: synchronous regeneration.
- `promotePlaylistToMood(playlistId, ...)`: keeps the same playlist ID and name; its current songs become `promoted` seeds.
- `addMoodSeed(id, songId)` / `removeMoodSeed(id, songId)`: manual anchors.
Generation rules:
- Only the owner's **music** folders are used. Shared long-form folders never enter a mood.
- Every seed is always included; if there are more seeds than `targetSize`, all seeds are kept.
- With `expandRelated`, remaining slots are filled from matching genres/artists and, if needed, other visible music.
- `ytSlots` is reserved for the YT Music recommendation phase.
- A scan marks all moods stale. `getPlaylist` regenerates stale moods in the background; `refreshMood` forces it immediately.
Liking in Symfonium:
- Symfonium's heart calls `star`.
- If a mood was opened within the last four hours, the starred song becomes a permanent `star` seed of that mood, even if it was not in the mix.
- Without an active mood, the song is pinned to every mood that already contains it.
- `unstar` removes only `star` seeds; manual and promoted seeds stay.
- `updatePlaylist` and `createPlaylist(playlistId=...)` reject song edits on mood playlists with error 50; renaming is still allowed.
Symfonium has no built-in weekly sync interval. Automatic sync runs on app start and when it detects server changes. For a deterministic refresh, schedule the Symfonium broadcast API with Tasker/MacroDroid/Automate:
```text
Action: app.symfonium.api.MEDIA_SYNC
Package: app.symfonik.music.player
Class: app.symfonium.remote.api.ApiReceiver
Extra: PROVIDER_ID = <provider id from Settings -> Providers>
```
---
## 8. Scanner
- Scan all configured folder roots recursively; accept `.mp3` (case-insensitive) only, log every skipped file.
- Tag-first: title, artist, album artist, album, track/disc, year, genre(s), duration, bitrate, embedded cover, lyrics.
- Fallback: parent folder as album, grandparent as artist; filename as title.
- Delta scan by `(file_mtime, size)`; unchanged files are skipped.
- Orphan cleanup: delete songs (and their state) whose files disappeared.
- Initial scan on startup only when the `songs` table is empty; otherwise rely on `startScan`.
- One scan at a time; a second `startScan` returns the current status.
- All writes in a single transaction per batch; streaming reads continue under WAL.
- Album/artist rows are recomputed after each scan (aggregated genre lists, sort names, cover references).
Cover extraction order: embedded APIC frame -> `cover.jpg`/`folder.jpg`/`front.jpg` next to the file -> album directory -> none.
---
## 9. Deployment (k3s + Helm)
### 9.1 Image
- Multi-stage Dockerfile: Gradle build -> glibc-based `eclipse-temurin:21-jre-jammy` with `ffmpeg` installed (Alpine/musl is unsupported by sqlite-jdbc's bundled native library).
- Runtime flags: `-XX:+UseSerialGC -Xmx256m` (adjustable via Helm).
- Non-root user (UID 10001), read-only root filesystem, `/tmp` as `emptyDir`, `/data` as PVC.
- Image: `gitea.tap-tap.win/taptap/audiomanager:<tag>` (registry paths must be lowercase).
- Gitea Actions builds and pushes `:sha-<commit>` and `:latest` on `main`, and the tag name for `v*` tags; `imagePullSecret` is provided by the cluster.
The CI image job uses kaniko and keeps `DOCKER_CONFIG=/kaniko/.docker` for the
whole job. A multi-stage build replaces `/` with the last stage's root
filesystem, which deletes `$HOME/.docker/config.json`; without this setting
kaniko would push anonymously and Gitea would reject it with
`401 reqPackageAccess`. For the same reason the image job checks out the source
with plain `git` instead of a JavaScript action, whose post step would lose its
Node runtime after the build.
To build and push manually (or to deploy from a workstation):
```bash
docker login gitea.tap-tap.win
scripts/release.sh # tag = git short SHA, pushes :<sha> and :latest
scripts/release.sh --tag v1.0.0 # explicit tag
scripts/release.sh --deploy # also apply the users secret and helm upgrade
```
### 9.2 Helm chart
`charts/audiomanager` renders:
- image repository/tag (defaults to `:latest` with `pullPolicy: Always`) and `imagePullSecrets`
- `library.folders` into a ConfigMap (`/etc/audiomanager/library.yaml`)
- `users.existingSecret` + key (Vault-managed; never put passwords in values)
- ingress host `subsonic.tap-tap.win`, ingress class, annotations and TLS secret;
cert-manager issues the certificate with the `letsencrypt-issuer` ClusterIssuer
by default
- `/data` PVC (size, storage class)
- `hostPath` mounts for music and long-form roots, both read-only
- resource requests/limits, non-root security context, probes on `/health`
- optional `nodeSelector`, `tolerations` and `affinity`
The registry is private, so the cluster needs an image pull secret, and the users
secret must contain the Vault-managed file. `scripts/release.sh --deploy` creates
both from the local `~/.docker/config.json` and `config/users.yaml`, then runs
`helm upgrade --install`:
```bash
docker login gitea.tap-tap.win
scripts/release.sh --tag latest --deploy
```
To deploy by hand:
```bash
kubectl create namespace audiomanager
kubectl -n audiomanager create secret generic audiomanager-users \
--from-file=users.yaml=config/users.yaml
kubectl -n audiomanager create secret generic audiomanager-registry \
--type=kubernetes.io/dockerconfigjson \
--from-file=.dockerconfigjson="$HOME/.docker/config.json"
helm upgrade --install audiomanager charts/audiomanager \
--namespace audiomanager \
--set image.tag=latest \
--set users.existingSecret=audiomanager-users \
--set imagePullSecrets[0].name=audiomanager-registry
```
Pin a specific build with `--set image.tag=sha-<commit>`; CI publishes that tag for
every push to `main`.
Adjust `library.folders`, `media.music.hostPath` and `media.longform.hostPath` for
the cluster. Every schedulable node must expose the same host paths; the chart
deliberately does not pin the pod to a node. The `/data` PVC must be writable by
UID 10001 (the chart sets `fsGroup: 10001`).
The server exits at startup if a configured root path is missing or unreadable, so
the pod will not pass its readiness probe in that case.
### 9.3 Chart repository
The chart source lives in `charts/audiomanager` in this repository. CI packages
it and publishes the `.tgz` plus `index.yaml` to the public `TapTap/Charts`
repository:
```text
https://gitea.tap-tap.win/TapTap/Charts/raw/branch/main
```
This is a classic Helm repository, so it supports both `helm repo add` and
Kustomize's `helmCharts` generator. The chart repository is public; the image
registry remains private.
Versions published by CI:
- `main`: `<chart version>-sha.<commit>`, for example `0.1.4-sha.<commit>`
- `v*` tag: the tag without the leading `v`, for example `v0.2.0` -> `0.2.0`
Use it with Helm:
```bash
helm repo add taptap \
https://gitea.tap-tap.win/TapTap/Charts/raw/branch/main
helm repo update
helm search repo taptap/audiomanager
```
Install a published chart:
```bash
VERSION="0.1.4-sha.<commit>"
helm upgrade --install audiomanager taptap/audiomanager \
--version "$VERSION" \
--namespace audiomanager \
--set users.existingSecret=audiomanager-users \
--set imagePullSecrets[0].name=audiomanager-registry
```
For Kustomize, use the same repository with `build --enable-helm`:
```yaml
helmCharts:
- name: audiomanager
releaseName: audiomanager
namespace: audiomanager
repo: https://gitea.tap-tap.win/TapTap/Charts/raw/branch/main
version: 0.1.4-sha.<commit>
valuesFile: values.yaml
```
```bash
kustomize build --enable-helm deploy/overlays/prod | kubectl apply -f -
```
`kubectl apply -k` does not run the Helm inflator; render with `kustomize build
--enable-helm` (or `kubectl kustomize --enable-helm`) first. Kustomize's Helm
generator cannot authenticate, which is why `TapTap/Charts` is public.
To publish from a workstation, either configure git credentials for
`TapTap/Charts` or pass a token explicitly:
```bash
CHARTS_TOKEN=<token> scripts/push-chart.sh
VERSION=0.2.0 CHARTS_TOKEN=<token> scripts/push-chart.sh
```
The CI job uses the `AUDIOMANAGER_CHARTS_TOKEN` repository secret. The script
clones the chart repository, adds the packaged chart, regenerates `index.yaml`
and pushes to `main`.
### 9.4 Operations
- Backups: documented manual copy of `DATA_DIR` (SQLite DB + covers) while the server is stopped, or via `sqlite3 .backup`.
- Logs to stdout (JSON optional), picked up by the cluster.
- Graceful shutdown: stop accepting requests, finish in-flight streams, close the DB pool.
---
## 10. Testing
- No audio files are committed to the repository. A local script generates tiny silent MP3 fixtures (and `.lrc` files) under `build/test-media`; integration tests skip if fixtures are absent.
- Ktor integration tests (`ktor-server-test-host`):
- token and password auth, `enc:` decoding, wrong credentials -> code 40
- deterministic IDs and normalization
- user visibility: user A cannot see or stream user B's music; both see long-form
- `Range` request -> `206 Partial Content` with exact bytes
- exact `size` / `duration` / `bitRate` / `suffix` / `contentType` fields
- browsing flow: `getArtists` -> `getArtist` -> `getAlbum` -> `stream`
- `search3` with Unicode and wildcards
- scrobble/star/rating/bookmark isolation between users
- playlist ownership and public visibility
- music directory browsing and server-side play queues
- transcoding: `maxBitRate` produces MP3, `timeOffset` seeks (skipped when `ffmpeg` is unavailable)
### Compatibility smoke test
`scripts/smoke.sh` generates a tagged MP3 with ffmpeg, starts the real server against a temporary library, and checks every endpoint a Subsonic client uses (browsing, feeds, search, stars, ratings, bookmarks, playlists, queues, stream with Range, download and covers). It prints PASS/FAIL per endpoint and exits non-zero on the first failure summary:
```bash
nix-shell --run scripts/smoke.sh
# or, with ffmpeg/sqlite/curl installed:
scripts/smoke.sh
```
The work directory can be passed as the first argument; the server log is kept there for debugging.
---
## 11. Implementation Order
Verify each step with `./gradlew test` and/or `curl`:
- [x] 1. **Setup & DB:** `build.gradle.kts`, `AppConfig.kt`, `LibraryConfig.kt`, `UsersConfig.kt`, `DatabaseFactory.kt` (WAL + Flyway), `Tables.kt`, `V1__init.sql`.
- [x] 2. **Scanner & covers:** `IdGenerator.kt`, `Mp3TagExtractor.kt`, `CoverArtService.kt`, `MediaScanner.kt` (delta + orphan cleanup + startup scan when empty).
- [x] 3. **Ktor core & auth:** `PartialContent`, content negotiation (`ignoreUnknownKeys`, `encodeDefaults`), `SubsonicAuth`, response wrapper, health endpoint.
- [x] 4. **Browsing & media:** system, browsing, genre and media routes (`stream`, `download`, `getCoverArt`).
- [x] 5. **Transcoding:** `FfmpegTranscoder.kt`, `maxBitRate`/`format`/`timeOffset`, extension declaration.
- [x] 6. **Interaction & playlists:** `getAlbumList2`, `getRandomSongs`, `getStarred2`, `search3`, `scrobble`, `star`/`unstar`, `setRating`, bookmarks, playlists.
- [x] 7. **Packaging:** multi-stage Dockerfile (with ffmpeg), Helm chart, Gitea Actions workflow.
- [x] 8. **Tests:** integration suite as described above, plus a `curl` smoke script.
- [x] 9. **Moods:** `V4__moods.sql`, `MoodRepository`, `MoodMixService`, `MoodService`, `MoodRoutes`, mood UI, Symfonium star pinning, and route tests.
- [ ] 10. **YT recommendations:** yt-dlp discovery, mood recommendation UI, liked/wishlist candidates, and the optional downloader.
---
## 12. Open Items / Assumptions
1. **Gitea registry path:** image is `gitea.tap-tap.win/taptap/audiomanager` (lowercased package owner `taptap`).
2. **Vault layout:** the chart expects a secret with a `users.yaml` key. Exact Vault path/agent template is deployment-side and not part of the app.
3. **Per-user music roots:** Vault/config must contain one folder entry per user root (e.g. `/mnt/media/music/<user>`); there is no implicit path convention.
4. **hostPath without pinning:** requires `/mnt/media/music` and `/mnt/media/longform` to exist on every node that can schedule the pod. Add a `nodeSelector` later if that changes.
5. **Long-form modelling:** audiobooks/podcasts are albums/tracks in long-form folders; no RSS ingestion or chapter model.
6. **No XML:** clients that insist on XML are unsupported.
7. **No user-management API:** user changes are applied on restart after the Vault secret/config is updated.
8. **Backups** are manual and documented; no backup endpoint.
9. **YT Music recommendations** are the next phase. `mood_recommendations` and `yt_slots` are already in the schema; generation currently falls back to library music.
10. **Mood playlists are fixed queues in Symfonium.** For fresh contents, let the lazy TTL refresh run and sync the client (Tasker recipe above).