feat: bootstrap MusicManager with scanner and Subsonic auth

- Gradle/Kotlin/Ktor project on JVM 21 with ShadowJar
- SQLite WAL, Flyway migrations and config-driven users/folders
- deterministic IDs, MP3 tag extraction, cover cache and delta scanner
- OpenSubsonic JSON envelope with token/password authentication
- system endpoints: ping, license, extensions, scan status
This commit is contained in:
TapTap committed 2026-11-13 11:42:57 +00:00
1 parent 961341b449
commit e194badefb
45 files changed
+4079 -126

No files matched your search

+470 -126
View File
@@ -1,152 +1,310 @@
# Implementation Plan: OpenSubsonic Music Server (Kotlin / Ktor)
# MusicManager — Self-hosted OpenSubsonic Server (Kotlin / Ktor)
## 1. Projektziel & Kontext
Entwickle einen leichtgewichtigen, selbstgehosteten **OpenSubsonic-kompatiblen Musik-Server** in **Kotlin**, der Plex für Musik-Streaming ablöst und speziell auf die Zusammenarbeit mit dem Android-Client **Symfonium** optimiert ist.
## 1. Goal & Context
### Kernanforderungen (MVP)
1. **Library-Indexing:** Einlesen eines lokalen Musik-Verzeichnisses (primär MP3, zusätzlich FLAC/OGG/M4A/OPUS), Auslesen der Metadaten (Tag-First mit Ordnerstruktur `Artist/Album/Track` als Fallback) und Persistierung in SQLite.
2. **Symfonium-Sync:** Vollständige Unterstützung der OpenSubsonic-Browsing-Endpunkte, damit Symfonium den gesamten Katalog sauber in seine lokale Offline-First-Datenbank synchronisiert.
3. **Streaming & Offline-Downloads:** Auslieferung der Audiodateien inkl. HTTP `Range`-Requests (`206 Partial Content`) sowie exakter Byte-Größen, damit Seeking und Symfoniums Offline-Download-Manager zuverlässig funktionieren.
4. **Bidirektionale Interaktion:** Scrobbling (Play-Counts), Favoriten (`star`/`unstar`), Bewertungen (`setRating`), Playlists und Startseiten-Feeds (`getAlbumList2`, `getRandomSongs`).
5. **Erweiterbarkeit:** Saubere Trennung zwischen Media-Scanner, Datenbank-Layer und HTTP-API, sodass später ein automatisierter Music-Downloader und eine Discovery-/Recommendation-Engine (z. B. über ListenBrainz/Last.fm oder Staging-Playlists) angedockt werden können.
Build a lightweight, self-hosted **OpenSubsonic-compatible music server** in **Kotlin** that replaces Plex for audio streaming and is optimised for the Android client **Symfonium**.
### Scope (MVP)
1. **Library indexing:** index MP3 files only for now (other extensions are skipped and logged). Tags come first, with the folder layout `Artist/Album/Track` as fallback. Data is persisted in SQLite.
2. **Symfonium sync:** implement the OpenSubsonic browsing endpoints so Symfonium can sync the full catalog into its offline-first database.
3. **Streaming & offline downloads:** serve audio files with HTTP `Range` (`206 Partial Content`), exact byte sizes and correct metadata so seeking and Symfonium's offline download manager work reliably.
4. **Two content types, never mixed:** *music* and *long-form* (audiobooks, podcasts, lectures) live in **separate music folders**. Clients see and select them independently.
5. **Multi-user:** users are defined outside the database (config + HashiCorp Vault). Music roots are assigned per user, long-form is shared. Stars, ratings, play counts, bookmarks and playlists are per user.
6. **Transcoding:** optional on-the-fly MP3 transcoding with `maxBitRate` and `timeOffset` support.
7. **Extensibility:** clean separation between scanner, database layer and HTTP API so a downloader and a discovery/recommendation engine can be added later.
### Explicit non-goals (MVP)
- Audio formats other than MP3 (FLAC/OGG/M4A/Opus are future work).
- XML responses (JSON only).
- Podcast RSS ingestion, chapters or audiobook-specific metadata.
- Upload, sharing, social features.
- User-management API (users come from config/Vault).
- TLS inside the app (terminated by the k3s ingress).
- Automatic filesystem watching (scans are manual or on boot when the DB is empty).
- Backups (documented manual procedure only).
### Roles
- **Admin:** may trigger scans.
- **User:** regular library access.
- All users share the same long-form roots; each user has their own music roots.
- Per-user state: play counts, last played, stars, ratings, bookmarks, playlists.
---
## 2. Tech-Stack & Dependencies
## 2. Tech Stack & Dependencies
* **Sprache & Runtime:** Kotlin 2.x auf JVM 21 (`eclipse-temurin:21-jre-alpine`)
* **Build-Tool:** Gradle (Kotlin DSL `build.gradle.kts`) mit `ShadowJar` Plugin (Fat-JAR)
* **Web-Framework:** **Ktor Server (Netty)**
* `ktor-server-core`, `ktor-server-netty`
* `ktor-server-content-negotiation` + `ktor-serialization-kotlinx-json`
* `ktor-server-partial-content` (Pflicht für HTTP `Range` / `206 Partial Content`)
* `ktor-server-status-pages`, `ktor-server-call-logging`, `ktor-server-auto-head-response`
* **Datenbank:** **SQLite** (im `WAL`-Mode) + **JetBrains Exposed** (`exposed-core`, `exposed-dao`, `exposed-jdbc`, `exposed-java-time`) + `sqlite-jdbc` + `HikariCP`
* **Audio-Metadaten & Cover:**
* `net.jthink:jaudiotagger` (Auslesen von ID3v1/v2, FLAC, Ogg, M4A, Dauer, Bitrate, Cover, Lyrics)
* `net.coobird:thumbnailator` (On-the-fly Resizing von Album-Covern für `getCoverArt?size=...`)
* **Logging:** `logback-classic` + `kotlin-logging-jvm`
- **Language & runtime:** Kotlin 2.x on JVM 21 (`eclipse-temurin:21-jre-alpine`)
- **Build:** Gradle Kotlin DSL (`build.gradle.kts`), ShadowJar fat JAR
- **Web:** Ktor Server (Netty)
- `ktor-server-core`, `ktor-server-netty`
- `ktor-server-content-negotiation` + `ktor-serialization-kotlinx-json`
- `ktor-server-partial-content` (HTTP `Range` / `206`)
- `ktor-server-status-pages`, `ktor-server-call-logging`, `ktor-server-auto-head-response`
- **Database:** SQLite in `WAL` mode + JetBrains Exposed (`exposed-core`, `exposed-dao`, `exposed-jdbc`, `exposed-java-time`) + `sqlite-jdbc` + HikariCP
- **Migrations:** Flyway (`flyway-core`) with versioned SQL under `src/main/resources/db/migration`
- **Audio metadata:** `net.jthink:jaudiotagger` (MP3/ID3 only for the MVP)
- **Covers:** `net.coobird:thumbnailator` for on-the-fly resizing
- **Transcoding:** `ffmpeg` binary in the runtime image (libmp3lame), invoked as a subprocess
- **Logging:** `logback-classic` + `kotlin-logging-jvm`
- **Config:** YAML (`library.yaml`, `users.yaml`) with env-var overrides for paths and secrets
- **Deployment:** multi-stage Dockerfile, Helm chart for k3s, Gitea Actions for image builds
---
## 3. Projektstruktur
## 3. Project Structure
```text
src/main/kotlin/com/musicserver/
├── Application.kt # Ktor Einstiegspunkt, Plugins, Graceful Shutdown
src/main/kotlin/win/taptap/musicmanager/
├── Application.kt # Ktor entry point, plugins, graceful shutdown
├── config/
│ └── AppConfig.kt # Env-Vars (MUSIC_DIR, DATA_DIR, SUBSONIC_USER, SUBSONIC_PASS, PORT)
│ ├── AppConfig.kt # Port, data dir, ffmpeg path, env overrides
│ ├── LibraryConfig.kt # Music folders and their type/path
│ └── UsersConfig.kt # Users, admins and folder assignments
├── db/
│ ├── DatabaseFactory.kt # SQLite Init, PRAGMA journal_mode=WAL, Schema-Migration
│ ├── Tables.kt # Exposed Table-Definitionen
│ └── Repositories.kt # CRUD- & Query-Logik für Artists, Albums, Songs, Playlists
│ ├── DatabaseFactory.kt # SQLite init, PRAGMA journal_mode=WAL, Flyway
│ ├── Tables.kt # Exposed table definitions
│ ├── Repositories.kt # CRUD & queries for artists, albums, songs, playlists
│ └── migrations/ # Flyway SQL (V1__init.sql, ...)
├── scanner/
│ ├── MediaScanner.kt # Rekursiver Verzeichnis-Scan, Delta-Updates, Orphan-Cleanup
│ ├── TagExtractor.kt # jaudiotagger Wrapper (Tags, Fallback auf Pfad, Dauer, Bitrate)
│ └── CoverArtService.kt # Cover-Extraktion aus Tags/Ordnern, Disk-Cache & Resizing
│ ├── MediaScanner.kt # Recursive scan, delta updates, orphan cleanup
│ ├── Mp3TagExtractor.kt # jaudiotagger wrapper (tags, path fallback, duration, bitrate)
│ └── CoverArtService.kt # Cover extraction from tags/folders, disk cache, resizing
├── subsonic/
│ ├── SubsonicAuth.kt # Ktor Plugin/Interceptor für Token-Auth (u, t, s) & Plaintext (p)
│ ├── SubsonicResponse.kt # Standardisierter JSON-Envelope ("subsonic-response") & Fehlercodes
│ ├── models/ # kotlinx.serialization DTOs für OpenSubsonic
│ ├── SubsonicAuth.kt # Token (u, t, s) and password (p, enc:) auth
│ ├── SubsonicResponse.kt # JSON envelope & error codes
│ ├── models/ # kotlinx.serialization DTOs
│ │ ├── SystemModels.kt
│ │ ├── BrowsingModels.kt
│ │ ├── MediaModels.kt
│ │ └── PlaylistModels.kt
│ └── routes/ # Routing unter /rest/{endpoint} und /rest/{endpoint}.view
│ ├── SystemRoutes.kt # ping, getLicense, getOpenSubsonicExtensions, startScan, getScanStatus
│ ├── BrowsingRoutes.kt # getMusicFolders, getIndexes, getArtists, getArtist, getAlbum, getSong, search3, getAlbumList2, getRandomSongs
│ ├── MediaRoutes.kt # stream, download, getCoverArt, getLyricsBySongId
│ ├── InteractionRoutes.kt # scrobble, star, unstar, setRating
│ └── PlaylistRoutes.kt # getPlaylists, getPlaylist, createPlaylist, updatePlaylist, deletePlaylist
│ └── routes/ # /rest/{endpoint} and /rest/{endpoint}.view
│ ├── SystemRoutes.kt # ping, getLicense, extensions, startScan, getScanStatus
│ ├── BrowsingRoutes.kt # indexes, artists, album, song, search3, lists, genres
│ ├── MediaRoutes.kt # stream, download, getCoverArt, lyrics
│ ├── InteractionRoutes.kt # scrobble, star/unstar, setRating, getStarred2
│ └── PlaylistRoutes.kt # playlists and playlist songs
├── transcoding/
│ └── FfmpegTranscoder.kt # MP3 transcoding with maxBitRate and timeOffset
└── util/
└── IdGenerator.kt # Deterministische SHA-256 ID-Generierung
└── IdGenerator.kt # Deterministic SHA-256 IDs
```
---
## 4. Kritische Architektur-Regeln (Symfonium-Kompatibilität)
## 4. Architecture Rules
1. **Deterministische IDs (Extrem wichtig!):**
* IDs dürfen sich bei einem erneuten Library-Scan **niemals** ändern, sonst verliert Symfonium den Offline-Download-Status und Playlist-Zuordnungen.
* **Artist-ID:** `"ar-" + sha256(normalizedAlbumArtist).take(16)`
* **Album-ID:** `"al-" + sha256(normalizedAlbumArtist + "|" + normalizedAlbumTitle).take(16)`
* **Song-ID:** `"sg-" + sha256(relativeFilePathFromMusicDir).take(16)`
* **Cover-ID:** Identisch zur `Album-ID` (`"al-..."`) bzw. `Artist-ID` (`"ar-..."`).
2. **Album-Gruppierung über `AlbumArtist`:**
* Alben müssen primär über den `ALBUM_ARTIST`-Tag gruppiert werden (Fallback: `ARTIST`-Tag, Fallback: Elternordner). Andernfalls zerfallen Alben mit Featuring-Artists in einzelne Fragmente.
3. **Exakte Datei-Metadaten für Offline-Downloads:**
* Jedes Song-Objekt muss zwingend die exakte Dateigröße in Bytes (`size: Long`), die Spieldauer in Sekunden (`duration: Int`), `bitRate: Int`, `suffix: String` (z. B. `"mp3"`) und `contentType: String` (z. B. `"audio/mpeg"`) liefern. Symfonium prüft nach jedem Offline-Download, ob die heruntergeladenen Bytes exakt mit `size` übereinstimmen.
4. **Routing-Konvention & `formPost`:**
* Symfonium sendet Requests wahlweise an `/rest/getArtists` oder `/rest/getArtists.view` und nutzt bei langen Parameterlisten (z. B. `updatePlaylist`) `POST` mit `application/x-www-form-urlencoded`.
* Registriere alle Endpunkte für **beide** URL-Varianten (`/{method}` und `/{method}.view`) und lies Parameter vereinheitlicht aus `call.request.queryParameters` **und** `call.receiveParameters()` (bei POST).
5. **OpenSubsonic JSON-Envelope:**
* Jede Antwort (Standard `f=json`) muss in folgendes Root-Objekt verpackt sein:
```json
{
"subsonic-response": {
"status": "ok",
"version": "1.16.1",
"type": "subsonic-kt",
"serverVersion": "0.1.0",
"openSubsonic": true
}
}
```
### 4.1 Deterministic IDs (critical)
IDs must never change on a re-scan; otherwise Symfonium loses offline download state and playlist assignments.
- IDs use **24 hex characters** (96 bit).
- **Artist ID:** `"ar-" + sha256(normalizedAlbumArtist).take(24)`
- **Album ID:** `"al-" + sha256(normalizedAlbumArtist + "|" + normalizedAlbumTitle).take(24)`
- **Song ID:** `"sg-" + sha256(folderId + ":" + relativePathFromFolderRoot).take(24)`
- **Cover ID:** same as the album ID (`"al-..."`). Artist artwork is not served in the MVP.
**Normalization is frozen** and must never change without a migration:
1. Unicode NFC
2. Trim and collapse internal whitespace
3. Lowercase with `Locale.ROOT`
Song IDs are path-based: renaming or moving a file creates a new ID. This is accepted.
### 4.2 Album grouping via album artist
Albums are grouped by the `ALBUM_ARTIST` tag. Fallbacks: `ARTIST` tag, then parent folder name. This prevents albums with featuring artists from fragmenting.
- `albums` have one or more artists (`album_artists`), position 0 is the primary album artist.
- Songs additionally keep a display artist string and their full artist list (`song_artists`).
- `song.artist_id` in API responses means the **album artist**; track credits are exposed via `displayArtist` / `artists[]`.
### 4.3 User-scoped library visibility
- The catalog is global; users see only folders assigned to them.
- Every browsing and media query filters songs by the requesting user's visible folder IDs.
- Because album IDs are deterministic and roots can contain the same album for different users, an album may aggregate songs from several folders. Therefore:
- `getAlbum(id)` returns only songs from folders visible to the caller.
- An album/artist is returned only if it has at least one visible song.
- Album-level names, years and covers are global; per-user state is never global.
- `getMusicFolders` returns the caller's assigned music folders plus the shared long-form folders.
- Songs store `folder_id`; `music_folders` is synced from config on startup and referenced by foreign key.
- Long-form content uses the same artist/album/song model; separation is via folder type, not a separate API.
### 4.4 Exact file metadata for offline downloads
Every song object must include:
- `size: Long` — exact byte size
- `duration: Int` — seconds
- `bitRate: Int` — kbps
- `suffix: String` — `"mp3"`
- `contentType: String` — `"audio/mpeg"`
Symfonium verifies downloaded bytes against `size` after each offline download.
### 4.5 Routing conventions & `formPost`
- Register every endpoint for **both** `/{method}` and `/{method}.view`.
- Read parameters from `call.request.queryParameters` **and** `call.receiveParameters()` (POST `application/x-www-form-urlencoded`).
- Declare the `formPost` OpenSubsonic extension.
- Repeatable parameters (`songId`, `songIdToAdd`, `songIndexToRemove`) must be read as lists.
### 4.6 JSON envelope
Every response, including errors, uses `status: "ok"` or `status: "failed"` and HTTP 200:
```json
{
"subsonic-response": {
"status": "ok",
"version": "1.16.1",
"type": "musicmanager",
"serverVersion": "0.1.0",
"openSubsonic": true
}
}
```
Error codes: `0` generic, `10` missing parameter, `20` incompatible client, `30` incompatible server, `40` wrong credentials, `50` not authorized, `70` not found.
### 4.7 Transcoding rules
- `stream` accepts `maxBitRate` (kbps), `format` and `timeOffset` (seconds).
- Direct play when no transcode is requested or when the source bitrate is already `<= maxBitRate`.
- Transcoding target is always **MP3** (`libmp3lame`). A requested `format` other than `mp3` falls back to MP3.
- `timeOffset` is implemented with `ffmpeg -ss`, declared as the `transcodeOffset` OpenSubsonic extension.
- Transcoded responses are chunked and do **not** support byte `Range`. Seeking uses `timeOffset`.
- `download` always returns the original file, never a transcode.
- Direct-play responses support `Range` / `206` via the `PartialContent` plugin.
---
## 5. Datenbank-Schema (SQLite)
## 5. Database Schema (SQLite)
Identity and folder tables are synced from config at startup; passwords are never stored in SQLite.
```sql
CREATE TABLE users (
name TEXT PRIMARY KEY,
is_admin BOOLEAN NOT NULL DEFAULT 0,
updated_at TEXT NOT NULL
);
CREATE TABLE music_folders (
id INTEGER PRIMARY KEY,
name TEXT NOT NULL,
type TEXT NOT NULL CHECK (type IN ('music', 'longform')),
root_path TEXT NOT NULL
);
CREATE TABLE artists (
id TEXT PRIMARY KEY,
name TEXT NOT NULL,
sort_name TEXT NOT NULL,
cover_art TEXT,
starred_at TEXT
search_text TEXT NOT NULL
);
CREATE TABLE albums (
id TEXT PRIMARY KEY,
artist_id TEXT NOT NULL REFERENCES artists(id) ON DELETE CASCADE,
name TEXT NOT NULL,
year INTEGER,
genre TEXT,
cover_art TEXT,
created_at TEXT NOT NULL,
starred_at TEXT,
user_rating INTEGER
search_text TEXT NOT NULL
);
CREATE TABLE album_artists (
album_id TEXT NOT NULL REFERENCES albums(id) ON DELETE CASCADE,
artist_id TEXT NOT NULL REFERENCES artists(id) ON DELETE CASCADE,
position INTEGER NOT NULL,
PRIMARY KEY (album_id, position)
);
CREATE TABLE songs (
id TEXT PRIMARY KEY,
album_id TEXT NOT NULL REFERENCES albums(id) ON DELETE CASCADE,
artist_id TEXT NOT NULL REFERENCES artists(id) ON DELETE CASCADE,
folder_id INTEGER NOT NULL REFERENCES music_folders(id),
title TEXT NOT NULL,
artist_name TEXT NOT NULL,
display_artist TEXT NOT NULL,
album_name TEXT NOT NULL,
track_number INTEGER,
disc_number INTEGER DEFAULT 1,
year INTEGER,
genre TEXT,
duration INTEGER NOT NULL,
bit_rate INTEGER NOT NULL,
size INTEGER NOT NULL,
suffix TEXT NOT NULL,
content_type TEXT NOT NULL,
relative_path TEXT NOT NULL UNIQUE,
relative_path TEXT NOT NULL,
file_mtime INTEGER NOT NULL,
created_at TEXT NOT NULL,
search_text TEXT NOT NULL,
UNIQUE (folder_id, relative_path)
);
CREATE TABLE song_artists (
song_id TEXT NOT NULL REFERENCES songs(id) ON DELETE CASCADE,
artist_id TEXT NOT NULL REFERENCES artists(id) ON DELETE CASCADE,
position INTEGER NOT NULL,
PRIMARY KEY (song_id, position)
);
-- Multi-value tags. Text keys keep the schema simple; uniqueness is enforced.
CREATE TABLE song_genres (
song_id TEXT NOT NULL REFERENCES songs(id) ON DELETE CASCADE,
genre TEXT NOT NULL,
PRIMARY KEY (song_id, genre)
);
CREATE TABLE album_genres (
album_id TEXT NOT NULL REFERENCES albums(id) ON DELETE CASCADE,
genre TEXT NOT NULL,
PRIMARY KEY (album_id, genre)
);
CREATE TABLE artist_genres (
artist_id TEXT NOT NULL REFERENCES artists(id) ON DELETE CASCADE,
genre TEXT NOT NULL,
PRIMARY KEY (artist_id, genre)
);
-- Per-user state
CREATE TABLE user_song_state (
user_name TEXT NOT NULL REFERENCES users(name) ON DELETE CASCADE,
song_id TEXT NOT NULL REFERENCES songs(id) ON DELETE CASCADE,
play_count INTEGER NOT NULL DEFAULT 0,
last_played_at TEXT,
starred_at TEXT,
user_rating INTEGER
rating INTEGER,
PRIMARY KEY (user_name, song_id)
);
CREATE TABLE user_stars (
user_name TEXT NOT NULL REFERENCES users(name) ON DELETE CASCADE,
item_type TEXT NOT NULL CHECK (item_type IN ('album', 'artist')),
item_id TEXT NOT NULL,
starred_at TEXT NOT NULL,
PRIMARY KEY (user_name, item_type, item_id)
);
CREATE TABLE bookmarks (
user_name TEXT NOT NULL REFERENCES users(name) ON DELETE CASCADE,
song_id TEXT NOT NULL REFERENCES songs(id) ON DELETE CASCADE,
position_ms INTEGER NOT NULL,
updated_at TEXT NOT NULL,
PRIMARY KEY (user_name, song_id)
);
CREATE TABLE playlists (
id TEXT PRIMARY KEY,
owner TEXT NOT NULL REFERENCES users(name) ON DELETE CASCADE,
name TEXT NOT NULL,
comment TEXT,
owner TEXT NOT NULL,
is_public BOOLEAN NOT NULL DEFAULT 1,
created_at TEXT NOT NULL,
changed_at TEXT NOT NULL
@@ -160,55 +318,241 @@ CREATE TABLE playlist_songs (
);
```
---
Required indexes:
## 6. Spezifikation der MVP-Endpunkte
- `albums(search_text)`, `artists(search_text)`, `songs(search_text)`
- `songs(album_id)`, `songs(folder_id)`, `songs(created_at)`
- `album_artists(artist_id)`, `song_artists(artist_id)`
- `user_song_state(user_name, starred_at)`, `user_stars(user_name, item_type)`
- `playlist_songs(song_id)`, `playlists(owner)`
### Authentifizierung (`SubsonicAuth`)
* Prüfe bei jedem `/rest/*`-Call (außer optional Healthcheck):
* `u`: Muss mit `SUBSONIC_USER` übereinstimmen.
* Entweder Token-Auth: `t` und `s` vorhanden -> prüfe `t.equals(md5(SUBSONIC_PASS + s), ignoreCase = true)`.
* Oder Passwort-Auth: `p` vorhanden -> falls `p.startsWith("enc:")`, Hex-dekodieren, sonst direkt mit `SUBSONIC_PASS` vergleichen.
* Bei Fehler: HTTP 200 mit Subsonic-Error-JSON (`code = 40`, `message = "Wrong username or password"`).
### Endpunkt-Liste
1. **System & Scan:**
* `ping`: Leeres `subsonic-response` mit `status: "ok"`.
* `getLicense`: `{"license": {"valid": true}}`.
* `getOpenSubsonicExtensions`: Deklariert `[{"name": "formPost", "versions": [1]}, {"name": "songLyrics", "versions": [1]}]`.
* `startScan` / `getScanStatus`: Startet asynchronen Coroutine-Scan und gibt `{"scanStatus": {"scanning": Boolean, "count": Long}}` zurück.
2. **Browsing & Suche:**
* `getMusicFolders`: Liefert `[{"id": 1, "name": "Music"}]`.
* `getIndexes`: Alphabetisch gruppierte Artists (`index: [{name: "A", artist: [...]}]`).
* `getArtists`: Identische Alphabet-Struktur für ID3-basiertes Browsing (`artists: {ignoredArticles: "The El La Los Las Le Les", index: [...]}`).
* `getArtist(id)`: Artist-Details inkl. Liste aller zugehörigen `album`-Objekte.
* `getAlbum(id)`: Album-Details inkl. aller `song`-Objekte (sortiert nach `discNumber`, `trackNumber`).
* `getSong(id)`: Einzelnes Song-Objekt.
* `search3(query, artistCount, artistOffset, albumCount, albumOffset, songCount, songOffset)`: SQL-`LIKE`-Suche über Artists, Alben und Songs (`query=""` darf leere oder paginierte Ergebnisse liefern).
3. **Home-Feeds:**
* `getAlbumList2(type, size, offset, fromYear, toYear, genre)`: Unterstützt `type` = `newest`, `recent`, `frequent`, `random`, `alphabeticalByName`, `alphabeticalByArtist`, `starred`, `byYear`, `byGenre`.
* `getRandomSongs(size, genre, fromYear, toYear)`: Zufällige Songs aus der DB.
4. **Media & Artwork:**
* `stream(id, maxBitRate, format)`: Liefert die Datei via `call.respondFile(file)` aus (Direct Play; `PartialContent`-Plugin übernimmt `Range`-Requests). Setzt `Content-Type` passend zur Datei.
* `download(id)`: Wie `stream`, zusätzlich mit `Content-Disposition: attachment; filename="..."`.
* `getCoverArt(id, size)`: Lädt das gecachte Cover aus `DATA_DIR/covers/{id}.jpg`. Falls `size` (Int) angegeben ist, skaliert `Thumbnailator` das Bild proportional auf `size x size` und cached die skalierte Variante (`{id}_{size}.jpg`).
* `getLyricsBySongId(id)`: Liest eingebettete Lyrics (`FieldKey.LYRICS`) oder eine gleichnamige `.lrc`-Datei neben der Audiodatei und gibt sie im OpenSubsonic-`lyricsList`-Format zurück.
5. **Interaktion & Playlists:**
* `scrobble(id, time, submission)`: Bei `submission=true` (Default) `play_count + 1` und `last_played_at` aktualisieren.
* `star(id, albumId, artistId)` / `unstar(id, albumId, artistId)`: Setzt bzw. löscht `starred_at`.
* `setRating(id, rating)`: Setzt `user_rating` (1–5; bei `0` auf `null`).
* `getPlaylists`, `getPlaylist(id)`, `createPlaylist(playlistId, name, songId)`, `updatePlaylist(playlistId, name, comment, public, songIdToAdd, songIndexToRemove)`, `deletePlaylist(playlistId)`.
All timestamps are ISO 8601 in UTC.
---
## 7. Schrittweise Implementierungs-Reihenfolge
## 6. Configuration
Setze das Projekt inkrementell in folgenden Schritten um und verifiziere jeden Schritt über `./gradlew test` bzw. `curl`:
### 6.1 `library.yaml` (ConfigMap)
1. **Setup & DB:** Erstelle `build.gradle.kts`, `AppConfig.kt`, `DatabaseFactory.kt` (inkl. SQLite WAL-Pragma) und `Tables.kt`.
2. **Scanner & Cover-Cache:** Implementiere `IdGenerator.kt`, `TagExtractor.kt` (`jaudiotagger`), `CoverArtService.kt` (`Thumbnailator`) und `MediaScanner.kt` (inkl. Entfernen gelöschter Dateien aus der DB). Führe beim Serverstart automatisch einen initialen Scan aus, falls die DB leer ist.
3. **Ktor Core & Auth:** Konfiguriere Ktor mit `PartialContent`, `ContentNegotiation` (`ignoreUnknownKeys = true`, `encodeDefaults = true`), `SubsonicAuth` und dem `SubsonicResponse`-Wrapper.
4. **Endpoints (Browsing + Media):** Implementiere System-, Browsing- und Media-Routes (`stream`, `download`, `getCoverArt`).
5. **Endpoints (Sync + Interaktion + Playlists):** Implementiere `getAlbumList2`, `search3`, `scrobble`, `star`/`unstar`, `setRating` und alle Playlist-Endpunkte.
6. **Containerisierung:** Erstelle ein Multi-Stage `Dockerfile` (Gradle Build -> `eclipse-temurin:21-jre-alpine` Runtime mit `-XX:+UseSerialGC -Xmx256m`) sowie eine `docker-compose.yml` mit Volumes für `/music:ro` und `/data`.
7. **Automatisierte Tests:** Schreibe Ktor-Integrationstests (`ktor-server-test-host`) für Auth-Hashing, deterministische IDs, `Range`-Header (`206 Partial Content`) beim Streaming und den kompletten Browsing-Flow (`getArtists` -> `getArtist` -> `getAlbum` -> `stream`).
```yaml
library:
folders:
- id: 1
name: "Alice — Music"
type: music
path: /mnt/media/music/alice
- id: 2
name: "Bob — Music"
type: music
path: /mnt/media/music/bob
- id: 10
name: "Audiobooks & Podcasts"
type: longform
path: /mnt/media/longform
```
### 6.2 `users.yaml` (Vault-managed Secret)
Mounted from a HashiCorp Vault secret and read at startup. Passwords exist only in that secret and in memory — never in SQLite, logs or the Helm chart.
```yaml
users:
- name: alice
password: "<from-vault>"
admin: true
folders: [1]
- name: bob
password: "<from-vault>"
admin: false
folders: [2]
```
- Long-form folders are shared automatically; users only list their music folders.
- Env override: `USERS_FILE` (default `./config/users.yaml`; in the container `/etc/musicmanager/users.yaml`).
- Env override: `LIBRARY_CONFIG` (default `./config/library.yaml`; in the container `/etc/musicmanager/library.yaml`).
- `users` and `music_folders` tables are reconciled on startup; removed users/folders are deleted with their per-user state or orphaned songs.
- If `users.yaml` is missing or empty, the server refuses to start (fail closed).
### 6.3 Environment variables
| Variable | Default | Purpose |
| --- | --- | --- |
| `PORT` | `8080` | HTTP listen port |
| `DATA_DIR` | `./data` | SQLite DB and cover cache (`/data` in Kubernetes) |
| `LIBRARY_CONFIG` | `./config/library.yaml` | Library folder definitions |
| `USERS_FILE` | `./config/users.yaml` | Vault-managed users file |
| `FFMPEG_PATH` | `ffmpeg` | ffmpeg binary used for transcoding |
### 6.4 Local development
```bash
cp config/library.example.yaml config/library.yaml
cp config/users.example.yaml config/users.yaml
mkdir -p media/music media/longform
# edit the paths in config/library.yaml to point at ./media/*
./gradlew run
```
The database and covers are written to `./data`. `config/users.yaml` is git-ignored.
---
## 7. MVP Endpoint Specification
All endpoints live under `/rest/{endpoint}` and `/rest/{endpoint}.view`.
### 7.1 Authentication (`SubsonicAuth`)
For every `/rest/*` call (except an optional health check):
- `u` must match a configured user.
- Token auth: `t` and `s` present → verify `t.equals(md5(password + s), ignoreCase = true)`.
- Password auth: `p` present → if `p` starts with `enc:`, hex-decode it; compare to the password.
- Comparisons use constant-time equality.
- On failure: HTTP 200 with a Subsonic error (`code = 40`).
- The authenticated user is attached to the call and used for all per-user filtering and state.
### 7.2 System & scan
- `ping`: empty `subsonic-response` with `status: "ok"`.
- `getLicense`: `{"license": {"valid": true}}`.
- `getOpenSubsonicExtensions`: `[{formPost, [1]}, {songLyrics, [1]}, {transcodeOffset, [1]}]`.
- `startScan`: admin only; starts an async coroutine scan.
- `getScanStatus`: `{"scanStatus": {"scanning": Boolean, "count": Long}}`.
### 7.3 Browsing & search
- `getMusicFolders`: the caller's music folders plus shared long-form folders.
- `getIndexes`: alphabetically grouped artists visible to the caller (`index: [{name, artist[]}]`), including `lastModified` and `ignoredArticles`.
- `getArtists`: ID3-based alphabet structure (`artists: {ignoredArticles, index[]}`), filtered by visibility.
- `getArtist(id)`: artist details plus all visible albums.
- `getAlbum(id)`: album details plus visible songs, sorted by `discNumber`, `trackNumber`.
- `getSong(id)`: single song, only if visible to the caller.
- `search3(query, …Count, …Offset)`: case-insensitive substring search over normalized `search_text` using `LIKE` with escaped wildcards. Empty `query` returns paginated results; `query="*"` returns everything visible.
- `getGenres`: distinct genres from visible songs.
- `getSongsByGenre(genre, count, offset)`: paginated visible songs in a genre.
- `getStarred2`: the caller's starred artists, albums and songs for the home screen.
### 7.4 Home feeds
- `getAlbumList2(type, size, offset, fromYear, toYear, genre)`: `newest`, `recent`, `frequent`, `random`, `alphabeticalByName`, `alphabeticalByArtist`, `starred`, `byYear`, `byGenre`. All results are visibility- and user-scoped.
- `getRandomSongs(size, genre, fromYear, toYear)`: random visible songs.
### 7.5 Media & artwork
- `stream(id, maxBitRate, format, timeOffset)`:
- direct play via `call.respondFile` (PartialContent handles `Range`);
- otherwise stream ffmpeg MP3 output with chunked transfer;
- set `Content-Type` from the actual response.
- `download(id)`: original file with `Content-Disposition: attachment; filename*=UTF-8''…` (RFC 5987 for non-ASCII names).
- `getCoverArt(id, size)`: serve `DATA_DIR/covers/{albumId}.jpg`; if `size` is given, resize proportionally with Thumbnailator and cache as `{albumId}_{size}.jpg`. Validate `size` (`1..2048`).
- `getLyricsBySongId(id)`: read embedded ID3 lyrics or a sibling `.lrc` file; return the OpenSubsonic `lyricsList` format.
### 7.6 Interaction, bookmarks & playlists
- `scrobble(id, time, submission)`: per user; `submission=true` (default) increments `play_count` and sets `last_played_at`.
- `star(id, albumId, artistId)` / `unstar(...)`: per user.
- `setRating(id, rating)`: per user, 1–5; `0` clears.
- `getBookmarks` / `createBookmark(id, position)` / `deleteBookmark(id)`: per user; `position` is milliseconds.
- `getPlaylists`: the caller's playlists plus all public ones.
- `getPlaylist(id)`: if public or owned by the caller.
- `createPlaylist(playlistId, name, songId)`: owner is the caller.
- `updatePlaylist(playlistId, name, comment, public, songIdToAdd, songIndexToRemove)`: owner only; indices are applied in order.
- `deletePlaylist(playlistId)`: owner only.
No `createUser` / `updateUser` / `deleteUser` endpoints: config/Vault is the source of truth.
---
## 8. Scanner
- Scan all configured folder roots recursively; accept `.mp3` (case-insensitive) only, log every skipped file.
- Tag-first: title, artist, album artist, album, track/disc, year, genre(s), duration, bitrate, embedded cover, lyrics.
- Fallback: parent folder as album, grandparent as artist; filename as title.
- Delta scan by `(file_mtime, size)`; unchanged files are skipped.
- Orphan cleanup: delete songs (and their state) whose files disappeared.
- Initial scan on startup only when the `songs` table is empty; otherwise rely on `startScan`.
- One scan at a time; a second `startScan` returns the current status.
- All writes in a single transaction per batch; streaming reads continue under WAL.
- Album/artist rows are recomputed after each scan (aggregated genre lists, sort names, cover references).
Cover extraction order: embedded APIC frame → `cover.jpg`/`folder.jpg`/`front.jpg` next to the file → album directory → none.
---
## 9. Deployment (k3s + Helm)
### 9.1 Image
- Multi-stage Dockerfile: Gradle build → `eclipse-temurin:21-jre-alpine` with `ffmpeg` installed.
- Runtime flags: `-XX:+UseSerialGC -Xmx256m` (adjustable via Helm).
- Non-root user (UID 10001), read-only root filesystem, `/tmp` as `emptyDir`, `/data` as PVC.
- Image: `gitea.tap-tap.win/tap-tap/audiomanager:<tag>` (registry paths must be lowercase).
- Gitea Actions builds and pushes on tags; `imagePullSecret` is provided by the cluster.
### 9.2 Helm chart
`charts/musicmanager` with values for:
- image repository/tag, `imagePullSecrets`
- `library` config (folders) rendered into a ConfigMap
- `users.existingSecret` + key (Vault-managed; never put passwords in values)
- ingress host `subsonic.tap-tap.win`, ingress class, TLS via cluster issuer
- `/data` PVC (size, storage class)
- `hostPath` mounts: `/mnt/media/music` and `/mnt/media/longform`, read-only
- no node pinning: every schedulable node must expose the same mounts (documented)
- resource requests/limits, security context, liveness/readiness probes (`/health`)
- optional `nodeSelector`/`tolerations` for future use
The server fails its readiness probe if a configured root path is missing or unreadable.
### 9.3 Operations
- Backups: documented manual copy of `DATA_DIR` (SQLite DB + covers) while the server is stopped, or via `sqlite3 .backup`.
- Logs to stdout (JSON optional), picked up by the cluster.
- Graceful shutdown: stop accepting requests, finish in-flight streams, close the DB pool.
---
## 10. Testing
- No audio files are committed to the repository. A local script generates tiny silent MP3 fixtures (and `.lrc` files) under `build/test-media`; integration tests skip if fixtures are absent.
- Ktor integration tests (`ktor-server-test-host`):
- token and password auth, `enc:` decoding, wrong credentials → code 40
- deterministic IDs and normalization
- user visibility: user A cannot see or stream user B's music; both see long-form
- `Range` request → `206 Partial Content` with exact bytes
- exact `size` / `duration` / `bitRate` / `suffix` / `contentType` fields
- browsing flow: `getArtists` → `getArtist` → `getAlbum` → `stream`
- `search3` with Unicode and wildcards
- scrobble/star/rating/bookmark isolation between users
- playlist ownership and public visibility
- transcoding: `maxBitRate` produces MP3, `timeOffset` seeks (skipped when `ffmpeg` is unavailable)
---
## 11. Implementation Order
Verify each step with `./gradlew test` and/or `curl`:
- [x] 1. **Setup & DB:** `build.gradle.kts`, `AppConfig.kt`, `LibraryConfig.kt`, `UsersConfig.kt`, `DatabaseFactory.kt` (WAL + Flyway), `Tables.kt`, `V1__init.sql`.
- [x] 2. **Scanner & covers:** `IdGenerator.kt`, `Mp3TagExtractor.kt`, `CoverArtService.kt`, `MediaScanner.kt` (delta + orphan cleanup + startup scan when empty).
- [x] 3. **Ktor core & auth:** `PartialContent`, content negotiation (`ignoreUnknownKeys`, `encodeDefaults`), `SubsonicAuth`, response wrapper, health endpoint.
- [ ] 4. **Browsing & media:** system, browsing, genre and media routes (`stream`, `download`, `getCoverArt`).
- [ ] 5. **Transcoding:** `FfmpegTranscoder.kt`, `maxBitRate`/`format`/`timeOffset`, extension declaration.
- [ ] 6. **Interaction & playlists:** `getAlbumList2`, `getRandomSongs`, `getStarred2`, `search3`, `scrobble`, `star`/`unstar`, `setRating`, bookmarks, playlists.
- [ ] 7. **Packaging:** multi-stage Dockerfile (with ffmpeg), Helm chart, Gitea Actions workflow.
- [ ] 8. **Tests:** integration suite as described above, plus a `curl` smoke script.
---
## 12. Open Items / Assumptions
1. **Gitea registry path:** image assumed to be `gitea.tap-tap.win/tap-tap/audiomanager` (lowercased repo `TapTap/AudioManager`).
2. **Vault layout:** the chart expects a secret with a `users.yaml` key. Exact Vault path/agent template is deployment-side and not part of the app.
3. **Per-user music roots:** Vault/config must contain one folder entry per user root (e.g. `/mnt/media/music/<user>`); there is no implicit path convention.
4. **hostPath without pinning:** requires `/mnt/media/music` and `/mnt/media/longform` to exist on every node that can schedule the pod. Add a `nodeSelector` later if that changes.
5. **Long-form modelling:** audiobooks/podcasts are albums/tracks in long-form folders; no RSS ingestion or chapter model.
6. **No XML:** clients that insist on XML are unsupported.
7. **No user-management API:** user changes are applied on restart after the Vault secret/config is updated.
8. **Backups** are manual and documented; no backup endpoint.