- getMusicDirectory for folder/album browsing - server-side play queues with savePlayQueue/getPlayQueue and index variants - jukebox status and internet radio compatibility responses - scripts/smoke.sh generates a tagged MP3 library, runs the server and verifies all endpoints clients use, including Range, download and covers
709 lines
31 KiB
Markdown
709 lines
31 KiB
Markdown
# MusicManager - Self-hosted OpenSubsonic Server (Kotlin / Ktor)
|
|
|
|
## How to use
|
|
|
|
This is the short version. The detailed design and endpoint reference follow in the sections below.
|
|
|
|
### 1. Prerequisites
|
|
|
|
- JVM 21 (or `nix-shell` from this repository, which provides JDK 21, Gradle, ffmpeg and sqlite)
|
|
- `ffmpeg` on `PATH` (used for transcoding; direct play works without it)
|
|
- An MP3 library on disk, laid out as `Artist/Album/Track.mp3` if tags are missing
|
|
|
|
### 2. Configure the library
|
|
|
|
Create the local config and media directories:
|
|
|
|
```bash
|
|
./gradlew setupLocal
|
|
```
|
|
|
|
This copies `config/library.example.yaml` to `config/library.yaml` (only when it does not exist), copies `config/users.example.yaml` to `config/users.yaml` and creates `media/music` plus `media/longform`. Both local config files are git-ignored, so `git pull` never touches them.
|
|
|
|
`config/library.yaml` defines every root the server indexes:
|
|
|
|
```yaml
|
|
library:
|
|
folders:
|
|
- id: 1
|
|
name: "Music"
|
|
type: music
|
|
path: ./media/music
|
|
- id: 10
|
|
name: "Audiobooks & Podcasts"
|
|
type: longform
|
|
path: ./media/longform
|
|
```
|
|
|
|
The example file already uses these relative paths, so no edit is needed for local development. Relative paths are resolved from the working directory (`./gradlew run` uses the project root). For deployment, replace them with absolute paths such as `/mnt/media/music/alice` (see Section 6.1).
|
|
|
|
- `type: music` roots are assigned per user; `type: longform` roots are shared with every user.
|
|
- `id` must be stable and unique (integers). It is part of song IDs, so do not renumber it after a scan.
|
|
- Paths must exist and be readable, otherwise the server refuses to start with a clear error naming the folder.
|
|
|
|
### 3. Configure users
|
|
|
|
`./gradlew setupLocal` already created it. To recreate it manually:
|
|
|
|
```bash
|
|
cp config/users.example.yaml config/users.yaml
|
|
```
|
|
|
|
```yaml
|
|
users:
|
|
- name: alice
|
|
password: "change-me"
|
|
admin: true
|
|
folders: [1]
|
|
```
|
|
|
|
- `folders` lists the *music* folder IDs the user may see; long-form folders are added automatically.
|
|
- At least one admin is needed to trigger scans.
|
|
- Passwords live only in this file and in memory. In production the file is provided by Vault (see Section 6.2); `config/users.yaml` is git-ignored.
|
|
|
|
### 4. Start the server
|
|
|
|
```bash
|
|
nix-shell # optional: provides the toolchain
|
|
./gradlew run
|
|
```
|
|
|
|
Or build and run the fat JAR:
|
|
|
|
```bash
|
|
./gradlew shadowJar
|
|
java -jar build/libs/musicmanager-0.1.0-all.jar
|
|
```
|
|
|
|
The server listens on `http://localhost:8080`. Override with `PORT`, `DATA_DIR`, `LIBRARY_CONFIG`, `USERS_FILE` and `FFMPEG_PATH` (see Section 6.3). The SQLite database and cover cache are written to `./data`.
|
|
|
|
On first start, the server scans automatically because the database is empty. After that, scans are manual.
|
|
|
|
### 5. Trigger a scan
|
|
|
|
Use an admin account. Token auth is what real clients use:
|
|
|
|
```bash
|
|
SALT=abc123
|
|
TOKEN=$(printf '%s%s' 'change-me' "$SALT" | md5sum | cut -d' ' -f1)
|
|
|
|
curl -s "http://localhost:8080/rest/startScan?u=alice&t=$TOKEN&s=$SALT&v=1.16.1&c=curl&f=json"
|
|
curl -s "http://localhost:8080/rest/getScanStatus?u=alice&t=$TOKEN&s=$SALT&v=1.16.1&c=curl&f=json"
|
|
```
|
|
|
|
For quick manual testing you can pass the password in plaintext instead: `&p=change-me` (use `p=enc:<hex>` if it contains special characters).
|
|
|
|
### 6. Connect a client (Symfonium)
|
|
|
|
1. Add a new **Subsonic** server.
|
|
2. URL: `http://<host>:8080` (or `https://subsonic.tap-tap.win` when deployed).
|
|
3. Username and password from `users.yaml`.
|
|
4. Let Symfonium sync. Music and long-form appear as separate **music folders** - select the one you want so the two content types never mix.
|
|
|
|
### 7. Verify it works
|
|
|
|
```bash
|
|
# health (no auth)
|
|
curl -s http://localhost:8080/health
|
|
|
|
# list the folders this user can see
|
|
curl -s "http://localhost:8080/rest/getMusicFolders?u=alice&t=$TOKEN&s=$SALT&v=1.16.1&c=curl&f=json"
|
|
|
|
# search and stream the first result
|
|
curl -s "http://localhost:8080/rest/search3?u=alice&t=$TOKEN&s=$SALT&v=1.16.1&c=curl&f=json&query=*&songCount=1"
|
|
curl -o track.mp3 "http://localhost:8080/rest/stream?u=alice&t=$TOKEN&s=$SALT&v=1.16.1&c=curl&f=json&id=<song-id>"
|
|
```
|
|
|
|
All responses are JSON (no XML). Errors are returned as HTTP 200 with a Subsonic `error.code`; HTTP status codes are reserved for transport-level problems.
|
|
|
|
### 8. Stop, back up, upgrade
|
|
|
|
- Stop the process gracefully (`Ctrl+C` or `SIGTERM`); in-flight streams are allowed to finish.
|
|
- Back up `DATA_DIR` (SQLite database + `covers/`) while stopped, or use `sqlite3 data/musicmanager.db ".backup backup.db"`.
|
|
- The schema is migrated automatically by Flyway on startup.
|
|
- Re-running a scan only reads changed files (delta by mtime + size); IDs stay stable across scans.
|
|
|
|
---
|
|
|
|
## 1. Goal & Context
|
|
|
|
Build a lightweight, self-hosted **OpenSubsonic-compatible music server** in **Kotlin** that replaces Plex for audio streaming and is optimised for the Android client **Symfonium**.
|
|
|
|
### Scope (MVP)
|
|
|
|
1. **Library indexing:** index MP3 files only for now (other extensions are skipped and logged). Tags come first, with the folder layout `Artist/Album/Track` as fallback. Data is persisted in SQLite.
|
|
2. **Symfonium sync:** implement the OpenSubsonic browsing endpoints so Symfonium can sync the full catalog into its offline-first database.
|
|
3. **Streaming & offline downloads:** serve audio files with HTTP `Range` (`206 Partial Content`), exact byte sizes and correct metadata so seeking and Symfonium's offline download manager work reliably.
|
|
4. **Two content types, never mixed:** *music* and *long-form* (audiobooks, podcasts, lectures) live in **separate music folders**. Clients see and select them independently.
|
|
5. **Multi-user:** users are defined outside the database (config + HashiCorp Vault). Music roots are assigned per user, long-form is shared. Stars, ratings, play counts, bookmarks and playlists are per user.
|
|
6. **Transcoding:** optional on-the-fly MP3 transcoding with `maxBitRate` and `timeOffset` support.
|
|
7. **Extensibility:** clean separation between scanner, database layer and HTTP API so a downloader and a discovery/recommendation engine can be added later.
|
|
|
|
### Explicit non-goals (MVP)
|
|
|
|
- Audio formats other than MP3 (FLAC/OGG/M4A/Opus are future work).
|
|
- XML responses (JSON only).
|
|
- Podcast RSS ingestion, chapters or audiobook-specific metadata.
|
|
- Upload, sharing, social features.
|
|
- User-management API (users come from config/Vault).
|
|
- TLS inside the app (terminated by the k3s ingress).
|
|
- Automatic filesystem watching (scans are manual or on boot when the DB is empty).
|
|
- Backups (documented manual procedure only).
|
|
|
|
### Roles
|
|
|
|
- **Admin:** may trigger scans.
|
|
- **User:** regular library access.
|
|
- All users share the same long-form roots; each user has their own music roots.
|
|
- Per-user state: play counts, last played, stars, ratings, bookmarks, playlists.
|
|
|
|
---
|
|
|
|
## 2. Tech Stack & Dependencies
|
|
|
|
- **Language & runtime:** Kotlin 2.x on JVM 21 (`eclipse-temurin:21-jre-alpine`)
|
|
- **Build:** Gradle Kotlin DSL (`build.gradle.kts`), ShadowJar fat JAR
|
|
- **Web:** Ktor Server (Netty)
|
|
- `ktor-server-core`, `ktor-server-netty`
|
|
- `ktor-server-content-negotiation` + `ktor-serialization-kotlinx-json`
|
|
- `ktor-server-partial-content` (HTTP `Range` / `206`)
|
|
- `ktor-server-status-pages`, `ktor-server-call-logging`, `ktor-server-auto-head-response`
|
|
- **Database:** SQLite in `WAL` mode + JetBrains Exposed (`exposed-core`, `exposed-dao`, `exposed-jdbc`, `exposed-java-time`) + `sqlite-jdbc` + HikariCP
|
|
- **Migrations:** Flyway (`flyway-core`) with versioned SQL under `src/main/resources/db/migration`
|
|
- **Audio metadata:** `net.jthink:jaudiotagger` (MP3/ID3 only for the MVP)
|
|
- **Covers:** `net.coobird:thumbnailator` for on-the-fly resizing
|
|
- **Transcoding:** `ffmpeg` binary in the runtime image (libmp3lame), invoked as a subprocess
|
|
- **Logging:** `logback-classic` + `kotlin-logging-jvm`
|
|
- **Config:** YAML (`library.yaml`, `users.yaml`) with env-var overrides for paths and secrets
|
|
- **Deployment:** multi-stage Dockerfile, Helm chart for k3s, Gitea Actions for image builds
|
|
|
|
---
|
|
|
|
## 3. Project Structure
|
|
|
|
```text
|
|
src/main/kotlin/win/taptap/musicmanager/
|
|
|-- Application.kt # Ktor entry point, plugins, graceful shutdown
|
|
|-- config/
|
|
| |-- AppConfig.kt # Port, data dir, ffmpeg path, env overrides
|
|
| |-- LibraryConfig.kt # Music folders and their type/path
|
|
| `-- UsersConfig.kt # Users, admins and folder assignments
|
|
|-- db/
|
|
| |-- DatabaseFactory.kt # SQLite init, PRAGMA journal_mode=WAL, Flyway
|
|
| |-- Tables.kt # Exposed table definitions
|
|
| |-- LibraryRepository.kt # Visibility-scoped browsing, feeds, genres, search
|
|
| |-- UserStateRepository.kt # Play counts, stars, ratings, bookmarks
|
|
| |-- PlaylistRepository.kt # Playlist CRUD and visibility
|
|
| |-- MediaRepository.kt # Scanner persistence and aggregate rebuilds
|
|
| `-- migrations/ # Flyway SQL (V1__init.sql, ...)
|
|
|-- scanner/
|
|
| |-- MediaScanner.kt # Recursive scan, delta updates, orphan cleanup
|
|
| |-- Mp3TagExtractor.kt # jaudiotagger wrapper (tags, path fallback, duration, bitrate)
|
|
| `-- CoverArtService.kt # Cover extraction from tags/folders, disk cache, resizing
|
|
|-- subsonic/
|
|
| |-- SubsonicAuth.kt # Token (u, t, s) and password (p, enc:) auth
|
|
| |-- SubsonicResponse.kt # JSON envelope & error codes
|
|
| |-- models/ # kotlinx.serialization DTOs
|
|
| | |-- SystemModels.kt
|
|
| | |-- BrowsingModels.kt
|
|
| | |-- MediaModels.kt
|
|
| | `-- PlaylistModels.kt
|
|
| `-- routes/ # /rest/{endpoint} and /rest/{endpoint}.view
|
|
| |-- SystemRoutes.kt # ping, getLicense, extensions, startScan, getScanStatus
|
|
| |-- BrowsingRoutes.kt # indexes, artists, album, song, search3, lists, genres
|
|
| |-- MediaRoutes.kt # stream, download, getCoverArt, lyrics
|
|
| |-- InteractionRoutes.kt # scrobble, star/unstar, setRating, getStarred2
|
|
| `-- PlaylistRoutes.kt # playlists and playlist songs
|
|
|-- transcoding/
|
|
| `-- FfmpegTranscoder.kt # MP3 transcoding with maxBitRate and timeOffset
|
|
`-- util/
|
|
`-- IdGenerator.kt # Deterministic SHA-256 IDs
|
|
```
|
|
|
|
---
|
|
|
|
## 4. Architecture Rules
|
|
|
|
### 4.1 Deterministic IDs (critical)
|
|
|
|
IDs must never change on a re-scan; otherwise Symfonium loses offline download state and playlist assignments.
|
|
|
|
- IDs use **24 hex characters** (96 bit).
|
|
- **Artist ID:** `"ar-" + sha256(normalizedAlbumArtist).take(24)`
|
|
- **Album ID:** `"al-" + sha256(normalizedAlbumArtist + "|" + normalizedAlbumTitle).take(24)`
|
|
- **Song ID:** `"sg-" + sha256(folderId + ":" + relativePathFromFolderRoot).take(24)`
|
|
- **Cover ID:** same as the album ID (`"al-..."`). Artist artwork is not served in the MVP.
|
|
|
|
**Normalization is frozen** and must never change without a migration:
|
|
|
|
1. Unicode NFC
|
|
2. Trim and collapse internal whitespace
|
|
3. Lowercase with `Locale.ROOT`
|
|
|
|
Song IDs are path-based: renaming or moving a file creates a new ID. This is accepted.
|
|
|
|
### 4.2 Album grouping via album artist
|
|
|
|
Albums are grouped by the `ALBUM_ARTIST` tag. Fallbacks: `ARTIST` tag, then parent folder name. This prevents albums with featuring artists from fragmenting.
|
|
|
|
- `albums` have one or more artists (`album_artists`), position 0 is the primary album artist.
|
|
- Songs additionally keep a display artist string and their full artist list (`song_artists`).
|
|
- `song.artist_id` in API responses means the **album artist**; track credits are exposed via `displayArtist` / `artists[]`.
|
|
|
|
### 4.3 User-scoped library visibility
|
|
|
|
- The catalog is global; users see only folders assigned to them.
|
|
- Every browsing and media query filters songs by the requesting user's visible folder IDs.
|
|
- Because album IDs are deterministic and roots can contain the same album for different users, an album may aggregate songs from several folders. Therefore:
|
|
- `getAlbum(id)` returns only songs from folders visible to the caller.
|
|
- An album/artist is returned only if it has at least one visible song.
|
|
- Album-level names, years and covers are global; per-user state is never global.
|
|
- `getMusicFolders` returns the caller's assigned music folders plus the shared long-form folders.
|
|
- Songs store `folder_id`; `music_folders` is synced from config on startup and referenced by foreign key.
|
|
- Long-form content uses the same artist/album/song model; separation is via folder type, not a separate API.
|
|
|
|
### 4.4 Exact file metadata for offline downloads
|
|
|
|
Every song object must include:
|
|
|
|
- `size: Long` - exact byte size
|
|
- `duration: Int` - seconds
|
|
- `bitRate: Int` - kbps
|
|
- `suffix: String` - `"mp3"`
|
|
- `contentType: String` - `"audio/mpeg"`
|
|
|
|
Symfonium verifies downloaded bytes against `size` after each offline download.
|
|
|
|
### 4.5 Routing conventions & `formPost`
|
|
|
|
- Register every endpoint for **both** `/{method}` and `/{method}.view`.
|
|
- Read parameters from `call.request.queryParameters` **and** `call.receiveParameters()` (POST `application/x-www-form-urlencoded`).
|
|
- Declare the `formPost` OpenSubsonic extension.
|
|
- Repeatable parameters (`songId`, `songIdToAdd`, `songIndexToRemove`) must be read as lists.
|
|
|
|
### 4.6 JSON envelope
|
|
|
|
Every response, including errors, uses `status: "ok"` or `status: "failed"` and HTTP 200:
|
|
|
|
```json
|
|
{
|
|
"subsonic-response": {
|
|
"status": "ok",
|
|
"version": "1.16.1",
|
|
"type": "musicmanager",
|
|
"serverVersion": "0.1.0",
|
|
"openSubsonic": true
|
|
}
|
|
}
|
|
```
|
|
|
|
Error codes: `0` generic, `10` missing parameter, `20` incompatible client, `30` incompatible server, `40` wrong credentials, `50` not authorized, `70` not found.
|
|
|
|
### 4.7 Transcoding rules
|
|
|
|
- `stream` accepts `maxBitRate` (kbps), `format` and `timeOffset` (seconds).
|
|
- Direct play when no transcode is requested or when the source bitrate is already `<= maxBitRate`.
|
|
- Transcoding target is always **MP3** (`libmp3lame`). A requested `format` other than `mp3` falls back to MP3.
|
|
- `timeOffset` is implemented with `ffmpeg -ss`, declared as the `transcodeOffset` OpenSubsonic extension.
|
|
- Transcoded responses are chunked and do **not** support byte `Range`. Seeking uses `timeOffset`.
|
|
- `download` always returns the original file, never a transcode.
|
|
- Direct-play responses support `Range` / `206` via the `PartialContent` plugin.
|
|
|
|
---
|
|
|
|
## 5. Database Schema (SQLite)
|
|
|
|
Identity and folder tables are synced from config at startup; passwords are never stored in SQLite.
|
|
|
|
```sql
|
|
CREATE TABLE users (
|
|
name TEXT PRIMARY KEY,
|
|
is_admin BOOLEAN NOT NULL DEFAULT 0,
|
|
updated_at TEXT NOT NULL
|
|
);
|
|
|
|
CREATE TABLE music_folders (
|
|
id INTEGER PRIMARY KEY,
|
|
name TEXT NOT NULL,
|
|
type TEXT NOT NULL CHECK (type IN ('music', 'longform')),
|
|
root_path TEXT NOT NULL
|
|
);
|
|
|
|
CREATE TABLE artists (
|
|
id TEXT PRIMARY KEY,
|
|
name TEXT NOT NULL,
|
|
sort_name TEXT NOT NULL,
|
|
cover_art TEXT,
|
|
search_text TEXT NOT NULL
|
|
);
|
|
|
|
CREATE TABLE albums (
|
|
id TEXT PRIMARY KEY,
|
|
name TEXT NOT NULL,
|
|
year INTEGER,
|
|
cover_art TEXT,
|
|
created_at TEXT NOT NULL,
|
|
search_text TEXT NOT NULL
|
|
);
|
|
|
|
CREATE TABLE album_artists (
|
|
album_id TEXT NOT NULL REFERENCES albums(id) ON DELETE CASCADE,
|
|
artist_id TEXT NOT NULL REFERENCES artists(id) ON DELETE CASCADE,
|
|
position INTEGER NOT NULL,
|
|
PRIMARY KEY (album_id, position)
|
|
);
|
|
|
|
CREATE TABLE songs (
|
|
id TEXT PRIMARY KEY,
|
|
album_id TEXT NOT NULL REFERENCES albums(id) ON DELETE CASCADE,
|
|
folder_id INTEGER NOT NULL REFERENCES music_folders(id),
|
|
title TEXT NOT NULL,
|
|
display_artist TEXT NOT NULL,
|
|
album_name TEXT NOT NULL,
|
|
track_number INTEGER,
|
|
disc_number INTEGER DEFAULT 1,
|
|
year INTEGER,
|
|
duration INTEGER NOT NULL,
|
|
bit_rate INTEGER NOT NULL,
|
|
size INTEGER NOT NULL,
|
|
suffix TEXT NOT NULL,
|
|
content_type TEXT NOT NULL,
|
|
relative_path TEXT NOT NULL,
|
|
file_mtime INTEGER NOT NULL,
|
|
created_at TEXT NOT NULL,
|
|
search_text TEXT NOT NULL,
|
|
UNIQUE (folder_id, relative_path)
|
|
);
|
|
|
|
CREATE TABLE song_artists (
|
|
song_id TEXT NOT NULL REFERENCES songs(id) ON DELETE CASCADE,
|
|
artist_id TEXT NOT NULL REFERENCES artists(id) ON DELETE CASCADE,
|
|
position INTEGER NOT NULL,
|
|
PRIMARY KEY (song_id, position)
|
|
);
|
|
|
|
-- Multi-value tags. Text keys keep the schema simple; uniqueness is enforced.
|
|
CREATE TABLE song_genres (
|
|
song_id TEXT NOT NULL REFERENCES songs(id) ON DELETE CASCADE,
|
|
genre TEXT NOT NULL,
|
|
PRIMARY KEY (song_id, genre)
|
|
);
|
|
|
|
CREATE TABLE album_genres (
|
|
album_id TEXT NOT NULL REFERENCES albums(id) ON DELETE CASCADE,
|
|
genre TEXT NOT NULL,
|
|
PRIMARY KEY (album_id, genre)
|
|
);
|
|
|
|
CREATE TABLE artist_genres (
|
|
artist_id TEXT NOT NULL REFERENCES artists(id) ON DELETE CASCADE,
|
|
genre TEXT NOT NULL,
|
|
PRIMARY KEY (artist_id, genre)
|
|
);
|
|
|
|
-- Per-user state
|
|
CREATE TABLE user_song_state (
|
|
user_name TEXT NOT NULL REFERENCES users(name) ON DELETE CASCADE,
|
|
song_id TEXT NOT NULL REFERENCES songs(id) ON DELETE CASCADE,
|
|
play_count INTEGER NOT NULL DEFAULT 0,
|
|
last_played_at TEXT,
|
|
starred_at TEXT,
|
|
rating INTEGER,
|
|
PRIMARY KEY (user_name, song_id)
|
|
);
|
|
|
|
CREATE TABLE user_stars (
|
|
user_name TEXT NOT NULL REFERENCES users(name) ON DELETE CASCADE,
|
|
item_type TEXT NOT NULL CHECK (item_type IN ('album', 'artist')),
|
|
item_id TEXT NOT NULL,
|
|
starred_at TEXT NOT NULL,
|
|
PRIMARY KEY (user_name, item_type, item_id)
|
|
);
|
|
|
|
CREATE TABLE bookmarks (
|
|
user_name TEXT NOT NULL REFERENCES users(name) ON DELETE CASCADE,
|
|
song_id TEXT NOT NULL REFERENCES songs(id) ON DELETE CASCADE,
|
|
position_ms INTEGER NOT NULL,
|
|
updated_at TEXT NOT NULL,
|
|
PRIMARY KEY (user_name, song_id)
|
|
);
|
|
|
|
CREATE TABLE playlists (
|
|
id TEXT PRIMARY KEY,
|
|
owner TEXT NOT NULL REFERENCES users(name) ON DELETE CASCADE,
|
|
name TEXT NOT NULL,
|
|
comment TEXT,
|
|
is_public BOOLEAN NOT NULL DEFAULT 1,
|
|
created_at TEXT NOT NULL,
|
|
changed_at TEXT NOT NULL
|
|
);
|
|
|
|
CREATE TABLE playlist_songs (
|
|
playlist_id TEXT NOT NULL REFERENCES playlists(id) ON DELETE CASCADE,
|
|
song_id TEXT NOT NULL REFERENCES songs(id) ON DELETE CASCADE,
|
|
position INTEGER NOT NULL,
|
|
PRIMARY KEY (playlist_id, position)
|
|
);
|
|
```
|
|
|
|
Required indexes:
|
|
|
|
- `albums(search_text)`, `artists(search_text)`, `songs(search_text)`
|
|
- `songs(album_id)`, `songs(folder_id)`, `songs(created_at)`
|
|
- `album_artists(artist_id)`, `song_artists(artist_id)`
|
|
- `user_song_state(user_name, starred_at)`, `user_stars(user_name, item_type)`
|
|
- `playlist_songs(song_id)`, `playlists(owner)`
|
|
|
|
All timestamps are ISO 8601 in UTC.
|
|
|
|
---
|
|
|
|
## 6. Configuration
|
|
|
|
### 6.1 `library.yaml` (ConfigMap)
|
|
|
|
```yaml
|
|
library:
|
|
folders:
|
|
- id: 1
|
|
name: "Alice - Music"
|
|
type: music
|
|
path: /mnt/media/music/alice
|
|
- id: 2
|
|
name: "Bob - Music"
|
|
type: music
|
|
path: /mnt/media/music/bob
|
|
- id: 10
|
|
name: "Audiobooks & Podcasts"
|
|
type: longform
|
|
path: /mnt/media/longform
|
|
```
|
|
|
|
### 6.2 `users.yaml` (Vault-managed Secret)
|
|
|
|
Mounted from a HashiCorp Vault secret and read at startup. Passwords exist only in that secret and in memory - never in SQLite, logs or the Helm chart.
|
|
|
|
```yaml
|
|
users:
|
|
- name: alice
|
|
password: "<from-vault>"
|
|
admin: true
|
|
folders: [1]
|
|
- name: bob
|
|
password: "<from-vault>"
|
|
admin: false
|
|
folders: [2]
|
|
```
|
|
|
|
- Long-form folders are shared automatically; users only list their music folders.
|
|
- Env override: `USERS_FILE` (default `./config/users.yaml`; in the container `/etc/musicmanager/users.yaml`).
|
|
- Env override: `LIBRARY_CONFIG` (default `./config/library.yaml`; in the container `/etc/musicmanager/library.yaml`).
|
|
- `users` and `music_folders` tables are reconciled on startup; removed users/folders are deleted with their per-user state or orphaned songs.
|
|
- If `users.yaml` is missing or empty, the server refuses to start (fail closed).
|
|
|
|
### 6.3 Environment variables
|
|
|
|
| Variable | Default | Purpose |
|
|
| --- | --- | --- |
|
|
| `PORT` | `8080` | HTTP listen port |
|
|
| `DATA_DIR` | `./data` | SQLite DB and cover cache (`/data` in Kubernetes) |
|
|
| `LIBRARY_CONFIG` | `./config/library.yaml` | Library folder definitions |
|
|
| `USERS_FILE` | `./config/users.yaml` | Vault-managed users file |
|
|
| `FFMPEG_PATH` | `ffmpeg` | ffmpeg binary used for transcoding |
|
|
|
|
### 6.4 Local development
|
|
|
|
See [How to use](#how-to-use) for the full setup. In short:
|
|
|
|
```bash
|
|
cp config/library.example.yaml config/library.yaml
|
|
cp config/users.example.yaml config/users.yaml
|
|
mkdir -p media/music media/longform
|
|
# edit the paths in config/library.yaml to point at ./media/*
|
|
nix-shell # optional: JDK 21, Gradle, ffmpeg and sqlite
|
|
./gradlew run
|
|
```
|
|
|
|
The database and covers are written to `./data`. `config/users.yaml` is git-ignored.
|
|
|
|
---
|
|
|
|
## 7. MVP Endpoint Specification
|
|
|
|
All endpoints live under `/rest/{endpoint}` and `/rest/{endpoint}.view`.
|
|
|
|
### 7.1 Authentication (`SubsonicAuth`)
|
|
|
|
For every `/rest/*` call (except an optional health check):
|
|
|
|
- `u` must match a configured user.
|
|
- Token auth: `t` and `s` present -> verify `t.equals(md5(password + s), ignoreCase = true)`.
|
|
- Password auth: `p` present -> if `p` starts with `enc:`, hex-decode it; compare to the password.
|
|
- Comparisons use constant-time equality.
|
|
- On failure: HTTP 200 with a Subsonic error (`code = 40`).
|
|
- The authenticated user is attached to the call and used for all per-user filtering and state.
|
|
|
|
### 7.2 System & scan
|
|
|
|
- `ping`: empty `subsonic-response` with `status: "ok"`.
|
|
- `getLicense`: `{"license": {"valid": true}}`.
|
|
- `getOpenSubsonicExtensions`: `[{formPost, [1]}, {songLyrics, [1]}, {transcodeOffset, [1]}]`.
|
|
- `startScan`: admin only; starts an async coroutine scan.
|
|
- `getScanStatus`: `{"scanStatus": {"scanning": Boolean, "count": Long}}`.
|
|
- `getUser(username)`: returns roles and visible folder IDs; users may only query themselves, admins may query anyone.
|
|
- `getUsers`: admin only; lists all configured users.
|
|
|
|
### 7.3 Browsing & search
|
|
|
|
- `getMusicFolders`: the caller's music folders plus shared long-form folders.
|
|
- `getIndexes`: alphabetically grouped artists visible to the caller (`index: [{name, artist[]}]`), including `lastModified` and `ignoredArticles`.
|
|
- `getArtists`: ID3-based alphabet structure (`artists: {ignoredArticles, index[]}`), filtered by visibility.
|
|
- `getArtist(id)`: artist details plus all visible albums.
|
|
- `getAlbum(id)`: album details plus visible songs, sorted by `discNumber`, `trackNumber`.
|
|
- `getSong(id)`: single song, only if visible to the caller.
|
|
- `search3(query, ...Count, ...Offset)`: case-insensitive substring search over normalized `search_text` using `LIKE` with escaped wildcards. Empty `query` returns paginated results; `query="*"` returns everything visible.
|
|
- `getGenres`: distinct genres from visible songs.
|
|
- `getSongsByGenre(genre, count, offset)`: paginated visible songs in a genre.
|
|
- `getStarred2`: the caller's starred artists, albums and songs for the home screen.
|
|
|
|
### 7.4 Home feeds
|
|
|
|
- `getAlbumList2(type, size, offset, fromYear, toYear, genre)`: `newest`, `recent`, `frequent`, `random`, `alphabeticalByName`, `alphabeticalByArtist`, `starred`, `byYear`, `byGenre`. All results are visibility- and user-scoped.
|
|
- `getRandomSongs(size, genre, fromYear, toYear)`: random visible songs.
|
|
|
|
### 7.5 Media & artwork
|
|
|
|
- `stream(id, maxBitRate, format, timeOffset)`:
|
|
- direct play via `call.respondFile` (PartialContent handles `Range`);
|
|
- otherwise stream ffmpeg MP3 output with chunked transfer;
|
|
- set `Content-Type` from the actual response.
|
|
- `download(id)`: original file with `Content-Disposition: attachment; filename*=UTF-8''...` (RFC 5987 for non-ASCII names).
|
|
- `getCoverArt(id, size)`: serve `DATA_DIR/covers/{albumId}.jpg`; if `size` is given, resize proportionally with Thumbnailator and cache as `{albumId}_{size}.jpg`. Validate `size` (`1..2048`).
|
|
- `getLyricsBySongId(id)`: read embedded ID3 lyrics or a sibling `.lrc` file; return the OpenSubsonic `lyricsList` format.
|
|
|
|
### 7.6 Interaction, bookmarks & playlists
|
|
|
|
- `scrobble(id, time, submission)`: per user; `submission=true` (default) increments `play_count` and sets `last_played_at`.
|
|
- `star(id, albumId, artistId)` / `unstar(...)`: per user.
|
|
- `setRating(id, rating)`: per user, 1-5; `0` clears.
|
|
- `getBookmarks` / `createBookmark(id, position)` / `deleteBookmark(id)`: per user; `position` is milliseconds.
|
|
- `getPlaylists`: the caller's playlists plus all public ones.
|
|
- `getPlaylist(id)`: if public or owned by the caller.
|
|
- `createPlaylist(playlistId, name, songId)`: owner is the caller.
|
|
- `updatePlaylist(playlistId, name, comment, public, songIdToAdd, songIndexToRemove)`: owner only; indices are applied in order.
|
|
- `deletePlaylist(playlistId)`: owner only.
|
|
- `getMusicDirectory(id)`: folder IDs return their albums as directories; album IDs return their songs. Used by folder-browsing clients.
|
|
- `savePlayQueue` / `getPlayQueue` (and the `*ByIndex` variants): per-user queue kept in memory.
|
|
- `jukeboxControl(action=status)` returns a stopped status; other jukebox actions report that jukebox is unsupported.
|
|
- `getInternetRadioStations` returns an empty list; radio mutation endpoints report that radio is unsupported.
|
|
|
|
No `createUser` / `updateUser` / `deleteUser` endpoints: config/Vault is the source of truth.
|
|
|
|
---
|
|
|
|
## 8. Scanner
|
|
|
|
- Scan all configured folder roots recursively; accept `.mp3` (case-insensitive) only, log every skipped file.
|
|
- Tag-first: title, artist, album artist, album, track/disc, year, genre(s), duration, bitrate, embedded cover, lyrics.
|
|
- Fallback: parent folder as album, grandparent as artist; filename as title.
|
|
- Delta scan by `(file_mtime, size)`; unchanged files are skipped.
|
|
- Orphan cleanup: delete songs (and their state) whose files disappeared.
|
|
- Initial scan on startup only when the `songs` table is empty; otherwise rely on `startScan`.
|
|
- One scan at a time; a second `startScan` returns the current status.
|
|
- All writes in a single transaction per batch; streaming reads continue under WAL.
|
|
- Album/artist rows are recomputed after each scan (aggregated genre lists, sort names, cover references).
|
|
|
|
Cover extraction order: embedded APIC frame -> `cover.jpg`/`folder.jpg`/`front.jpg` next to the file -> album directory -> none.
|
|
|
|
---
|
|
|
|
## 9. Deployment (k3s + Helm)
|
|
|
|
### 9.1 Image
|
|
|
|
- Multi-stage Dockerfile: Gradle build -> `eclipse-temurin:21-jre-alpine` with `ffmpeg` installed.
|
|
- Runtime flags: `-XX:+UseSerialGC -Xmx256m` (adjustable via Helm).
|
|
- Non-root user (UID 10001), read-only root filesystem, `/tmp` as `emptyDir`, `/data` as PVC.
|
|
- Image: `gitea.tap-tap.win/tap-tap/audiomanager:<tag>` (registry paths must be lowercase).
|
|
- Gitea Actions builds and pushes on tags; `imagePullSecret` is provided by the cluster.
|
|
|
|
### 9.2 Helm chart
|
|
|
|
`charts/musicmanager` with values for:
|
|
|
|
- image repository/tag, `imagePullSecrets`
|
|
- `library` config (folders) rendered into a ConfigMap
|
|
- `users.existingSecret` + key (Vault-managed; never put passwords in values)
|
|
- ingress host `subsonic.tap-tap.win`, ingress class, TLS via cluster issuer
|
|
- `/data` PVC (size, storage class)
|
|
- `hostPath` mounts: `/mnt/media/music` and `/mnt/media/longform`, read-only
|
|
- no node pinning: every schedulable node must expose the same mounts (documented)
|
|
- resource requests/limits, security context, liveness/readiness probes (`/health`)
|
|
- optional `nodeSelector`/`tolerations` for future use
|
|
|
|
The server fails its readiness probe if a configured root path is missing or unreadable.
|
|
|
|
### 9.3 Operations
|
|
|
|
- Backups: documented manual copy of `DATA_DIR` (SQLite DB + covers) while the server is stopped, or via `sqlite3 .backup`.
|
|
- Logs to stdout (JSON optional), picked up by the cluster.
|
|
- Graceful shutdown: stop accepting requests, finish in-flight streams, close the DB pool.
|
|
|
|
---
|
|
|
|
## 10. Testing
|
|
|
|
- No audio files are committed to the repository. A local script generates tiny silent MP3 fixtures (and `.lrc` files) under `build/test-media`; integration tests skip if fixtures are absent.
|
|
- Ktor integration tests (`ktor-server-test-host`):
|
|
- token and password auth, `enc:` decoding, wrong credentials -> code 40
|
|
- deterministic IDs and normalization
|
|
- user visibility: user A cannot see or stream user B's music; both see long-form
|
|
- `Range` request -> `206 Partial Content` with exact bytes
|
|
- exact `size` / `duration` / `bitRate` / `suffix` / `contentType` fields
|
|
- browsing flow: `getArtists` -> `getArtist` -> `getAlbum` -> `stream`
|
|
- `search3` with Unicode and wildcards
|
|
- scrobble/star/rating/bookmark isolation between users
|
|
- playlist ownership and public visibility
|
|
- music directory browsing and server-side play queues
|
|
- transcoding: `maxBitRate` produces MP3, `timeOffset` seeks (skipped when `ffmpeg` is unavailable)
|
|
|
|
### Compatibility smoke test
|
|
|
|
`scripts/smoke.sh` generates a tagged MP3 with ffmpeg, starts the real server against a temporary library, and checks every endpoint a Subsonic client uses (browsing, feeds, search, stars, ratings, bookmarks, playlists, queues, stream with Range, download and covers). It prints PASS/FAIL per endpoint and exits non-zero on the first failure summary:
|
|
|
|
```bash
|
|
nix-shell --run scripts/smoke.sh
|
|
# or, with ffmpeg/sqlite/curl installed:
|
|
scripts/smoke.sh
|
|
```
|
|
|
|
The work directory can be passed as the first argument; the server log is kept there for debugging.
|
|
|
|
---
|
|
|
|
## 11. Implementation Order
|
|
|
|
Verify each step with `./gradlew test` and/or `curl`:
|
|
|
|
- [x] 1. **Setup & DB:** `build.gradle.kts`, `AppConfig.kt`, `LibraryConfig.kt`, `UsersConfig.kt`, `DatabaseFactory.kt` (WAL + Flyway), `Tables.kt`, `V1__init.sql`.
|
|
- [x] 2. **Scanner & covers:** `IdGenerator.kt`, `Mp3TagExtractor.kt`, `CoverArtService.kt`, `MediaScanner.kt` (delta + orphan cleanup + startup scan when empty).
|
|
- [x] 3. **Ktor core & auth:** `PartialContent`, content negotiation (`ignoreUnknownKeys`, `encodeDefaults`), `SubsonicAuth`, response wrapper, health endpoint.
|
|
- [x] 4. **Browsing & media:** system, browsing, genre and media routes (`stream`, `download`, `getCoverArt`).
|
|
- [ ] 5. **Transcoding:** `FfmpegTranscoder.kt`, `maxBitRate`/`format`/`timeOffset`, extension declaration.
|
|
- [x] 6. **Interaction & playlists:** `getAlbumList2`, `getRandomSongs`, `getStarred2`, `search3`, `scrobble`, `star`/`unstar`, `setRating`, bookmarks, playlists.
|
|
- [ ] 7. **Packaging:** multi-stage Dockerfile (with ffmpeg), Helm chart, Gitea Actions workflow.
|
|
- [ ] 8. **Tests:** integration suite as described above, plus a `curl` smoke script.
|
|
|
|
---
|
|
|
|
## 12. Open Items / Assumptions
|
|
|
|
1. **Gitea registry path:** image assumed to be `gitea.tap-tap.win/tap-tap/audiomanager` (lowercased repo `TapTap/AudioManager`).
|
|
2. **Vault layout:** the chart expects a secret with a `users.yaml` key. Exact Vault path/agent template is deployment-side and not part of the app.
|
|
3. **Per-user music roots:** Vault/config must contain one folder entry per user root (e.g. `/mnt/media/music/<user>`); there is no implicit path convention.
|
|
4. **hostPath without pinning:** requires `/mnt/media/music` and `/mnt/media/longform` to exist on every node that can schedule the pod. Add a `nodeSelector` later if that changes.
|
|
5. **Long-form modelling:** audiobooks/podcasts are albums/tracks in long-form folders; no RSS ingestion or chapter model.
|
|
6. **No XML:** clients that insist on XML are unsupported.
|
|
7. **No user-management API:** user changes are applied on restart after the Vault secret/config is updated.
|
|
8. **Backups** are manual and documented; no backup endpoint.
|