feat: rebuild AudioManager around downloadable moods
CI / chart (push) Successful in 3s
CI / test (push) Successful in 2m2s
CI / chart-push (push) Successful in 4s
CI / image (push) Successful in 3m20s

- V8 wipe migration and the shared song store keyed by YouTube video id
- media store, indexer, downloader, recommendation top-up and refcounted GC
- mood songs plus the recommendation lifecycle and mixed radio pools
- new search, song and recommendation API; remove library/longform/seed routes
- chart PVC/media env, docs, Nix, README and an end-to-end smoke test
This commit is contained in:
TapTap committed 2026-11-15 21:37:39 +00:00
1 parent 50d8b6c580
commit 824eb36dd2
59 files changed
+3779 -5217

No files matched your search

+2 -2
View File
@@ -9,8 +9,8 @@ build/
.vscode/ .vscode/
# Local runtime data # Local runtime data
data/ /data/
media/ /media/
config/library.yaml config/library.yaml
config/users.yaml config/users.yaml
*.db *.db
+2 -1
View File
@@ -28,7 +28,8 @@ WORKDIR /app
COPY --from=build /src/build/libs/*-all.jar /app/musicmanager.jar COPY --from=build /src/build/libs/*-all.jar /app/musicmanager.jar
ENV DATA_DIR=/data \ ENV DATA_DIR=/data \
LIBRARY_CONFIG=/etc/musicmanager/library.yaml \ MEDIA_DIR=/data/media \
DOWNLOAD_DIR=/data/downloads \
USERS_FILE=/etc/musicmanager/users.yaml \ USERS_FILE=/etc/musicmanager/users.yaml \
FFMPEG_PATH=ffmpeg \ FFMPEG_PATH=ffmpeg \
YTDLP_PATH=yt-dlp \ YTDLP_PATH=yt-dlp \
+182 -284
View File
@@ -1,359 +1,257 @@
# AudioManager # AudioManager
A self-hosted, mood-first music server in Kotlin/Ktor, plus a Kotlin Multiplatform client. A self-hosted, mood-first music server. You create private moods, add tracks
from YouTube search or a pasted URL, and the server downloads them into a shared
media store. Recommendations are discovered and downloaded automatically so a
mood keeps playing without a mounted music library.
AudioManager replaces static playlists with **moods**: curated anchors plus a radio queue that is generated live by the server and learns from what you finish and skip. There is no like button and no OpenSubsonic layer — the native API and the native client are designed together around this model. - Server: Kotlin, Ktor, SQLite (Exposed + Flyway), `yt-dlp`, `ffmpeg`.
- Client: Compose Multiplatform (Android, desktop, Wasm).
- No uploads, no mounted library, no likes, no podcasts.
## Contents ## Contents
1. [Quick start](#quick-start) - [How it works](#how-it-works)
2. [Configuration](#configuration) - [Quick start](#quick-start)
3. [The mood model](#the-mood-model) - [Configuration](#configuration)
4. [Native API](#native-api) - [Native API](#native-api)
5. [Architecture](#architecture) - [Architecture](#architecture)
6. [Database](#database) - [Media store and GC](#media-store-and-gc)
7. [Scanner](#scanner) - [Database](#database)
8. [Deployment](#deployment) - [Deployment](#deployment)
9. [Client](#client) - [Client](#client)
10. [Testing](#testing) - [Testing](#testing)
11. [Roadmap](#roadmap)
--- ## How it works
- **Moods are private.** A user sees only their own moods. Songs are shared
server-wide and deduplicated by YouTube video id.
- **Songs are the collection.** A mood needs at least one song before it can
play. Removing a song only detaches it; the file is deleted when no mood,
recommendation slot or open playback session references it.
- **Recommendations are active slots.** A mood keeps `recommendationCount`
(0–30, default 10) recommendations in `downloading`, `ready` or `failed`
state. Hidden `new` candidates refill the slots automatically.
- **Add / Not interested are permanent.** Adding promotes a recommendation into
the collection. Rejecting is remembered per mood. Both free a slot and the
server refills it.
- **Playback mixes both pools.** Every queue slot rolls `collectionPercent`;
the rest comes from ready recommendations. An empty pool falls back to the
other one. Small pools replay least-recently-played tracks.
- **Downloads are automatic.** Recommendation downloads run in the background
up to the configured count even if the mood is never played. Manual adds have
higher priority.
## Quick start ## Quick start
Requirements: JDK 21, ffmpeg, SQLite (bundled through JDBC). With Nix, `nix-shell` provides the JDK, Gradle, ffmpeg, sqlite, Python, Helm, Kustomize and kubectl. Requirements: JDK 21, `ffmpeg`, `yt-dlp`, and (for the smoke test) `sqlite3`,
`curl`, `python3`. `nix-shell` provides everything.
```bash ```bash
# 1. Build # 1. Build and create local config
gradle shadowJar ./gradlew setupLocal
# edit config/users.yaml (a password is required)
# 2. Point the server at a library # 2. Run
mkdir -p data ./gradlew run
cat > config/library.yaml <<'YAML' # health: http://localhost:8080/health
library: # API: http://localhost:8080/api/v1
folders:
- id: 1
name: Music
type: music
path: /absolute/path/to/music
- id: 10
name: Long-form
type: longform
path: /absolute/path/to/longform
YAML
# 3. Define users (passwords stay outside SQLite)
cat > config/users.yaml <<'YAML'
users:
- name: alice
password: change-me
admin: true
folders: [1]
YAML
# 4. Run
DATA_DIR=./data LIBRARY_CONFIG=./config/library.yaml USERS_FILE=./config/users.yaml \
java -jar build/libs/musicmanager-*-all.jar
``` ```
The server exposes the native API under `/api/v1`; use the Kotlin Multiplatform client for browsing and playback. The first start scans the library automatically because the database is empty. Later scans are manual: The server creates `./data/media` (the media store), `./data/downloads`
(download temp files) and `./data/musicmanager.db`.
```bash Downloads require `yt-dlp` to be able to reach YouTube. If YouTube applies bot
curl -X POST -H "Authorization: Bearer $TOKEN" http://localhost:8080/api/v1/scan checks, search and downloads fail with `502 search_failed` / `failed`
``` recommendations; retry later or update `yt-dlp`.
## Configuration ## Configuration
### `library.yaml` Everything is environment variables:
```yaml | Variable | Default | Meaning |
library: |---|---|---|
folders: | `PORT` | `8080` | HTTP port |
- id: 1 | `DATA_DIR` | `./data` | Database, cover cache, default media root |
name: Music | `MEDIA_DIR` | `<DATA_DIR>/media` | Server-owned song store |
type: music | `DOWNLOAD_DIR` | `<DATA_DIR>/downloads` | yt-dlp temp/work directory |
path: /mnt/media/music | `USERS_FILE` | `./config/users.yaml` | User accounts |
- id: 10 | `FFMPEG_PATH` | `ffmpeg` | Transcoder binary |
name: Long-form | `YTDLP_PATH` | `yt-dlp` | Downloader / search binary |
type: longform
path: /mnt/media/longform
```
- `type` is either `music` or `longform`. They are never mixed in a mood.
- IDs are stable and referenced by songs; never reuse an ID for a different path.
- Long-form folders are shared with every user automatically.
### `users.yaml` ### `users.yaml`
```yaml ```yaml
users: users:
- name: alice - name: admin
password: change-me password: "change-me"
admin: true admin: true
folders: [1]
- name: bob
password: another-secret
admin: false
folders: [1]
``` ```
Passwords are read into memory at startup and never written to SQLite. In production this file is a Vault-managed Secret. Restart the deployment after changing users. Passwords live only in this file (use a Vault-managed Secret in Kubernetes).
`admin` is only needed for the maintenance scan endpoints. Old `folders:` keys
### Environment are accepted and ignored.
| Variable | Default | Meaning |
| --- | --- | --- |
| `PORT` | `8080` | HTTP port |
| `DATA_DIR` | `./data` | SQLite database and cover cache |
| `LIBRARY_CONFIG` | `./config/library.yaml` | Library folders |
| `USERS_FILE` | `./config/users.yaml` | Users and passwords |
| `FFMPEG_PATH` | `ffmpeg` | Transcoder binary |
| `YTDLP_PATH` | `yt-dlp` | `yt-dlp` binary used for discovery and downloads |
| `DOWNLOADS_ENABLED` | `true` | Download accepted recommendations that are not in the library |
| `DOWNLOAD_DIR` | `$DATA_DIR/downloads` | Temporary files for the download worker |
| `DOWNLOAD_FOLDER_ID` | first visible music folder | Folder that receives downloaded MP3s |
## The mood model
A mood is a private collection owned by one user:
- **Seeds** are permanent anchors. Every seed is always eligible for the radio.
- **Seed mix** (`seedPercent`, 0–100) chooses how much of each batch comes from the seeds. The rest is filled from the generated recommendation pool (genres, artists, text matches and the rest of the library). Every slot rolls the percentage independently; when one pool is exhausted the other fills in. The default is 50/50.
- **Genres / free-text hint** widen the candidate pool.
- **Target size** is the desired batch and session size, not a hard playlist length.
- **Radio** is generated by the server for each playback session. It mixes seeds, related genres and artists, text matches, and the rest of the owner's music. Tracks started, finished or skipped in the last 7 days are skipped while the library is large enough, and artists/albums are spaced across the batch.
- **Signals** come from playback: completed tracks gain weight, skips weigh three times as much. There are no likes.
- **Add to mood** is explicit. The client offers *add to current mood* and *choose another mood*, and the server turns the track into a manual seed.
- **Long-form is excluded** at seed validation and radio generation.
A playback session belongs to one mood and one user. The server remembers queued tracks so a batch never repeats within the session, and a client can resume after a restart.
## Native API ## Native API
All endpoints live under `/api/v1`. JSON in, JSON out, standard HTTP status codes. Errors use: All endpoints are under `/api/v1` and require a bearer token, except login.
Media URLs may use `?token=` because HTML audio and images cannot send headers.
```json
{ "error": { "code": "not_found", "message": "Mood not found" } }
```
### Authentication ### Authentication
```http | Method | Path | Purpose |
POST /api/v1/auth/login |---|---|---|
Content-Type: application/json | `POST` | `/auth/login` | `{username,password,deviceName,platform}` → token |
| `POST` | `/auth/logout` | Revoke the current token |
{ "username": "alice", "password": "change-me", "deviceName": "Pixel", "platform": "android" } | `GET` | `/me` | Current user |
``` | `GET` | `/auth/devices` | List device tokens |
| `DELETE` | `/auth/devices/{id}` | Revoke a device |
```json
{
"token": "…",
"expiresAt": "2027-05-13T…",
"user": { "name": "alice", "admin": true, "musicFolders": [1], "longformFolders": [10] }
}
```
Send `Authorization: Bearer <token>` on every request. Tokens are stored as SHA-256 hashes and expire after 180 days. Media URLs may also use `?token=` because HTML audio and `<img>` tags cannot set headers.
- `POST /auth/logout` revokes the current token.
- `GET /auth/devices` lists devices; `DELETE /auth/devices/{id}` revokes one.
- `GET /me` returns the user and folder IDs.
### Library sync
- `GET /library/songs?cursor=<id>&limit=500&scope=music|longform|all` — snapshot page, ordered by song ID. Loop while `nextCursor` is present.
- `GET /library/songs?since=<revision>` — delta with `songs` and `deletedSongIds`. Loop while `nextSince` is present.
- `GET /library/search?q=&limit=&scope=` — normalized substring search.
- `GET /songs/{id}/stream` — original MP3 with Range support. Optional `maxBitRate`, `timeOffset`, `format`, `download=true`.
- `GET /covers/{albumIdOrSongId}?size=320` — cached JPEG cover.
Song objects contain deterministic IDs (`sg-…`), album/artist names, duration, size, bitrate, genres, `folderId`, `relativePath`, and `updatedAt`, so a client can build an offline library without extra requests.
### Moods ### Moods
- `GET /moods` | Method | Path | Purpose |
- `POST /moods` — `{ name, description?, query?, genres[], targetSize, refreshHours, ytEnabled, seedSongIds[], seedPercent? }` |---|---|---|
- `seedPercent` is 0–100 (default 50); the mood DTO also reports `recommendationPercent = 100 - seedPercent`. | `GET` | `/moods` | List the user's moods |
- `GET /moods/{id}` — mood, seeds and recommendations. | `POST` | `/moods` | Create `{name,description,query,genres,collectionPercent,recommendationCount,refreshHours}` |
- `PATCH /moods/{id}` — partial settings update, including `seedPercent`; `clearDescription` / `clearQuery` clear fields. | `GET` | `/moods/{id}` | Mood detail: songs and active recommendations |
- `DELETE /moods/{id}` | `PATCH` | `/moods/{id}` | Update options |
- `POST /moods/{id}/seeds` — `{ songId }` | `DELETE` | `/moods/{id}` | Delete mood, detach songs and run GC |
- `DELETE /moods/{id}/seeds/{songId}` | `GET` | `/moods/{id}/preview` | Unpersisted radio preview |
- `GET /moods/{id}/preview?limit=20` — unpersisted radio preview. | `POST` | `/moods/{id}/songs` | Add `{songId}`, `{videoId}` or `{url}` |
- `GET /moods/{id}/recommendations` — YT Music candidates with state and matched library song. | `DELETE` | `/moods/{id}/songs/{songId}` | Detach a song |
- `POST /moods/{id}/recommendations/refresh` — run discovery now (bypasses `refreshHours`). | `GET` | `/moods/{id}/recommendations` | Active recommendation slots |
- `POST /moods/{id}/recommendations/{videoId}/accept` — accept; the library match becomes a seed immediately. | `POST` | `/moods/{id}/recommendations/refresh` | Force discovery + top-up |
- `POST /moods/{id}/recommendations/{videoId}/reject` — reject; the candidate is not offered again. | `POST` | `/moods/{id}/recommendations/{videoId}/add` | Add to the collection |
| `POST` | `/moods/{id}/recommendations/{videoId}/reject` | Hide permanently |
| `POST` | `/moods/{id}/recommendations/{videoId}/retry` | Retry a failed download |
Accepted recommendations without a library match are queued for download (`download.state` = `queued|downloading|done|failed` in the recommendation). The worker writes MP3s into `<music folder>/AudioManager Imports/<artist>/<title>/`, embeds the video thumbnail as cover art, runs a scan, matches the new song by its deterministic path, adds it as a seed and updates the recommendation. The nested layout keeps per-track covers from overwriting each other. Adding an already downloaded song attaches instantly (`200`). A new video is
queued for download (`202`, `downloading: true`) and is attached when the job
finishes.
### Radio sessions ### Search
- `POST /moods/{id}/sessions` — `{ limit?: 15 }` → session plus the first queue. | Method | Path | Purpose |
- `GET /sessions/{id}` — resume: session and queued tracks. |---|---|---|
- `POST /sessions/{id}/next` — `{ limit?, excludeSongIds[] }` → the next batch. | `GET` | `/search?q=...&limit=...` | YouTube search |
- `POST /sessions/{id}/events` — `{ events: [{ songId, type: "started"|"completed"|"skipped", positionSeconds? }] }`
- `POST /sessions/{id}/add-to-mood` — `{ songId, moodId? }`. Without `moodId`, the session's mood is used. This is the replacement for likes.
- `DELETE /sessions/{id}` — close.
### Long-form Results longer than 10 minutes (or shorter than 30 seconds) are hidden. Each
result reports `downloaded` and `songId` when the track is already in the store.
- `GET /library/songs?scope=longform` — the long-form library (shared with every user). ### Playback
- `GET /longform/progress` — resume points, newest first, with song data.
- `PUT /longform/progress/{songId}` — `{ positionSeconds, durationSeconds?, completed? }`.
Long-form folders are never valid mood seeds and never enter radio. | Method | Path | Purpose |
|---|---|---|
| `POST` | `/moods/{id}/sessions` | Start a queue (`400 empty_mood` without songs) |
| `GET` | `/sessions/{id}` | Resume a session |
| `POST` | `/sessions/{id}/next` | Fetch more tracks |
| `POST` | `/sessions/{id}/events` | Report `started`, `completed`, `skipped` |
| `POST` | `/sessions/{id}/add-to-mood` | Add the playing track to a mood |
| `DELETE` | `/sessions/{id}` | Close the session |
### System ### Media and maintenance
- `GET /scan` — `{ scanning, count, error }` | Method | Path | Purpose |
- `POST /scan` — admin only; 202 Accepted. |---|---|---|
- `GET /health` — unauthenticated liveness probe. | `GET` | `/songs/{id}/stream` | Range-aware stream; `maxBitRate`, `timeOffset`, `download` |
| `GET` | `/songs/{id}/cover?size=` | Embedded artwork |
| `GET` | `/scan` | Scan status |
| `POST` | `/scan` | Start a scan (admin only) |
## Architecture ## Architecture
```text ```
src/main/kotlin/win/taptap/musicmanager/ Ktor routes
|-- Application.kt # Ktor entry point, plugins, graceful shutdown └── MoodRepository ── moods, mood_songs, recommendation lifecycle
|-- config/ └── SongRepository ── shared song store keyed by video id
| |-- AppConfig.kt # Port, data dir, ffmpeg, env overrides └── RadioService ── mixes collection + ready recommendations
| |-- LibraryConfig.kt # Folder IDs, types and paths └── DownloadService── single worker: yt-dlp → media store → index → attach
| `-- UsersConfig.kt # Users, admins and folder assignments └── DiscoveryService── yt-dlp search → hidden `new` candidates
|-- discovery/ └── TopUpService ── keeps active slots at recommendationCount
| |-- YtSearch.kt # yt-dlp search and result parsing └── MediaScanner ── rebuilds the index from the media root
| `-- DiscoveryService.kt # Queries, matching, scoring, pruning └── MediaGcService ── refcounted file deletion
|-- download/
| |-- DownloadService.kt # Queue worker, import folder, scan, promotion
| `-- YtDlpDownloader.kt # MP3 download with embedded metadata
|-- api/
| |-- ApiRoutes.kt # /api/v1 wiring, auth and system endpoints
| |-- ApiModels.kt # Native DTOs
| |-- ApiSupport.kt # Token resolution, errors, folder scoping
| |-- LibraryApi.kt # Sync, search, stream, covers
| |-- MoodApi.kt # Mood CRUD, seeds, preview, recommendations
| |-- LongformApi.kt # Long-form resume points
| `-- SessionApi.kt # Radio sessions, events, add-to-mood
|-- radio/
| `-- RadioService.kt # Candidate pool, scoring, diversity, batches
|-- db/
| |-- DatabaseFactory.kt # SQLite WAL + Flyway
| |-- Tables.kt # Exposed definitions
| |-- ApiTokenRepository.kt # Hashed bearer tokens
| |-- RadioRepository.kt # Sessions, events, affinity
| |-- MoodRepository.kt # Moods, seeds, recommendations
| |-- LibraryRepository.kt # Scoped browsing, sync, search
| `-- MediaRepository.kt # Scanner persistence and tombstones
|-- scanner/ # Recursive MP3 scan, tags, covers
|-- transcoding/ # ffmpeg MP3 transcoding
`-- util/ # Deterministic IDs, sort names
``` ```
YT discovery runs in the background for stale moods (`refreshHours`) and on demand from the mood screen. Queries come from the mood text, seed artists and genres; results are deduplicated, filtered to music-length videos, matched to library songs by normalized title and artist, and scored. Undecided candidates not seen in a refresh are pruned; accepted and rejected decisions are permanent. Accepted tracks that are not in the library are downloaded by the worker into `<music>/AudioManager Imports`, scanned, matched and added as seeds automatically. Background loops: download worker (continuous), recommendation top-up (hourly),
media GC (hourly). A scan runs at startup.
The old OpenSubsonic layer (`/rest`, `subsonic/`, playlist materialization) was removed in 0.2.0. The server and the KMP client are the two halves of the same product; there is no compatibility layer to preserve and no server-rendered UI. ## Media store and GC
Layout:
```
<MEDIA_DIR>/<videoId>/<sanitized-title>.mp3
```
The first path segment is the YouTube video id, so the startup scan never
depends on tags. One row and one file exist per video, no matter how many moods
or users reference it. Covers are cached under `<DATA_DIR>/covers` keyed by song
id.
A file is deleted only when it has no references from `mood_songs`, an active
recommendation (`downloading`, `ready`, `added`, `failed`), or an open playback
session. GC runs after removals, after rejections, after mood deletion, and
hourly.
## Database ## Database
SQLite in WAL mode, migrations through Flyway: SQLite in WAL mode. Migrations are Flyway scripts under
`src/main/resources/db/migration`.
- `V1__init.sql` — users, folders, artists, albums, songs, user state. - **V8 is destructive.** It drops every table from the mounted-library design
- `V2__user_item_ratings.sql`, `V3__song_lyrics.sql`. (folders, artists, albums, songs, moods, seeds, sessions, longform progress,
- `V4__moods.sql` — the first playlist-based mood prototype. downloads) and creates the mood-first schema. `users` and `api_tokens` are
- `V5__native_api.sql` — native moods, tokens, playback sessions, affinity, tombstones; drops playlists; adds `songs.updated_at`. kept.
- There is no import from an old media directory. Back up externally and start
Key invariants: with an empty `MEDIA_DIR`.
- IDs are deterministic (`ar-`, `al-`, `sg-`, `mo-`) with frozen normalization (NFC, trimmed, collapsed whitespace, lowercase).
- Albums are grouped by album artist, not by track artist.
- Deleted songs leave tombstones so offline clients can remove them.
- Passwords never appear in the database.
## Scanner
- Indexes `.mp3` recursively; other extensions are skipped and logged.
- Tags first, `Artist/Album/Track` path fallback.
- Delta scan: unchanged file mtimes are skipped.
- Orphan cleanup writes song tombstones.
- The first start scans automatically when the songs table is empty.
- Manual scans come from `POST /api/v1/scan` or a client.
## Deployment ## Deployment
Images and charts are published by Gitea Actions to `gitea.tap-tap.win`. The Helm chart is in `charts/audiomanager`. It renders one PVC at `/data`:
```text ```
Image: gitea.tap-tap.win/taptap/audiomanager:<tag> /data/musicmanager.db
Chart: https://gitea.tap-tap.win/TapTap/Charts/raw/branch/main /data/media/<videoId>/<title>.mp3
/data/covers/
/data/downloads/
``` ```
The [Charts README](https://gitea.tap-tap.win/TapTap/Charts) always shows the No hostPath, no `library.yaml`, no init container. `fsGroup: 10001` makes the
newest published chart version; `scripts/push-chart.sh` regenerates that section PVC writable by the non-root container.
on every publish.
Kustomize example:
```yaml
helmCharts:
- name: audiomanager
repo: https://gitea.tap-tap.win/TapTap/Charts/raw/branch/main
releaseName: audiomanager
version: 0.2.4-sha.<commit>
namespace: audiomanager
valuesFile: values.yaml
```
The chart renders every resource into the namespace passed to Helm
(`--namespace` or `helmCharts[].namespace`) and refuses to render when that
namespace is missing or falls back to `default`, so a forgotten namespace cannot
deploy into the wrong place. Set `allowDefaultNamespace=true` to opt in to
`default`. `scripts/check-chart.sh` lints the chart and enforces this in CI.
The chart expects:
- a `users.yaml` Secret (Vault-managed) referenced by `users.existingSecret`;
- a writable music hostPath when automatic downloads are enabled. The chart runs a `prepare-imports` init container (`downloads.prepare`) that creates `<music>/AudioManager Imports` and gives it to UID 10001. If the host denies the write, jobs fail immediately with the underlying error instead of retrying;
- a `library.yaml` ConfigMap;
- a writable music hostPath for imports (or `downloads.enabled=false`) and a read-only long-form hostPath;
- a registry pull secret for the private image.
Ingress host: `subsonic.tap-tap.win` (the historical name is kept, but the server no longer speaks the Subsonic protocol).
```bash ```bash
kustomize build --enable-helm . | kubectl apply -f - kubectl create secret generic audiomanager-users \
kubectl -n audiomanager rollout status deployment/audiomanager --from-file=users.yaml=config/users.yaml
helm upgrade --install audiomanager charts/audiomanager \
--namespace audiomanager --create-namespace \
--set users.existingSecret=audiomanager-users \
--set image.tag=<sha>
``` ```
Flyway applies `V5__native_api.sql` automatically. Existing playlist-based moods are migrated to native moods; old playlists and user stars are dropped. `scripts/release.sh --deploy` builds/pushes the image and runs the same Helm
upgrade. CI builds `:latest` and `:sha-<commit>` on every push to `main`, and
publishes the packaged chart.
Before deploying the mood-first version over an old instance, wipe the old
database and media: V8 drops the content, but the old mounted files are no
longer referenced and should be removed by the operator.
## Client ## Client
The client lives in `TapTap/AudioManagerClient` and is built with Kotlin Multiplatform + Compose Multiplatform: Compose Multiplatform client: `/home/taptap/AudioManagerClient`.
- **Android**: Media3/ExoPlayer, background playback, offline downloads. - Android, desktop and Wasm from `commonMain`.
- **Desktop (JVM)**: shared Compose UI and a JVM audio backend. - Screens: Moods, Mood Detail, Search, Downloads, Settings.
- **Web (Wasm)**: the same Compose UI, HTML audio, tokenized stream URLs. - Android background playback and media notifications are kept.
The shared module owns the API client, auth store, JSON offline cache, mood radio queue, and platform player abstractions. Downloads are original MP3s from `/stream?download=true`. Web is streaming-only (the browser owns its cache).
## Testing ## Testing
```bash ```bash
gradle test # unit + API integration tests ./gradlew test # server unit/integration tests
GRADLE=gradle scripts/smoke.sh # end-to-end server smoke test ./scripts/smoke.sh # end-to-end with a fake yt-dlp
scripts/check-chart.sh # lint the chart and enforce the namespace rule scripts/check-chart.sh # lint + render the Helm chart
``` ```
The smoke test generates tagged MP3s, starts the server, and verifies login, sync, moods, radio, add-to-mood, streaming, transcoding, covers, scan and logout. The smoke test seeds a tiny media store, puts a deterministic fake `yt-dlp` on
`PATH`, and exercises auth, search filtering, adding songs, URL resolve,
## Roadmap downloads, recommendations, radio sessions, streaming, covers and logout.
- [x] MP3 scanner, covers, deterministic IDs, SQLite + Flyway.
- [x] Native bearer-auth API, sync deltas, tombstones.
- [x] Mood CRUD, seeds, server-driven radio, playback events, add-to-mood.
- [x] Kotlin Multiplatform client foundation (Android, desktop, web).
- [x] Offline downloads and offline playback across clients.
- [x] YT Music discovery through `yt-dlp`, recommendation list, accept/reject.
- [x] Downloader that writes accepted tracks into an import folder and rescans.
- [ ] Home-screen widgets, Android Auto, and richer radio tuning.
+10 -28
View File
@@ -69,44 +69,26 @@ tasks.test {
} }
/** /**
* Prepares a local development checkout: creates the media directories and copies * Prepares a local development checkout: creates the data/media directories and
* the example configs when they are missing. Existing files are never overwritten. * copies the example users file when it is missing. Existing files are never
* overwritten.
*/ */
val setupLocal by tasks.registering { val setupLocal by tasks.registering {
group = "application" group = "application"
description = "Create local media folders and copy example configs if missing." description = "Create local data folders and copy the example users config if missing."
doLast { doLast {
project.mkdir("media/music") project.mkdir("data/media")
project.mkdir("media/longform") project.mkdir("data/downloads")
listOf("library", "users").forEach { name -> val target = project.file("config/users.yaml")
val target = project.file("config/$name.yaml") val example = project.file("config/users.example.yaml")
val example = project.file("config/$name.example.yaml")
if (!target.exists()) { if (!target.exists()) {
target.parentFile.mkdirs() target.parentFile.mkdirs()
example.copyTo(target) example.copyTo(target)
println("Created config/$name.yaml from config/$name.example.yaml") println("Created config/users.yaml from config/users.example.yaml")
} else { } else {
println("Keeping existing config/$name.yaml") println("Keeping existing config/users.yaml")
}
}
val libraryConfig = project.file("config/library.yaml")
if (libraryConfig.exists()) {
val stale = libraryConfig.readText().lineSequence()
.map { it.trim() }
.filter { it.startsWith("path:") }
.map { it.removePrefix("path:").trim() }
.filter { it.startsWith("/mnt/media/") }
.toList()
if (stale.isNotEmpty()) {
logger.warn(
"config/library.yaml still points at /mnt/media paths that are not available " +
"locally: $stale. Replace it with the local example: " +
"cp config/library.example.yaml config/library.yaml",
)
}
} }
} }
} }
+1 -1
View File
@@ -2,5 +2,5 @@ apiVersion: v2
name: audiomanager name: audiomanager
description: Mood-first self-hosted music server description: Mood-first self-hosted music server
type: application type: application
version: 0.2.4 version: 0.3.0
appVersion: "0.1.0" appVersion: "0.1.0"
@@ -1,17 +0,0 @@
apiVersion: v1
kind: ConfigMap
metadata:
name: {{ include "audiomanager.fullname" . }}-library
namespace: {{ include "audiomanager.namespace" . }}
labels:
{{- include "audiomanager.labels" . | nindent 4 }}
data:
library.yaml: |
library:
folders:
{{- range .Values.library.folders }}
- id: {{ .id }}
name: {{ .name | quote }}
type: {{ .type }}
path: {{ .path | quote }}
{{- end }}
+4 -52
View File
@@ -16,8 +16,6 @@ spec:
metadata: metadata:
labels: labels:
{{- include "audiomanager.selectorLabels" . | nindent 8 }} {{- include "audiomanager.selectorLabels" . | nindent 8 }}
annotations:
checksum/library: {{ include (print $.Template.BasePath "/configmap-library.yaml") . | sha256sum }}
spec: spec:
{{- with .Values.imagePullSecrets }} {{- with .Values.imagePullSecrets }}
imagePullSecrets: imagePullSecrets:
@@ -25,26 +23,6 @@ spec:
{{- end }} {{- end }}
securityContext: securityContext:
{{- toYaml .Values.podSecurityContext | nindent 8 }} {{- toYaml .Values.podSecurityContext | nindent 8 }}
{{- if and .Values.downloads.enabled .Values.downloads.prepare }}
initContainers:
- name: prepare-imports
image: "{{ include "audiomanager.image" . }}"
imagePullPolicy: {{ .Values.image.pullPolicy }}
command:
- sh
- -c
- |
dir='{{ .Values.media.music.mountPath }}/AudioManager Imports'
mkdir -p "$dir" \
&& chown -R 10001:10001 "$dir" \
&& chmod -R u+rwX,g+rwX "$dir" \
|| echo "AudioManager: cannot prepare $dir; check the music hostPath permissions"
securityContext:
runAsUser: 0
volumeMounts:
- name: music
mountPath: {{ .Values.media.music.mountPath }}
{{- end }}
containers: containers:
- name: audiomanager - name: audiomanager
image: "{{ include "audiomanager.image" . }}" image: "{{ include "audiomanager.image" . }}"
@@ -56,22 +34,16 @@ spec:
value: "8080" value: "8080"
- name: DATA_DIR - name: DATA_DIR
value: /data value: /data
- name: LIBRARY_CONFIG - name: MEDIA_DIR
value: /etc/audiomanager/library.yaml value: /data/media
- name: DOWNLOAD_DIR
value: /data/downloads
- name: USERS_FILE - name: USERS_FILE
value: /etc/audiomanager/users.yaml value: /etc/audiomanager/users.yaml
- name: FFMPEG_PATH - name: FFMPEG_PATH
value: ffmpeg value: ffmpeg
- name: YTDLP_PATH - name: YTDLP_PATH
value: yt-dlp value: yt-dlp
- name: DOWNLOADS_ENABLED
value: {{ .Values.downloads.enabled | quote }}
- name: DOWNLOAD_DIR
value: /data/downloads
{{- with .Values.downloads.folderId }}
- name: DOWNLOAD_FOLDER_ID
value: {{ . | quote }}
{{- end }}
{{- with .Values.env }} {{- with .Values.env }}
{{- toYaml . | nindent 12 }} {{- toYaml . | nindent 12 }}
{{- end }} {{- end }}
@@ -93,10 +65,6 @@ spec:
resources: resources:
{{- toYaml .Values.resources | nindent 12 }} {{- toYaml .Values.resources | nindent 12 }}
volumeMounts: volumeMounts:
- name: library
mountPath: /etc/audiomanager/library.yaml
subPath: library.yaml
readOnly: true
- name: users - name: users
mountPath: /etc/audiomanager/users.yaml mountPath: /etc/audiomanager/users.yaml
subPath: {{ .Values.users.key }} subPath: {{ .Values.users.key }}
@@ -105,15 +73,7 @@ spec:
mountPath: /data mountPath: /data
- name: tmp - name: tmp
mountPath: /tmp mountPath: /tmp
- name: music
mountPath: {{ .Values.media.music.mountPath }}
- name: longform
mountPath: {{ .Values.media.longform.mountPath }}
readOnly: true
volumes: volumes:
- name: library
configMap:
name: {{ include "audiomanager.fullname" . }}-library
- name: users - name: users
secret: secret:
secretName: {{ required "users.existingSecret is required" .Values.users.existingSecret }} secretName: {{ required "users.existingSecret is required" .Values.users.existingSecret }}
@@ -126,14 +86,6 @@ spec:
{{- end }} {{- end }}
- name: tmp - name: tmp
emptyDir: {} emptyDir: {}
- name: music
hostPath:
path: {{ .Values.media.music.hostPath }}
type: Directory
- name: longform
hostPath:
path: {{ .Values.media.longform.hostPath }}
type: Directory
{{- with .Values.nodeSelector }} {{- with .Values.nodeSelector }}
nodeSelector: nodeSelector:
{{- toYaml . | nindent 8 }} {{- toYaml . | nindent 8 }}
+5 -36
View File
@@ -15,52 +15,21 @@ imagePullSecrets: []
nameOverride: "" nameOverride: ""
fullnameOverride: "" fullnameOverride: ""
# Rendered into a ConfigMap as /etc/audiomanager/library.yaml.
# Keep folder IDs stable: they are part of song IDs.
library:
folders:
- id: 1
name: "Music"
type: music
path: /mnt/media/music
- id: 10
name: "Audiobooks & Podcasts"
type: longform
path: /mnt/media/longform
# Vault-managed users.yaml. The secret must already exist and contain the key below. # Vault-managed users.yaml. The secret must already exist and contain the key below.
users: users:
existingSecret: audiomanager-users existingSecret: audiomanager-users
key: users.yaml key: users.yaml
# The media root, database, cover cache and download temp dir all live here.
data: data:
persistence: persistence:
enabled: true enabled: true
size: 5Gi # Songs are downloaded automatically, so plan for growth.
size: 20Gi
storageClass: "" storageClass: ""
accessModes: accessModes:
- ReadWriteOnce - ReadWriteOnce
# Automatic downloads for accepted recommendations. Files land in
# "<music>/AudioManager Imports" and are picked up by a scan, so the music
# hostPath must be writable by the pod.
downloads:
enabled: true
# Runs a root init container that creates "<music>/AudioManager Imports" and
# gives it to UID 10001. Disable only if the hostPath is already writable.
prepare: true
# Empty uses the first music folder visible to the user.
folderId: null
# hostPath mounts shared by every schedulable node.
media:
music:
hostPath: /mnt/media/music
mountPath: /mnt/media/music
longform:
hostPath: /mnt/media/longform
mountPath: /mnt/media/longform
service: service:
type: ClusterIP type: ClusterIP
port: 8080 port: 8080
@@ -82,8 +51,8 @@ resources:
cpu: 100m cpu: 100m
memory: 256Mi memory: 256Mi
limits: limits:
cpu: 1000m cpu: 2000m
memory: 512Mi memory: 1024Mi
podSecurityContext: podSecurityContext:
fsGroup: 10001 fsGroup: 10001
-21
View File
@@ -1,21 +0,0 @@
# Example library configuration.
#
# The paths below are relative to the working directory and are meant for local
# development: `./gradlew run` creates a `./media` tree after you run
# `mkdir -p media/music media/longform`.
#
# In Kubernetes the real file is mounted as a ConfigMap and should use absolute
# paths, e.g. /mnt/media/music/alice and /mnt/media/longform.
#
# Folder IDs are stable identifiers used by songs and users. Never reuse an ID
# for a different path.
library:
folders:
- id: 1
name: "Music"
type: music
path: ./media/music
- id: 10
name: "Audiobooks & Podcasts"
type: longform
path: ./media/longform
+3 -3
View File
@@ -1,10 +1,10 @@
# Example users configuration. In the cluster this file comes from a # Example users configuration. In the cluster this file comes from a
# Vault-managed Secret; never commit a real one. # Vault-managed Secret; never commit a real one.
# #
# Long-form folders are shared automatically, so users only list their music # Moods are strictly per-user and all music lives in the server-owned media
# folders here. # store, so there are no folder assignments. An old `folders:` key is accepted
# and ignored.
users: users:
- name: admin - name: admin
password: "change-me" password: "change-me"
admin: true admin: true
folders: [1]
+55
View File
@@ -600,3 +600,58 @@ Client:
- A recommendation that is `added` while downloading must not turn into a - A recommendation that is `added` while downloading must not turn into a
duplicate collection row. duplicate collection row.
- The migration is destructive by design; document it prominently. - The migration is destructive by design; document it prominently.
## Implementation status (2026-11-15)
Phases 1-8 are implemented in both repositories.
### Server
- `V8__mood_first.sql` drops the mounted-library schema and creates
`songs`, `moods`, `mood_songs`, `mood_recommendations`, `download_jobs`,
playback tables and `song_affinity`. Users and device tokens survive.
The `mood_recommendations` table has an additional `rank` column so
promotion follows the discovery score order.
- `SongRepository` dedupes by video id and is the only song table access.
- `MediaStore` owns `<MEDIA_DIR>/<videoId>/<title>.mp3`; `SongIndexer` is shared
by the scanner and the downloader.
- `MoodRepository` implements the song collection, hidden `new` candidates,
active slots and the add/reject/retry lifecycle.
- `DiscoveryService` + `TopUpService` implement collection-based discovery and
slot refills. `TopUpService` coalesces work behind a mutex and can be triggered
fire-and-forget from API handlers.
- `DownloadService` is a single global worker with `song` jobs outranking
`recommendation` jobs. It indexes immediately after moving the file.
- `MediaGcService` implements refcounted deletion (moods, active recommendations,
open sessions).
- The API surface is the one in section 4.8. Differences: search returns
`{"results": [...]}`, recommendation decisions are `add`/`reject`/`retry`, and
covers use `/songs/{id}/cover`. `/moods/{id}/songs` returns `202` with
`downloading: true` when a new video was queued.
- `AppConfig` is `port`, `dataDir`, `usersFile`, `ffmpegPath`, `ytDlpPath`,
`mediaDir`, `downloadDir`. `LibraryConfig` and the folder APIs are deleted.
- Background loops: download worker (continuous), recommendation top-up
(hourly), media GC (hourly), startup scan.
- Chart 0.3.0 uses one `/data` PVC, `MEDIA_DIR=/data/media`, no hostPath, no
library ConfigMap and no init container.
- Tests: 73 server tests, 48-step smoke test with a fake `yt-dlp`.
### Client
- Models, `ApiClient`, `AppCache` (version 2) and `AppStore` match the new API.
- Screens are `Moods | MoodDetail | Search | Downloads | Settings`.
`LibraryScreen`, longform UI, seed management and global shuffle are gone.
- Search supports a target-mood picker, downloaded badges and paste-a-URL.
- Mood detail shows songs, add music, recommendation states (Add / Not
interested / Retry), next batch and the new options.
- Playback, sleep timer, queue editing, Android media session and device
downloads are unchanged apart from song-keyed covers.
- Tests: 16 client tests; desktop, Android debug and Wasm compile.
### Known follow-ups
- Real-device Android background playback and desktop audio checks still need a
machine with an audio device.
- Automatic downloads are bounded by `recommendationCount` but can still consume
noticeable disk/bandwidth for unplayed moods.
- yt-dlp bot checks remain an operational risk; failed rows keep Retry.
+123 -82
View File
@@ -1,18 +1,19 @@
#!/usr/bin/env bash #!/usr/bin/env bash
# #
# End-to-end smoke test for the native mood-first API. # End-to-end smoke test for the mood-first API.
# #
# Generates a tiny tagged MP3 library, starts the server against it, and # Seeds a tiny media store, starts the server with a fake `yt-dlp` on PATH, and
# exercises auth, sync, moods, radio sessions, add-to-mood and streaming. # exercises auth, search, moods, downloads, recommendations, radio sessions,
# streaming and covers.
# #
# Requirements: ffmpeg, sqlite3, curl, python3, java. # Requirements: ffmpeg, sqlite3, curl, python3, java, bash.
# Usage: scripts/smoke.sh [work-dir] # Usage: scripts/smoke.sh [work-dir]
# #
set -euo pipefail set -euo pipefail
ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
GRADLE="${GRADLE:-./gradlew}" GRADLE="${GRADLE:-./gradlew}"
WORK="${1:-$(mktemp -d "${TMPDIR:-/tmp}/musicmanager-smoke.XXXXXX")}" WORK="${1:-$(mktemp -d "${TMPDIR:-/tmp}/audiomanager-smoke.XXXXXX")}"
PORT="${SMOKE_PORT:-18099}" PORT="${SMOKE_PORT:-18099}"
USER="smoke" USER="smoke"
PASS="smoke-pass" PASS="smoke-pass"
@@ -30,51 +31,72 @@ fail() { FAILED=$((FAILED + 1)); log "FAIL $1: $2"; }
json_field() { python3 -c 'import json,sys;d=json.load(sys.stdin);print(eval("d"+sys.argv[1]))' "$1"; } json_field() { python3 -c 'import json,sys;d=json.load(sys.stdin);print(eval("d"+sys.argv[1]))' "$1"; }
mkdir -p "$WORK/music/Smoke Artist/Smoke Album" "$WORK/music/Second Artist/Second Album" \ MEDIA="$WORK/data/media"
"$WORK/longform/Long Narrator/Long Book" "$WORK/data" mkdir -p "$MEDIA/vid-seed-1" "$MEDIA/vid-seed-2" "$WORK/bin" "$WORK/data"
log "Generated test library in $WORK" log "Seeding the media store in $MEDIA"
ffmpeg -hide_banner -loglevel error -y -f lavfi -i "color=c=red:s=64x64" -frames:v 1 "$WORK/cover.jpg"
ffmpeg -hide_banner -loglevel error -y \ ffmpeg -hide_banner -loglevel error -y \
-f lavfi -i "sine=frequency=440:duration=2" \ -f lavfi -i "sine=frequency=440:duration=2" -i "$WORK/cover.jpg" \
-metadata title="Smoke Song" -metadata artist="Smoke Artist" \ -map 0:a -map 1:v -c:v copy -c:a libmp3lame -b:a 192k -id3v2_version 3 \
-metadata album="Smoke Album" -metadata album_artist="Smoke Artist" \ -metadata title="Seed One" -metadata artist="Seed Artist" -metadata album="Seed Album" \
-metadata genre="Rock" -metadata track="1" -metadata date="2026" \ -metadata:s:v title="Album cover" -metadata:s:v comment="Cover (front)" \
-b:a 192k "$WORK/music/Smoke Artist/Smoke Album/01 - Smoke Song.mp3" "$MEDIA/vid-seed-1/Seed One.mp3"
ffmpeg -hide_banner -loglevel error -y \ ffmpeg -hide_banner -loglevel error -y \
-f lavfi -i "sine=frequency=550:duration=2" \ -f lavfi -i "sine=frequency=550:duration=2" \
-metadata title="Second Song" -metadata artist="Second Artist" \ -c:a libmp3lame -b:a 192k -id3v2_version 3 \
-metadata album="Second Album" -metadata album_artist="Second Artist" \ -metadata title="Seed Two" -metadata artist="Second Artist" -metadata album="Second Album" \
-metadata genre="Ambient" -metadata track="1" -metadata date="2026" \ "$MEDIA/vid-seed-2/Seed Two.mp3"
-b:a 192k "$WORK/music/Second Artist/Second Album/01 - Second Song.mp3"
ffmpeg -hide_banner -loglevel error -y \
-f lavfi -i "sine=frequency=660:duration=2" \
-metadata title="Chapter One" -metadata artist="Long Narrator" \
-metadata album="Long Book" -metadata album_artist="Long Narrator" \
-metadata genre="Audiobook" -metadata track="1" -metadata date="2026" \
-b:a 128k "$WORK/longform/Long Narrator/Long Book/01 - Chapter One.mp3"
ffmpeg -hide_banner -loglevel error -y \
-f lavfi -i "color=c=red:s=64x64" -frames:v 1 \
"$WORK/music/Smoke Artist/Smoke Album/cover.jpg"
cat > "$WORK/library.yaml" <<EOF # Deterministic stand-in for yt-dlp: search, URL resolve and download.
library: cat > "$WORK/bin/yt-dlp" <<'FAKE'
folders: #!/usr/bin/env bash
- id: 1 set -euo pipefail
name: "Music" out=""
type: music prev=""
path: $WORK/music for arg in "$@"; do
- id: 10 if [[ "$prev" == "-o" ]]; then out="$arg"; fi
name: "Long-form" prev="$arg"
type: longform done
path: $WORK/longform
EOF if [[ " $* " == *" --flat-playlist "* ]]; then
cat <<'JSON'
{"entries":[
{"id":"vid-seed-1","title":"Seed Artist - Seed One","duration":180,"channel":"Seed Artist"},
{"id":"vid-rec-a","title":"New Artist - Downloaded Track","duration":180,"channel":"New Artist","thumbnail":"https://example/a.jpg"},
{"id":"vid-rec-b","title":"Other Artist - Another Track","duration":200,"channel":"Other Artist"},
{"id":"vid-long","title":"Long Set","duration":900,"channel":"DJ"},
{"id":"vid-short","title":"Interlude","duration":10,"channel":"DJ"}
]}
JSON
exit 0
fi
if [[ " $* " == *" --dump-single-json "* ]]; then
url="${*: -1}"
id="${url##*=}"; id="${id##*/}"
printf '{"id":"%s","title":"URL Artist - URL Track","channel":"URL Artist","duration":200}' "$id"
exit 0
fi
url="${*: -1}"
id="${url##*=}"
dir="$(dirname "$out")"
mkdir -p "$dir"
ffmpeg -hide_banner -loglevel error -y \
-f lavfi -i "sine=frequency=700:duration=1" \
-c:a libmp3lame -b:a 128k -id3v2_version 3 \
-metadata title="Downloaded Track" -metadata artist="New Artist" \
"$dir/$id.mp3"
printf '%s\n' "$dir/$id.mp3"
FAKE
chmod +x "$WORK/bin/yt-dlp"
cat > "$WORK/users.yaml" <<EOF cat > "$WORK/users.yaml" <<EOF
users: users:
- name: $USER - name: $USER
password: "$PASS" password: "$PASS"
admin: true admin: true
folders: [1]
EOF EOF
log "Building fat JAR..." log "Building fat JAR..."
@@ -86,32 +108,30 @@ if [[ -z "$JAR" ]]; then
exit 1 exit 1
fi fi
DATA_DIR="$WORK/data" LIBRARY_CONFIG="$WORK/library.yaml" USERS_FILE="$WORK/users.yaml" PORT="$PORT" \ PATH="$WORK/bin:$PATH" DATA_DIR="$WORK/data" USERS_FILE="$WORK/users.yaml" PORT="$PORT" \
java -jar "$JAR" > "$WORK/server.log" 2>&1 & java -jar "$JAR" > "$WORK/server.log" 2>&1 &
SERVER_PID=$! SERVER_PID=$!
trap 'kill $SERVER_PID 2>/dev/null || true' EXIT trap 'kill $SERVER_PID 2>/dev/null || true' EXIT
log "Waiting for the server and initial scan..." log "Waiting for the server and the startup scan..."
SONGS=0 SONGS=0
for _ in $(seq 1 120); do for _ in $(seq 1 120); do
if curl -sf "$BASE/health" >/dev/null 2>&1; then if curl -sf "$BASE/health" >/dev/null 2>&1; then
SONGS="$(sqlite3 "$WORK/data/musicmanager.db" 'select count(*) from songs' 2>/dev/null || echo 0)" SONGS="$(sqlite3 "$WORK/data/musicmanager.db" 'select count(*) from songs' 2>/dev/null || echo 0)"
[[ "$SONGS" -ge 3 ]] && break [[ "$SONGS" -ge 2 ]] && break
fi fi
sleep 1 sleep 1
done done
if [[ "${SONGS:-0}" -lt 3 ]]; then if [[ "${SONGS:-0}" -lt 2 ]]; then
log "Server did not scan the test library; see $WORK/server.log" log "Server did not index the media store; see $WORK/server.log"
tail -20 "$WORK/server.log" || true tail -20 "$WORK/server.log" || true
exit 1 exit 1
fi fi
SONG_ID="$(sqlite3 "$WORK/data/musicmanager.db" "select id from songs where title='Smoke Song'")" SEED_ONE="$(sqlite3 "$WORK/data/musicmanager.db" "select id from songs where video_id='vid-seed-1'")"
SONG2_ID="$(sqlite3 "$WORK/data/musicmanager.db" "select id from songs where title='Second Song'")" SEED_TWO="$(sqlite3 "$WORK/data/musicmanager.db" "select id from songs where video_id='vid-seed-2'")"
LONG_ID="$(sqlite3 "$WORK/data/musicmanager.db" "select id from songs where title='Chapter One'")" log "Indexed: seed1=$SEED_ONE seed2=$SEED_TWO"
ALBUM_ID="$(sqlite3 "$WORK/data/musicmanager.db" "select album_id from songs where id='$SONG_ID'")"
log "Scanned: song=$SONG_ID second=$SONG2_ID longform=$LONG_ID"
# api <method> <path> [json-body] [token-override] # api <method> <path> [json-body] [token-override]
api() { api() {
@@ -146,6 +166,21 @@ expect_eq() {
fi fi
} }
# wait_for <seconds> <name> <python-condition> <path>
wait_for() {
local seconds="$1" name="$2" condition="$3" path="$4" body=""
for _ in $(seq 1 "$seconds"); do
api GET "$path"
if [[ "$HTTP_STATUS" == "200" ]] \
&& printf '%s' "$HTTP_BODY" | python3 -c "import json,sys;d=json.load(sys.stdin);sys.exit(0 if ($condition) else 1)" 2>/dev/null; then
pass "$name"
return 0
fi
sleep 1
done
fail "$name" "timed out; last body: ${HTTP_BODY:0:240}"
}
log "" log ""
log "Authentication" log "Authentication"
LOGIN_STATUS="$(curl -s -o "$WORK/login.json" -w '%{http_code}' -X POST \ LOGIN_STATUS="$(curl -s -o "$WORK/login.json" -w '%{http_code}' -X POST \
@@ -163,62 +198,68 @@ BAD_STATUS="$(curl -s -o /dev/null -w '%{http_code}' -X POST \
expect_eq "login rejects bad password" "401" "$BAD_STATUS" expect_eq "login rejects bad password" "401" "$BAD_STATUS"
UNAUTH_STATUS="$(curl -s -o /dev/null -w '%{http_code}' "$BASE/api/v1/moods")" UNAUTH_STATUS="$(curl -s -o /dev/null -w '%{http_code}' "$BASE/api/v1/moods")"
expect_eq "unauthenticated request rejected" "401" "$UNAUTH_STATUS" expect_eq "unauthenticated request rejected" "401" "$UNAUTH_STATUS"
expect "me" "200" "GET" "/me" expect "me" "200" "GET" "/me"
expect_eq "me name" "$USER" "$(printf '%s' "$HTTP_BODY" | json_field '["name"]')" expect_eq "me name" "$USER" "$(printf '%s' "$HTTP_BODY" | json_field '["name"]')"
log "" log ""
log "Library sync" log "Search"
expect "library snapshot (music)" "200" "GET" "/library/songs?scope=music" expect "youtube search" "200" "GET" "/search?q=anything"
MUSIC_COUNT="$(printf '%s' "$HTTP_BODY" | json_field '["songs"].__len__()' 2>/dev/null || echo 0)" expect_eq "search hides long and short tracks" "3" "$(printf '%s' "$HTTP_BODY" | json_field '["results"].__len__()')"
expect_eq "music snapshot has 2 songs" "2" "$MUSIC_COUNT" expect_eq "search marks the seeded video" "true" "$(printf '%s' "$HTTP_BODY" | python3 -c 'import json,sys;print(str(any(r["videoId"]=="vid-seed-1" and r["downloaded"] for r in json.load(sys.stdin)["results"])).lower())')"
expect_eq "snapshot contains smoke song" "1" "$(printf '%s' "$HTTP_BODY" | python3 -c 'import json,sys;print(sum(1 for s in json.load(sys.stdin)["songs"] if s["id"]=="'"$SONG_ID"'"))')"
expect "library snapshot (longform)" "200" "GET" "/library/songs?scope=longform"
expect_eq "longform snapshot has 1 song" "1" "$(printf '%s' "$HTTP_BODY" | json_field '["songs"].__len__()')"
expect "search" "200" "GET" "/library/search?q=smoke&scope=music"
expect_eq "search finds one song" "1" "$(printf '%s' "$HTTP_BODY" | json_field '["songs"].__len__()')"
log "" log ""
log "Moods" log "Moods"
expect "create mood" "201" "POST" "/moods" \ expect "create mood" "201" "POST" "/moods" \
"{\"name\":\"Smoke Mood\",\"genres\":[\"Rock\"],\"targetSize\":10,\"seedSongIds\":[\"$SONG_ID\"]}" '{"name":"Smoke Mood","genres":["Rock"],"collectionPercent":50,"recommendationCount":2}'
MOOD_ID="$(printf '%s' "$HTTP_BODY" | json_field '["mood"]["id"]' 2>/dev/null || true)" MOOD_ID="$(printf '%s' "$HTTP_BODY" | json_field '["mood"]["id"]' 2>/dev/null || true)"
if [[ -n "$MOOD_ID" ]]; then pass "mood id"; else fail "mood id" "${HTTP_BODY:0:200}"; fi if [[ -n "$MOOD_ID" ]]; then pass "mood id"; else fail "mood id" "${HTTP_BODY:0:200}"; fi
expect_eq "mood seed count" "1" "$(printf '%s' "$HTTP_BODY" | json_field '["mood"]["seedCount"]')" expect "empty mood cannot start a session" "400" "POST" "/moods/$MOOD_ID/sessions" '{"limit":5}'
expect_eq "empty mood error code" "empty_mood" "$(printf '%s' "$HTTP_BODY" | json_field '["error"]["code"]')"
expect "add an existing song" "200" "POST" "/moods/$MOOD_ID/songs" "{\"songId\":\"$SEED_ONE\"}"
expect_eq "song count" "1" "$(printf '%s' "$HTTP_BODY" | json_field '["detail"]["mood"]["songCount"]')"
expect "add a second song" "200" "POST" "/moods/$MOOD_ID/songs" "{\"songId\":\"$SEED_TWO\"}"
expect_eq "song count after second add" "2" "$(printf '%s' "$HTTP_BODY" | json_field '["detail"]["mood"]["songCount"]')"
expect "add by URL" "202" "POST" "/moods/$MOOD_ID/songs" '{"url":"https://youtu.be/vid-url"}'
wait_for 60 "URL download attaches to the mood" 'd["mood"]["songCount"] >= 3' "/moods/$MOOD_ID"
expect "list moods" "200" "GET" "/moods" expect "list moods" "200" "GET" "/moods"
expect_eq "list contains mood" "1" "$(printf '%s' "$HTTP_BODY" | python3 -c 'import json,sys;print(sum(1 for m in json.load(sys.stdin)["moods"] if m["id"]=="'"$MOOD_ID"'"))')" expect_eq "list contains mood" "1" "$(printf '%s' "$HTTP_BODY" | python3 -c 'import json,sys;print(sum(1 for m in json.load(sys.stdin)["moods"] if m["id"]=="'"$MOOD_ID"'"))')"
expect "get mood" "200" "GET" "/moods/$MOOD_ID"
expect "mood preview" "200" "GET" "/moods/$MOOD_ID/preview?limit=10" expect "mood preview" "200" "GET" "/moods/$MOOD_ID/preview?limit=10"
expect_eq "preview is not empty" "yes" "$(printf '%s' "$HTTP_BODY" | python3 -c 'import json,sys;print("yes" if json.load(sys.stdin)["songs"] else "no")')" expect_eq "preview is not empty" "yes" "$(printf '%s' "$HTTP_BODY" | python3 -c 'import json,sys;print("yes" if json.load(sys.stdin)["songs"] else "no")')"
expect "add seed" "200" "POST" "/moods/$MOOD_ID/seeds" "{\"songId\":\"$SONG2_ID\"}" expect "remove a song" "204" "DELETE" "/moods/$MOOD_ID/songs/$SEED_TWO"
expect_eq "seed count after add" "2" "$(printf '%s' "$HTTP_BODY" | json_field '["mood"]["seedCount"]')"
expect "remove seed" "204" "DELETE" "/moods/$MOOD_ID/seeds/$SONG2_ID" log ""
expect "get mood after seed removal" "200" "GET" "/moods/$MOOD_ID" log "Recommendations"
expect_eq "seed count after remove" "1" "$(printf '%s' "$HTTP_BODY" | json_field '["mood"]["seedCount"]')" expect "refresh recommendations" "200" "POST" "/moods/$MOOD_ID/recommendations/refresh"
REC_VIDEO="$(printf '%s' "$HTTP_BODY" | python3 -c 'import json,sys;r=json.load(sys.stdin)["recommendations"];print(r[0]["videoId"] if r else "")')"
wait_for 90 "recommendations download" 'd["mood"]["recommendationReady"] >= 1' "/moods/$MOOD_ID"
expect "add a recommendation" "200" "POST" "/moods/$MOOD_ID/recommendations/$REC_VIDEO/add"
SECOND_REC="$(printf '%s' "$HTTP_BODY" | python3 -c 'import json,sys;r=json.load(sys.stdin)["detail"]["recommendations"];print(r[0]["videoId"] if r else "")')"
if [[ -n "$SECOND_REC" ]]; then
expect "reject a recommendation" "200" "POST" "/moods/$MOOD_ID/recommendations/$SECOND_REC/reject"
expect_eq "rejected state" "rejected" "$(printf '%s' "$HTTP_BODY" | json_field '["recommendation"]["state"]')"
else
fail "reject a recommendation" "no second recommendation was available"
fi
log "" log ""
log "Radio sessions" log "Radio sessions"
expect "start session" "201" "POST" "/moods/$MOOD_ID/sessions" '{"limit":10}' expect "start session" "201" "POST" "/moods/$MOOD_ID/sessions" '{"limit":10}'
SESSION_ID="$(printf '%s' "$HTTP_BODY" | json_field '["id"]' 2>/dev/null || true)" SESSION_ID="$(printf '%s' "$HTTP_BODY" | json_field '["id"]' 2>/dev/null || true)"
if [[ -n "$SESSION_ID" ]]; then pass "session id"; else fail "session id" "${HTTP_BODY:0:200}"; fi if [[ -n "$SESSION_ID" ]]; then pass "session id"; else fail "session id" "${HTTP_BODY:0:200}"; fi
expect_eq "queue contains seed" "yes" "$(printf '%s' "$HTTP_BODY" | python3 -c 'import json,sys;print("yes" if any(s["id"]=="'"$SONG_ID"'" for s in json.load(sys.stdin)["queue"]) else "no")')" expect_eq "queue contains a collection song" "yes" "$(printf '%s' "$HTTP_BODY" | python3 -c 'import json,sys;print("yes" if json.load(sys.stdin)["queue"] else "no")')"
expect_eq "queue excludes longform" "yes" "$(printf '%s' "$HTTP_BODY" | python3 -c 'import json,sys;print("yes" if all(s["id"]!="'"$LONG_ID"'" for s in json.load(sys.stdin)["queue"]) else "no")')"
expect "next tracks" "200" "POST" "/sessions/$SESSION_ID/next" '{"limit":10}' expect "next tracks" "200" "POST" "/sessions/$SESSION_ID/next" '{"limit":10}'
expect "playback event" "200" "POST" "/sessions/$SESSION_ID/events" \ expect "playback event" "200" "POST" "/sessions/$SESSION_ID/events" \
"{\"events\":[{\"songId\":\"$SONG_ID\",\"type\":\"completed\",\"positionSeconds\":2}]}" "{\"events\":[{\"songId\":\"$SEED_ONE\",\"type\":\"completed\",\"positionSeconds\":2}]}"
expect_eq "accepted events" "1" "$(printf '%s' "$HTTP_BODY" | json_field '["accepted"]')" expect_eq "accepted events" "1" "$(printf '%s' "$HTTP_BODY" | json_field '["accepted"]')"
expect "add playing track to current mood" "200" "POST" "/sessions/$SESSION_ID/add-to-mood" \
"{\"songId\":\"$SONG2_ID\"}"
expect_eq "seed count after add-to-mood" "2" "$(printf '%s' "$HTTP_BODY" | json_field '["mood"]["seedCount"]')"
expect "resume session" "200" "GET" "/sessions/$SESSION_ID" expect "resume session" "200" "GET" "/sessions/$SESSION_ID"
expect_eq "session mood" "$MOOD_ID" "$(printf '%s' "$HTTP_BODY" | json_field '["moodId"]')"
expect "close session" "204" "DELETE" "/sessions/$SESSION_ID" expect "close session" "204" "DELETE" "/sessions/$SESSION_ID"
log "" log ""
log "Media" log "Media"
STREAM_STATUS="$(curl -s -o "$WORK/stream.mp3" -w '%{http_code}' \ STREAM_STATUS="$(curl -s -o "$WORK/stream.mp3" -w '%{http_code}' \
"$BASE/api/v1/songs/$SONG_ID/stream?token=$TOKEN")" "$BASE/api/v1/songs/$SEED_ONE/stream?token=$TOKEN")"
if [[ "$STREAM_STATUS" == "200" && -s "$WORK/stream.mp3" ]]; then if [[ "$STREAM_STATUS" == "200" && -s "$WORK/stream.mp3" ]]; then
pass "stream with query token" pass "stream with query token"
else else
@@ -226,11 +267,11 @@ else
fi fi
RANGE_STATUS="$(curl -s -o /dev/null -w '%{http_code}' -H 'Range: bytes=0-9' \ RANGE_STATUS="$(curl -s -o /dev/null -w '%{http_code}' -H 'Range: bytes=0-9' \
"$BASE/api/v1/songs/$SONG_ID/stream?token=$TOKEN")" "$BASE/api/v1/songs/$SEED_ONE/stream?token=$TOKEN")"
expect_eq "stream range" "206" "$RANGE_STATUS" expect_eq "stream range" "206" "$RANGE_STATUS"
TRANSCODE_STATUS="$(curl -s -D "$WORK/transcode.headers" -o "$WORK/transcoded.mp3" -w '%{http_code}' \ TRANSCODE_STATUS="$(curl -s -D "$WORK/transcode.headers" -o "$WORK/transcoded.mp3" -w '%{http_code}' \
"$BASE/api/v1/songs/$SONG_ID/stream?token=$TOKEN&maxBitRate=64")" "$BASE/api/v1/songs/$SEED_ONE/stream?token=$TOKEN&maxBitRate=64")"
if [[ "$TRANSCODE_STATUS" == "200" && -s "$WORK/transcoded.mp3" ]] \ if [[ "$TRANSCODE_STATUS" == "200" && -s "$WORK/transcoded.mp3" ]] \
&& grep -qi '^content-type: audio/mpeg' "$WORK/transcode.headers"; then && grep -qi '^content-type: audio/mpeg' "$WORK/transcode.headers"; then
pass "stream transcode" pass "stream transcode"
@@ -239,16 +280,16 @@ else
fi fi
DOWNLOAD_STATUS="$(curl -s -o "$WORK/download.mp3" -w '%{http_code}' \ DOWNLOAD_STATUS="$(curl -s -o "$WORK/download.mp3" -w '%{http_code}' \
"$BASE/api/v1/songs/$SONG_ID/stream?token=$TOKEN&download=true")" "$BASE/api/v1/songs/$SEED_ONE/stream?token=$TOKEN&download=true")"
if [[ "$DOWNLOAD_STATUS" == "200" && -s "$WORK/download.mp3" ]]; then if [[ "$DOWNLOAD_STATUS" == "200" && -s "$WORK/download.mp3" ]]; then
pass "download original" pass "download original"
else else
fail "download original" "HTTP $DOWNLOAD_STATUS" fail "download original" "HTTP $DOWNLOAD_STATUS"
fi fi
COVER_STATUS="$(curl -s -o "$WORK/cover.jpg" -w '%{http_code}' \ COVER_STATUS="$(curl -s -o "$WORK/cover-out.jpg" -w '%{http_code}' \
"$BASE/api/v1/covers/$ALBUM_ID?size=32&token=$TOKEN")" "$BASE/api/v1/songs/$SEED_ONE/cover?size=32&token=$TOKEN")"
if [[ "$COVER_STATUS" == "200" && -s "$WORK/cover.jpg" ]]; then if [[ "$COVER_STATUS" == "200" && -s "$WORK/cover-out.jpg" ]]; then
pass "cover art" pass "cover art"
else else
fail "cover art" "HTTP $COVER_STATUS" fail "cover art" "HTTP $COVER_STATUS"
+1 -1
View File
@@ -18,7 +18,7 @@ pkgs.mkShell {
JAVA_HOME = pkgs.jdk21.home; JAVA_HOME = pkgs.jdk21.home;
shellHook = '' shellHook = ''
echo "MusicManager dev shell (JDK 21, Gradle, ffmpeg, yt-dlp, sqlite, helm, kustomize)" echo "AudioManager dev shell (JDK 21, Gradle, ffmpeg, yt-dlp, sqlite, helm, kustomize)"
java -version java -version
echo "" echo ""
echo "Test: gradle test" echo "Test: gradle test"
@@ -20,7 +20,6 @@ import io.ktor.server.response.respondText
import io.ktor.server.routing.get import io.ktor.server.routing.get
import io.ktor.server.routing.routing import io.ktor.server.routing.routing
import java.nio.file.Files import java.nio.file.Files
import java.time.Instant
import java.util.concurrent.TimeUnit import java.util.concurrent.TimeUnit
import kotlinx.coroutines.CoroutineScope import kotlinx.coroutines.CoroutineScope
import kotlinx.coroutines.Dispatchers import kotlinx.coroutines.Dispatchers
@@ -33,87 +32,96 @@ import kotlinx.coroutines.launch
import kotlinx.coroutines.runBlocking import kotlinx.coroutines.runBlocking
import kotlinx.serialization.json.Json import kotlinx.serialization.json.Json
import org.slf4j.LoggerFactory import org.slf4j.LoggerFactory
import win.taptap.musicmanager.config.AppConfig
import win.taptap.musicmanager.config.LibraryConfig
import win.taptap.musicmanager.config.UsersConfig
import win.taptap.musicmanager.api.ApiErrorBody
import win.taptap.musicmanager.api.ApiError import win.taptap.musicmanager.api.ApiError
import win.taptap.musicmanager.api.ApiErrorBody
import win.taptap.musicmanager.api.ApiException import win.taptap.musicmanager.api.ApiException
import win.taptap.musicmanager.api.apiRoutes import win.taptap.musicmanager.api.apiRoutes
import win.taptap.musicmanager.config.AppConfig
import win.taptap.musicmanager.config.UsersConfig
import win.taptap.musicmanager.db.ConfigRepository import win.taptap.musicmanager.db.ConfigRepository
import win.taptap.musicmanager.db.DatabaseFactory import win.taptap.musicmanager.db.DatabaseFactory
import win.taptap.musicmanager.db.MediaRepository
import win.taptap.musicmanager.db.MoodRepository import win.taptap.musicmanager.db.MoodRepository
import win.taptap.musicmanager.discovery.DiscoveryService import win.taptap.musicmanager.discovery.DiscoveryService
import win.taptap.musicmanager.discovery.TopUpService
import win.taptap.musicmanager.discovery.YtDlpSearch import win.taptap.musicmanager.discovery.YtDlpSearch
import win.taptap.musicmanager.download.DownloadService import win.taptap.musicmanager.download.DownloadService
import win.taptap.musicmanager.download.YtDlpDownloader import win.taptap.musicmanager.download.YtDlpDownloader
import win.taptap.musicmanager.media.MediaGcService
import win.taptap.musicmanager.media.MediaStore
import win.taptap.musicmanager.radio.RadioService import win.taptap.musicmanager.radio.RadioService
import win.taptap.musicmanager.scanner.CoverArtService import win.taptap.musicmanager.scanner.CoverArtService
import win.taptap.musicmanager.scanner.MediaScanner import win.taptap.musicmanager.scanner.MediaScanner
import win.taptap.musicmanager.scanner.SongIndexer
import win.taptap.musicmanager.transcoding.FfmpegTranscoder import win.taptap.musicmanager.transcoding.FfmpegTranscoder
private val log = LoggerFactory.getLogger("win.taptap.musicmanager.Application") private val log = LoggerFactory.getLogger("win.taptap.musicmanager.Application")
fun main() { fun main() {
val config = AppConfig.fromEnv() val config = AppConfig.fromEnv()
val users = UsersConfig.load(config.usersFile)
val library = LibraryConfig.load(config.libraryConfigPath).also { it.validatePaths() } Files.createDirectories(config.dataDir)
val users = UsersConfig.load(config.usersFile).also { it.validateAgainst(library) } Files.createDirectories(config.mediaDir)
Files.createDirectories(config.downloadDir)
val db = DatabaseFactory.connect(config) val db = DatabaseFactory.connect(config)
try { try {
ConfigRepository.sync(library, users) ConfigRepository.sync(users)
val applicationScope = CoroutineScope(SupervisorJob() + Dispatchers.IO) val applicationScope = CoroutineScope(SupervisorJob() + Dispatchers.IO)
val coverArtService = CoverArtService(config.dataDir) val coverArtService = CoverArtService(config.dataDir)
val mediaStore = MediaStore(config.mediaDir)
val indexer = SongIndexer(mediaStore, coverArtService)
val gc = MediaGcService(mediaStore, coverArtService)
val transcoder = FfmpegTranscoder(config.ffmpegPath) val transcoder = FfmpegTranscoder(config.ffmpegPath)
val radioService = RadioService() val radioService = RadioService()
val discoveryService = DiscoveryService(YtDlpSearch(config.ytDlpPath)) val ytSearch = YtDlpSearch(config.ytDlpPath)
val scanner = MediaScanner( val discoveryService = DiscoveryService(ytSearch)
library = library, val topUpService = TopUpService(discoveryService, applicationScope)
coverArtService = coverArtService, val scanner = MediaScanner(mediaStore, indexer)
) val downloadService = DownloadService(
val downloadService = if (config.downloadsEnabled) {
Files.createDirectories(config.downloadDir)
DownloadService(
users = users,
library = library,
downloader = YtDlpDownloader(config.ytDlpPath), downloader = YtDlpDownloader(config.ytDlpPath),
mediaStore = mediaStore,
indexer = indexer,
gc = gc,
tempDir = config.downloadDir, tempDir = config.downloadDir,
startScan = { scanner.start(applicationScope) },
isScanning = { scanner.status().scanning },
folderId = config.downloadFolderId,
) )
} else {
null // The media root is the only source of songs, so index it at startup.
} log.info("Scanning media root {}", config.mediaDir)
if (MediaRepository.countSongs() == 0L) {
log.info("Song table is empty, starting initial library scan")
scanner.start(applicationScope) scanner.start(applicationScope)
applicationScope.launch(Dispatchers.IO) {
delay(TimeUnit.SECONDS.toMillis(5))
while (isActive) {
val worked = runCatching { downloadService.processOne() }
.onFailure { log.warn("Download worker error", it) }
.getOrDefault(false)
delay(if (worked) 1_000 else 15_000)
}
} }
// Accepted recommendations that are not in the library are fetched in // Refill recommendation slots and run due discovery passes.
// the background and promoted after a scan.
downloadService?.let { service ->
applicationScope.launch(Dispatchers.IO) { applicationScope.launch(Dispatchers.IO) {
delay(TimeUnit.MINUTES.toMillis(1)) delay(TimeUnit.MINUTES.toMillis(1))
while (isActive) { while (isActive) {
val worked = runCatching { service.processOne() } runCatching {
.onFailure { log.warn("Download worker error", it) } MoodRepository.all()
.getOrDefault(false) .filter { it.recommendationCount > 0 }
delay(if (worked) 1_000 else 20_000) .forEach { mood ->
topUpService.topUp(mood, force = false)
} }
}.onFailure { log.warn("Background recommendation top-up failed", it) }
delay(TimeUnit.HOURS.toMillis(1))
} }
} }
// Moods refresh their YT recommendations in the background; the client // Delete files that no mood, recommendation or open session references.
// can also force a pass from the mood screen.
applicationScope.launch(Dispatchers.IO) { applicationScope.launch(Dispatchers.IO) {
delay(TimeUnit.MINUTES.toMillis(2)) delay(TimeUnit.MINUTES.toMillis(10))
while (isActive) { while (isActive) {
runCatching { discoverStaleMoods(users, library, discoveryService) } runCatching { gc.collectGarbage() }
.onFailure { log.warn("Background discovery pass failed", it) } .onFailure { log.warn("Media GC failed", it) }
delay(TimeUnit.HOURS.toMillis(1)) delay(TimeUnit.HOURS.toMillis(1))
} }
} }
@@ -183,12 +191,15 @@ fun main() {
} }
apiRoutes( apiRoutes(
users = users, users = users,
library = library,
coverArtService = coverArtService, coverArtService = coverArtService,
transcoder = transcoder, transcoder = transcoder,
radioService = radioService, radioService = radioService,
discoveryService = discoveryService, discoveryService = discoveryService,
downloadsEnabled = config.downloadsEnabled, topUpService = topUpService,
ytSearch = ytSearch,
ytResolver = ytSearch,
mediaStore = mediaStore,
gc = gc,
startScan = { scanner.start(applicationScope) }, startScan = { scanner.start(applicationScope) },
scanStatus = { scanner.status() }, scanStatus = { scanner.status() },
) )
@@ -205,10 +216,10 @@ fun main() {
) )
log.info( log.info(
"Starting MusicManager on port {} with {} library folders and {} users", "Starting AudioManager on port {} with {} users and media root {}",
config.port, config.port,
library.folders.size,
users.users.size, users.users.size,
config.mediaDir,
) )
server.start(wait = true) server.start(wait = true)
} catch (t: Throwable) { } catch (t: Throwable) {
@@ -216,15 +227,3 @@ fun main() {
throw t throw t
} }
} }
private suspend fun discoverStaleMoods(
users: UsersConfig,
library: LibraryConfig,
discovery: DiscoveryService,
) {
MoodRepository.staleDiscovery(Instant.now(), 10).forEach { mood ->
val owner = users.byName(mood.owner) ?: return@forEach
runCatching { discovery.refresh(mood, owner, library) }
.onFailure { log.warn("Discovery for mood {} failed", mood.name, it) }
}
}
@@ -2,7 +2,8 @@ package win.taptap.musicmanager.api
import kotlinx.serialization.Serializable import kotlinx.serialization.Serializable
import win.taptap.musicmanager.db.MoodRecord import win.taptap.musicmanager.db.MoodRecord
import win.taptap.musicmanager.db.MoodSeedRecord import win.taptap.musicmanager.db.MoodSongRecord
import win.taptap.musicmanager.db.RecommendationRecord
import win.taptap.musicmanager.db.SongRecord import win.taptap.musicmanager.db.SongRecord
@Serializable @Serializable
@@ -23,8 +24,6 @@ data class LoginRequest(
data class UserDto( data class UserDto(
val name: String, val name: String,
val admin: Boolean, val admin: Boolean,
val musicFolders: List<Int>,
val longformFolders: List<Int>,
) )
@Serializable @Serializable
@@ -48,49 +47,40 @@ data class DeviceDto(
@Serializable @Serializable
data class SongDto( data class SongDto(
val id: String, val id: String,
val videoId: String,
val title: String, val title: String,
val artist: String, val artist: String,
val album: String, val album: String? = null,
val albumId: String,
val trackNumber: Int? = null,
val discNumber: Int = 1,
val year: Int? = null,
val duration: Int, val duration: Int,
val bitRate: Int, val bitRate: Int,
val size: Long, val size: Long,
val suffix: String, val suffix: String,
val genres: List<String> = emptyList(), val thumbnail: String? = null,
val folderId: Int,
val relativePath: String,
val updatedAt: String,
) )
fun SongRecord.toDto(): SongDto = SongDto( fun SongRecord.toDto(): SongDto = SongDto(
id = id, id = id,
videoId = videoId,
title = title, title = title,
artist = displayArtist, artist = artist,
album = albumName, album = album,
albumId = albumId,
trackNumber = trackNumber,
discNumber = discNumber,
year = year,
duration = duration, duration = duration,
bitRate = bitRate, bitRate = bitRate,
size = size, size = size,
suffix = suffix, suffix = suffix,
genres = genres, thumbnail = thumbnail,
folderId = folderId,
relativePath = relativePath,
updatedAt = updatedAt,
) )
@Serializable @Serializable
data class SeedDto( data class MoodSongDto(
val song: SongDto, val song: SongDto,
val origin: String, val origin: String,
val addedAt: String, val addedAt: String,
) )
fun MoodSongRecord.toDto(song: SongRecord): MoodSongDto =
MoodSongDto(song = song.toDto(), origin = origin, addedAt = addedAt)
@Serializable @Serializable
data class MoodDto( data class MoodDto(
val id: String, val id: String,
@@ -98,17 +88,18 @@ data class MoodDto(
val description: String? = null, val description: String? = null,
val query: String? = null, val query: String? = null,
val genres: List<String> = emptyList(), val genres: List<String> = emptyList(),
val targetSize: Int, val collectionPercent: Int,
val seedPercent: Int,
val recommendationPercent: Int, val recommendationPercent: Int,
val recommendationCount: Int,
val refreshHours: Int, val refreshHours: Int,
val ytEnabled: Boolean,
val lastDiscoveryAt: String? = null, val lastDiscoveryAt: String? = null,
val createdAt: String, val createdAt: String,
val changedAt: String, val changedAt: String,
val lastError: String? = null, val lastError: String? = null,
val seedCount: Int, val songCount: Int,
val newRecommendations: Int, val recommendationReady: Int,
val recommendationDownloading: Int,
val recommendationFailed: Int,
) )
fun MoodRecord.toDto(): MoodDto = MoodDto( fun MoodRecord.toDto(): MoodDto = MoodDto(
@@ -117,45 +108,93 @@ fun MoodRecord.toDto(): MoodDto = MoodDto(
description = description, description = description,
query = query, query = query,
genres = genres, genres = genres,
targetSize = targetSize, collectionPercent = collectionPercent,
seedPercent = seedPercent, recommendationPercent = 100 - collectionPercent,
recommendationPercent = 100 - seedPercent, recommendationCount = recommendationCount,
refreshHours = refreshHours, refreshHours = refreshHours,
ytEnabled = ytEnabled,
lastDiscoveryAt = lastDiscoveryAt, lastDiscoveryAt = lastDiscoveryAt,
createdAt = createdAt, createdAt = createdAt,
changedAt = changedAt, changedAt = changedAt,
lastError = lastError, lastError = lastError,
seedCount = seedCount, songCount = songCount,
newRecommendations = newRecommendations, recommendationReady = recommendationReady,
recommendationDownloading = recommendationDownloading,
recommendationFailed = recommendationFailed,
) )
@Serializable @Serializable
data class MoodDetailDto( data class MoodDetailDto(
val mood: MoodDto, val mood: MoodDto,
val seeds: List<SeedDto>, val songs: List<MoodSongDto> = emptyList(),
val recommendations: List<RecommendationDto> = emptyList(), val recommendations: List<RecommendationDto> = emptyList(),
) )
@Serializable @Serializable
data class RecommendationDecisionDto( data class RecommendationDto(
val detail: MoodDetailDto, val videoId: String,
val recommendation: RecommendationDto, val title: String,
val seedAdded: Boolean = false, val artist: String,
val album: String? = null,
val duration: Int? = null,
val thumbnail: String? = null,
val state: String,
val song: SongDto? = null,
val download: DownloadDto? = null,
val error: String? = null,
val firstSeen: String,
val lastSeen: String,
) )
fun RecommendationRecord.toDto(
song: SongRecord? = null,
download: DownloadDto? = null,
): RecommendationDto = RecommendationDto(
videoId = videoId,
title = title,
artist = artist,
album = album,
duration = duration,
thumbnail = thumbnail,
state = state,
song = song?.toDto(),
download = download,
error = error,
firstSeen = firstSeen,
lastSeen = lastSeen,
)
@Serializable
data class DownloadDto(
val state: String,
val attempts: Int,
val error: String? = null,
val updatedAt: String,
)
@Serializable
data class SearchResultDto(
val videoId: String,
val title: String,
val artist: String,
val album: String? = null,
val duration: Int? = null,
val thumbnail: String? = null,
val songId: String? = null,
val downloaded: Boolean = false,
)
@Serializable
data class SearchResponse(val results: List<SearchResultDto> = emptyList())
@Serializable @Serializable
data class CreateMoodRequest( data class CreateMoodRequest(
val name: String, val name: String,
val description: String? = null, val description: String? = null,
val query: String? = null, val query: String? = null,
val genres: List<String> = emptyList(), val genres: List<String> = emptyList(),
val targetSize: Int = 30, val collectionPercent: Int = 50,
val recommendationCount: Int = 10,
val refreshHours: Int = 24, val refreshHours: Int = 24,
val ytEnabled: Boolean = true,
val seedSongIds: List<String> = emptyList(),
/** Share of each batch that comes from the seed songs; the rest is recommended. */
val seedPercent: Int = 50,
) )
@Serializable @Serializable
@@ -166,14 +205,36 @@ data class UpdateMoodRequest(
val query: String? = null, val query: String? = null,
val clearQuery: Boolean = false, val clearQuery: Boolean = false,
val genres: List<String>? = null, val genres: List<String>? = null,
val targetSize: Int? = null, val collectionPercent: Int? = null,
val recommendationCount: Int? = null,
val refreshHours: Int? = null, val refreshHours: Int? = null,
val ytEnabled: Boolean? = null, )
val seedPercent: Int? = null,
/**
* Adding music accepts one of:
* - `songId`: an already downloaded song, attached instantly;
* - `videoId`: a YouTube id from search, downloaded if necessary;
* - `url`: a pasted watch URL resolved by yt-dlp.
*/
@Serializable
data class AddSongRequest(
val songId: String? = null,
val videoId: String? = null,
val url: String? = null,
) )
@Serializable @Serializable
data class AddSeedRequest(val songId: String, val origin: String = "manual") data class AddSongResponse(
val detail: MoodDetailDto,
val added: Boolean,
val downloading: Boolean = false,
)
@Serializable
data class RecommendationDecisionDto(
val detail: MoodDetailDto,
val recommendation: RecommendationDto,
)
@Serializable @Serializable
data class AddToMoodRequest(val songId: String, val moodId: String? = null) data class AddToMoodRequest(val songId: String, val moodId: String? = null)
@@ -185,16 +246,7 @@ data class AddToMoodResponse(
) )
@Serializable @Serializable
data class LibraryPage( data class TracksResponse(val songs: List<SongDto>)
val songs: List<SongDto>,
val deletedSongIds: List<String> = emptyList(),
val revision: String,
val nextCursor: String? = null,
val nextSince: String? = null,
)
@Serializable
data class SearchResponse(val songs: List<SongDto>)
@Serializable @Serializable
data class StartSessionRequest(val limit: Int? = null) data class StartSessionRequest(val limit: Int? = null)
@@ -234,49 +286,3 @@ data class ScanStatusDto(
val count: Long, val count: Long,
val error: String? = null, val error: String? = null,
) )
@Serializable
data class DownloadDto(
val state: String,
val attempts: Int,
val error: String? = null,
val updatedAt: String,
)
@Serializable
data class RecommendationDto(
val videoId: String,
val title: String,
val artist: String,
val album: String? = null,
val duration: Int? = null,
val thumbnail: String? = null,
val state: String,
val matchedSong: SongDto? = null,
val timesAdded: Int,
val firstSeen: String,
val lastSeen: String,
val download: DownloadDto? = null,
)
@Serializable
data class LongformProgressDto(
val song: SongDto,
val positionSeconds: Int,
val durationSeconds: Int,
val completed: Boolean,
val updatedAt: String,
)
@Serializable
data class LongformProgressResponse(val items: List<LongformProgressDto> = emptyList())
@Serializable
data class SaveLongformProgressRequest(
val positionSeconds: Int,
val durationSeconds: Int? = null,
val completed: Boolean = false,
)
fun seedDto(seed: MoodSeedRecord, song: SongRecord): SeedDto =
SeedDto(song = song.toDto(), origin = seed.origin, addedAt = seed.addedAt)
@@ -8,15 +8,14 @@ import io.ktor.server.routing.get
import io.ktor.server.routing.post import io.ktor.server.routing.post
import io.ktor.server.routing.route import io.ktor.server.routing.route
import java.security.MessageDigest import java.security.MessageDigest
import win.taptap.musicmanager.api.DeviceDto
import win.taptap.musicmanager.api.LoginRequest
import win.taptap.musicmanager.api.LoginResponse
import win.taptap.musicmanager.api.ScanStatusDto
import win.taptap.musicmanager.api.UserDto
import win.taptap.musicmanager.config.LibraryConfig
import win.taptap.musicmanager.config.UsersConfig import win.taptap.musicmanager.config.UsersConfig
import win.taptap.musicmanager.db.ApiTokenRepository import win.taptap.musicmanager.db.ApiTokenRepository
import win.taptap.musicmanager.discovery.DiscoveryService import win.taptap.musicmanager.discovery.DiscoveryService
import win.taptap.musicmanager.discovery.TopUpService
import win.taptap.musicmanager.discovery.YtResolver
import win.taptap.musicmanager.discovery.YtSearch
import win.taptap.musicmanager.media.MediaGcService
import win.taptap.musicmanager.media.MediaStore
import win.taptap.musicmanager.radio.RadioService import win.taptap.musicmanager.radio.RadioService
import win.taptap.musicmanager.scanner.CoverArtService import win.taptap.musicmanager.scanner.CoverArtService
import win.taptap.musicmanager.scanner.ScanStatus import win.taptap.musicmanager.scanner.ScanStatus
@@ -30,12 +29,15 @@ import win.taptap.musicmanager.transcoding.FfmpegTranscoder
*/ */
fun Route.apiRoutes( fun Route.apiRoutes(
users: UsersConfig, users: UsersConfig,
library: LibraryConfig,
coverArtService: CoverArtService, coverArtService: CoverArtService,
transcoder: FfmpegTranscoder, transcoder: FfmpegTranscoder,
radioService: RadioService, radioService: RadioService,
discoveryService: DiscoveryService, discoveryService: DiscoveryService,
downloadsEnabled: Boolean = true, topUpService: TopUpService,
ytSearch: YtSearch,
ytResolver: YtResolver,
mediaStore: MediaStore,
gc: MediaGcService,
startScan: () -> Boolean, startScan: () -> Boolean,
scanStatus: () -> ScanStatus, scanStatus: () -> ScanStatus,
) { ) {
@@ -66,23 +68,18 @@ fun Route.apiRoutes(
LoginResponse( LoginResponse(
token = issued.raw, token = issued.raw,
expiresAt = issued.record.expiresAt, expiresAt = issued.record.expiresAt,
user = user.toDto(library), user = user.toDto(),
), ),
) )
} }
get("/me") { get("/me") {
val user = call.requireApiUser(users) val user = call.requireApiUser(users)
call.respond(UserDto( call.respond(user.toDto())
name = user.name,
admin = user.admin,
musicFolders = user.musicFolderIds(library).sorted(),
longformFolders = user.longformFolderIds(library).sorted(),
))
} }
post("/auth/logout") { post("/auth/logout") {
val user = call.requireApiUser(users) call.requireApiUser(users)
ApiTokenRepository.revokeRaw(call.bearerToken().orEmpty()) ApiTokenRepository.revokeRaw(call.bearerToken().orEmpty())
call.respond(HttpStatusCode.NoContent) call.respond(HttpStatusCode.NoContent)
} }
@@ -127,10 +124,9 @@ fun Route.apiRoutes(
) )
} }
libraryRoutes(users, library, coverArtService, transcoder) songRoutes(users, ytSearch, mediaStore, coverArtService, transcoder)
longformRoutes(users, library) moodRoutes(users, radioService, discoveryService, topUpService, ytResolver, mediaStore, gc)
moodRoutes(users, library, radioService, discoveryService, downloadsEnabled) sessionRoutes(users, radioService, gc)
sessionRoutes(users, library, radioService)
} }
} }
@@ -6,14 +6,8 @@ import io.ktor.server.application.ApplicationCall
import io.ktor.server.plugins.BadRequestException import io.ktor.server.plugins.BadRequestException
import io.ktor.server.plugins.ContentTransformationException import io.ktor.server.plugins.ContentTransformationException
import io.ktor.server.request.receive import io.ktor.server.request.receive
import io.ktor.server.response.respond
import io.ktor.util.AttributeKey import io.ktor.util.AttributeKey
import kotlinx.serialization.SerializationException import kotlinx.serialization.SerializationException
import win.taptap.musicmanager.api.ApiError
import win.taptap.musicmanager.api.ApiErrorBody
import win.taptap.musicmanager.api.UserDto
import win.taptap.musicmanager.config.FolderType
import win.taptap.musicmanager.config.LibraryConfig
import win.taptap.musicmanager.config.UserAccount import win.taptap.musicmanager.config.UserAccount
import win.taptap.musicmanager.config.UsersConfig import win.taptap.musicmanager.config.UsersConfig
import win.taptap.musicmanager.db.ApiTokenRecord import win.taptap.musicmanager.db.ApiTokenRecord
@@ -29,26 +23,7 @@ class ApiException(
val ApiUserKey = AttributeKey<UserAccount>("musicmanager.api.user") val ApiUserKey = AttributeKey<UserAccount>("musicmanager.api.user")
val ApiTokenKey = AttributeKey<ApiTokenRecord>("musicmanager.api.token") val ApiTokenKey = AttributeKey<ApiTokenRecord>("musicmanager.api.token")
fun UserAccount.musicFolderIds(library: LibraryConfig): Set<Int> = fun UserAccount.toDto(): UserDto = UserDto(name = name, admin = admin)
library.folders
.asSequence()
.filter { it.type == FolderType.MUSIC && it.id in folders }
.map { it.id }
.toSet()
fun UserAccount.longformFolderIds(library: LibraryConfig): Set<Int> =
library.folders
.asSequence()
.filter { it.type == FolderType.LONGFORM }
.map { it.id }
.toSet()
fun UserAccount.toDto(library: LibraryConfig): UserDto = UserDto(
name = name,
admin = admin,
musicFolders = musicFolderIds(library).sorted(),
longformFolders = longformFolderIds(library).sorted(),
)
fun ApplicationCall.apiUser(): UserAccount = fun ApplicationCall.apiUser(): UserAccount =
attributes.getOrNull(ApiUserKey) ?: throw ApiException( attributes.getOrNull(ApiUserKey) ?: throw ApiException(
@@ -76,8 +51,7 @@ fun ApplicationCall.bearerToken(): String? {
} }
/** /**
* Resolves the token and stores the user on the call. Returns null after sending * Resolves the token and stores the user on the call.
* a 401, so handlers can simply `?: return@get`.
*/ */
suspend fun ApplicationCall.requireApiUser(users: UsersConfig): UserAccount { suspend fun ApplicationCall.requireApiUser(users: UsersConfig): UserAccount {
val raw = bearerToken() val raw = bearerToken()
@@ -1,206 +0,0 @@
package win.taptap.musicmanager.api
import io.ktor.http.ContentType
import io.ktor.http.HttpHeaders
import io.ktor.http.HttpStatusCode
import io.ktor.server.response.respond
import io.ktor.server.response.header
import io.ktor.server.response.respondFile
import io.ktor.server.response.respondOutputStream
import io.ktor.server.routing.Route
import io.ktor.server.routing.get
import java.net.URLEncoder
import java.time.Instant
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.withContext
import org.slf4j.LoggerFactory
import win.taptap.musicmanager.api.LibraryPage
import win.taptap.musicmanager.api.SearchResponse
import win.taptap.musicmanager.api.toDto
import win.taptap.musicmanager.config.LibraryConfig
import win.taptap.musicmanager.config.UsersConfig
import win.taptap.musicmanager.db.LibraryRepository
import win.taptap.musicmanager.scanner.CoverArtService
import win.taptap.musicmanager.transcoding.FfmpegTranscoder
private val log = LoggerFactory.getLogger("win.taptap.musicmanager.api.LibraryRoutes")
fun Route.libraryRoutes(
users: UsersConfig,
library: LibraryConfig,
coverArtService: CoverArtService,
transcoder: FfmpegTranscoder,
) {
/**
* Snapshot sync (`cursor`) and delta sync (`since=revision`) in one endpoint.
* Deletions are returned explicitly so offline libraries can drop them.
*/
get("/library/songs") {
val user = call.requireApiUser(users)
val limit = (call.request.queryParameters["limit"]?.toIntOrNull() ?: 500).coerceIn(1, 1000)
val scope = call.request.queryParameters["scope"] ?: "all"
val folders = user.scopeFolders(library, scope)
val now = Instant.now().toString()
val since = call.request.queryParameters["since"]?.takeIf { it.isNotBlank() }
val cursor = call.request.queryParameters["cursor"]?.takeIf { it.isNotBlank() }
// Snapshot pagination must keep the timestamp of the first page. Using
// each page's own time could hide a song inserted between pages, whose
// id sorts before the cursor but whose updatedAt predates the final
// revision (so no later delta would fetch it).
val revision = call.request.queryParameters["revision"]?.takeIf { it.isNotBlank() } ?: now
if (since != null) {
val changed = LibraryRepository.changedSongs(since, folders, limit, user.name)
val deletedRows = LibraryRepository.tombstones(since, limit)
val deleted = deletedRows.map { it.entityId }
val complete = changed.size < limit && deletedRows.size < limit
// Either list can fill a page. Continue from the older of the two
// page ends, so a full tombstone page is not lost when fewer songs
// changed. Re-reading a few rows is harmless; skipping them is not.
val nextSince = when {
changed.size == limit && deletedRows.size == limit ->
listOf(changed.last().updatedAt, deletedRows.last().deletedAt).minOrNull()
changed.size == limit -> changed.last().updatedAt
deletedRows.size == limit -> deletedRows.last().deletedAt
else -> null
}
call.respond(
LibraryPage(
songs = changed.map { it.toDto() },
deletedSongIds = deleted,
revision = if (complete) now else nextSince ?: now,
nextSince = nextSince,
),
)
return@get
}
val songs = LibraryRepository.syncSongs(folders, cursor.orEmpty(), limit, user.name)
call.respond(
LibraryPage(
songs = songs.map { it.toDto() },
revision = revision,
nextCursor = if (songs.size == limit) songs.last().id else null,
),
)
}
get("/library/search") {
val user = call.requireApiUser(users)
val query = call.request.queryParameters["q"]?.trim().orEmpty()
if (query.isEmpty()) {
call.respond(SearchResponse(emptyList()))
return@get
}
val limit = (call.request.queryParameters["limit"]?.toIntOrNull() ?: 50).coerceIn(1, 200)
val folders = user.scopeFolders(library, call.request.queryParameters["scope"] ?: "all")
val songs = LibraryRepository.searchSongs(query, limit, 0, folders, user.name)
call.respond(SearchResponse(songs.map { it.toDto() }))
}
get("/songs/{id}/stream") {
val user = call.requireApiUser(users)
val id = call.parameters["id"] ?: badRequest("Song id is required")
val parameters = call.request.queryParameters
val rawMaxBitRate = parameters["maxBitRate"]
val maxBitRate = rawMaxBitRate?.toIntOrNull()
if (rawMaxBitRate != null && (maxBitRate == null || maxBitRate < 0)) {
badRequest("Invalid maxBitRate")
}
val rawOffset = parameters["timeOffset"]
val offset = rawOffset?.toIntOrNull()
if (rawOffset != null && (offset == null || offset < 0)) {
badRequest("Invalid timeOffset")
}
val resolved = LibraryRepository.resolveSong(
id,
user.visibleFolderIds(library),
user.name,
library,
) ?: notFound("Song")
if (!transcoder.shouldTranscode(
resolved.song.bitRate,
maxBitRate,
parameters["format"],
offset ?: 0,
)
) {
call.response.header(HttpHeaders.ContentType, resolved.song.contentType)
if (parameters["download"] == "true") {
call.response.header(
HttpHeaders.ContentDisposition,
contentDisposition(resolved.file.fileName.toString()),
)
}
call.respondFile(resolved.file.toFile())
return@get
}
val process = try {
transcoder.start(resolved.file, maxBitRate, offset ?: 0)
} catch (e: Exception) {
log.error("Failed to start ffmpeg for {}", resolved.file, e)
throw ApiException(HttpStatusCode.InternalServerError, "transcode_failed", "Transcoding failed")
}
try {
call.response.headers.append(HttpHeaders.CacheControl, "no-store")
call.respondOutputStream(ContentType.parse("audio/mpeg")) {
val output = this
withContext(Dispatchers.IO) {
process.inputStream.use { input -> input.copyTo(output) }
}
}
} finally {
transcoder.stop(process)
}
}
get("/covers/{id}") {
val user = call.requireApiUser(users)
val id = call.parameters["id"] ?: badRequest("Cover id is required")
val rawSize = call.request.queryParameters["size"]
val size = rawSize?.toIntOrNull()
if (rawSize != null && (size == null || size !in 1..2048)) {
badRequest("Cover size must be between 1 and 2048")
}
val folders = user.visibleFolderIds(library)
val albumId = when {
id.startsWith("sg-") -> LibraryRepository.albumIdForSong(id, folders)
LibraryRepository.album(id, folders, user.name) != null -> id
else -> null
} ?: notFound("Cover")
val cover = if (size == null) {
coverArtService.coverFile(albumId)
} else {
coverArtService.resizedCover(albumId, size)
} ?: notFound("Cover")
call.respondFile(cover.toFile())
}
}
private fun win.taptap.musicmanager.config.UserAccount.scopeFolders(
library: LibraryConfig,
scope: String,
): Set<Int> = when (scope) {
"music" -> musicFolderIds(library)
"longform" -> longformFolderIds(library)
else -> visibleFolderIds(library)
}
private fun contentDisposition(fileName: String): String {
val encoded = URLEncoder.encode(fileName, Charsets.UTF_8).replace("+", "%20")
val ascii = fileName.map { character ->
if (character.code in 32..126 && character != '"' && character != '\\') character else '_'
}.joinToString("")
return "attachment; filename=\"$ascii\"; filename*=UTF-8''$encoded"
}
@@ -1,70 +0,0 @@
package win.taptap.musicmanager.api
import io.ktor.http.HttpStatusCode
import io.ktor.server.response.respond
import io.ktor.server.routing.Route
import io.ktor.server.routing.get
import io.ktor.server.routing.put
import win.taptap.musicmanager.api.LongformProgressDto
import win.taptap.musicmanager.api.LongformProgressResponse
import win.taptap.musicmanager.api.SaveLongformProgressRequest
import win.taptap.musicmanager.api.toDto
import win.taptap.musicmanager.config.LibraryConfig
import win.taptap.musicmanager.config.UsersConfig
import win.taptap.musicmanager.db.LibraryRepository
import win.taptap.musicmanager.db.LongformRepository
/**
* Resume points for audiobooks and podcasts. The library itself is synced
* through `/library/songs?scope=longform`; this only tracks where the user was.
*/
fun Route.longformRoutes(
users: UsersConfig,
library: LibraryConfig,
) {
get("/longform/progress") {
val user = call.requireApiUser(users)
val folders = user.longformFolderIds(library)
val records = LongformRepository.forUser(user.name)
val songs = LibraryRepository
.songsByIds(records.map { it.songId }, folders, user.name)
.associateBy { it.id }
call.respond(
LongformProgressResponse(
items = records.mapNotNull { record ->
songs[record.songId]?.let { song ->
LongformProgressDto(
song = song.toDto(),
positionSeconds = record.positionSeconds,
durationSeconds = record.durationSeconds,
completed = record.completed,
updatedAt = record.updatedAt,
)
}
},
),
)
}
put("/longform/progress/{songId}") {
val user = call.requireApiUser(users)
val songId = call.parameters["songId"] ?: badRequest("Song id is required")
val song = LibraryRepository.song(songId, user.longformFolderIds(library), user.name)
?: notFound("Longform track")
val request = call.receiveJson<SaveLongformProgressRequest>()
val duration = (request.durationSeconds ?: song.duration).coerceAtLeast(0)
val position = request.positionSeconds.coerceIn(0, maxOf(duration, song.duration))
val completed = request.completed || (duration > 0 && position >= duration - 5)
LongformRepository.save(
userName = user.name,
songId = song.id,
positionSeconds = position,
durationSeconds = duration,
completed = completed,
)
call.respond(HttpStatusCode.NoContent)
}
}
@@ -7,40 +7,39 @@ import io.ktor.server.routing.delete
import io.ktor.server.routing.get import io.ktor.server.routing.get
import io.ktor.server.routing.patch import io.ktor.server.routing.patch
import io.ktor.server.routing.post import io.ktor.server.routing.post
import java.nio.file.Files
import kotlinx.coroutines.Dispatchers import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.withContext import kotlinx.coroutines.withContext
import kotlinx.serialization.Serializable import kotlinx.serialization.Serializable
import win.taptap.musicmanager.api.AddSeedRequest
import win.taptap.musicmanager.api.AddToMoodResponse
import win.taptap.musicmanager.api.CreateMoodRequest
import win.taptap.musicmanager.api.MoodDetailDto
import win.taptap.musicmanager.api.MoodDto
import win.taptap.musicmanager.api.DownloadDto
import win.taptap.musicmanager.api.RecommendationDecisionDto
import win.taptap.musicmanager.api.RecommendationDto
import win.taptap.musicmanager.api.SearchResponse
import win.taptap.musicmanager.api.UpdateMoodRequest
import win.taptap.musicmanager.api.seedDto
import win.taptap.musicmanager.api.toDto
import win.taptap.musicmanager.config.LibraryConfig
import win.taptap.musicmanager.config.UserAccount import win.taptap.musicmanager.config.UserAccount
import win.taptap.musicmanager.config.UsersConfig import win.taptap.musicmanager.config.UsersConfig
import win.taptap.musicmanager.db.DownloadRepository import win.taptap.musicmanager.db.DownloadRepository
import win.taptap.musicmanager.db.LibraryRepository
import win.taptap.musicmanager.db.MoodRecord import win.taptap.musicmanager.db.MoodRecord
import win.taptap.musicmanager.db.MoodRepository import win.taptap.musicmanager.db.MoodRepository
import win.taptap.musicmanager.db.SongRecord
import win.taptap.musicmanager.db.SongRepository
import win.taptap.musicmanager.discovery.DiscoveryService import win.taptap.musicmanager.discovery.DiscoveryService
import win.taptap.musicmanager.discovery.TopUpService
import win.taptap.musicmanager.discovery.YtResolver
import win.taptap.musicmanager.discovery.isMusicLength
import win.taptap.musicmanager.media.MediaGcService
import win.taptap.musicmanager.media.MediaStore
import win.taptap.musicmanager.radio.RadioService import win.taptap.musicmanager.radio.RadioService
@Serializable @Serializable
data class MoodsResponse(val moods: List<MoodDto>) data class MoodsResponse(val moods: List<MoodDto>)
@Serializable
data class RecommendationsResponse(val recommendations: List<RecommendationDto>)
fun Route.moodRoutes( fun Route.moodRoutes(
users: UsersConfig, users: UsersConfig,
library: LibraryConfig,
radioService: RadioService, radioService: RadioService,
discoveryService: DiscoveryService, discoveryService: DiscoveryService,
downloadsEnabled: Boolean, topUpService: TopUpService,
resolver: YtResolver,
mediaStore: MediaStore,
gc: MediaGcService,
) { ) {
get("/moods") { get("/moods") {
val user = call.requireApiUser(users) val user = call.requireApiUser(users)
@@ -53,30 +52,24 @@ fun Route.moodRoutes(
val name = request.name.trim() val name = request.name.trim()
if (name.isEmpty()) badRequest("Mood name is required") if (name.isEmpty()) badRequest("Mood name is required")
val folders = user.musicFolderIds(library)
val seedIds = request.seedSongIds
.distinct()
.filter { LibraryRepository.song(it, folders, user.name) != null }
val mood = MoodRepository.create( val mood = MoodRepository.create(
owner = user.name, owner = user.name,
name = name, name = name,
description = request.description?.trim()?.takeIf { it.isNotEmpty() }, description = request.description?.trim()?.takeIf { it.isNotEmpty() },
query = request.query?.trim()?.takeIf { it.isNotEmpty() }, query = request.query?.trim()?.takeIf { it.isNotEmpty() },
genres = request.genres, genres = request.genres,
targetSize = request.targetSize, collectionPercent = request.collectionPercent,
recommendationCount = request.recommendationCount,
refreshHours = request.refreshHours, refreshHours = request.refreshHours,
ytEnabled = request.ytEnabled,
seedSongIds = seedIds,
seedPercent = request.seedPercent,
) )
call.respond(HttpStatusCode.Created, detail(mood, user, library)) if (mood.recommendationCount > 0) topUpService.trigger(mood)
call.respond(HttpStatusCode.Created, detail(mood))
} }
get("/moods/{id}") { get("/moods/{id}") {
val user = call.requireApiUser(users) val user = call.requireApiUser(users)
val mood = ownedMood(call.parameters["id"], user) val mood = ownedMood(call.parameters["id"], user)
call.respond(detail(mood, user, library)) call.respond(detail(mood))
} }
patch("/moods/{id}") { patch("/moods/{id}") {
@@ -102,45 +95,20 @@ fun Route.moodRoutes(
description = description, description = description,
query = query, query = query,
genres = request.genres ?: mood.genres, genres = request.genres ?: mood.genres,
targetSize = request.targetSize ?: mood.targetSize, collectionPercent = request.collectionPercent ?: mood.collectionPercent,
recommendationCount = request.recommendationCount ?: mood.recommendationCount,
refreshHours = request.refreshHours ?: mood.refreshHours, refreshHours = request.refreshHours ?: mood.refreshHours,
ytEnabled = request.ytEnabled ?: mood.ytEnabled,
seedPercent = request.seedPercent ?: mood.seedPercent,
) ?: notFound("Mood") ) ?: notFound("Mood")
call.respond(detail(updated, user, library)) topUpService.trigger(updated)
call.respond(detail(updated))
} }
delete("/moods/{id}") { delete("/moods/{id}") {
val user = call.requireApiUser(users) val user = call.requireApiUser(users)
val mood = ownedMood(call.parameters["id"], user) val mood = ownedMood(call.parameters["id"], user)
MoodRepository.delete(mood.id, user.name) MoodRepository.delete(mood.id, user.name)
call.respond(HttpStatusCode.NoContent) gc.collectGarbage()
}
post("/moods/{id}/seeds") {
val user = call.requireApiUser(users)
val mood = ownedMood(call.parameters["id"], user)
val request = call.receiveJson<AddSeedRequest>()
val song = requireMusicSong(request.songId, user, library)
val added = MoodRepository.addSeed(
id = mood.id,
owner = user.name,
songId = song.id,
origin = request.origin,
)
val refreshed = MoodRepository.get(mood.id, user.name) ?: mood
call.respond(
HttpStatusCode.OK,
AddToMoodResponse(mood = refreshed.toDto(), added = added),
)
}
delete("/moods/{id}/seeds/{songId}") {
val user = call.requireApiUser(users)
val mood = ownedMood(call.parameters["id"], user)
val songId = call.parameters["songId"] ?: badRequest("Song id is required")
MoodRepository.removeSeed(mood.id, user.name, songId)
call.respond(HttpStatusCode.NoContent) call.respond(HttpStatusCode.NoContent)
} }
@@ -149,71 +117,169 @@ fun Route.moodRoutes(
val user = call.requireApiUser(users) val user = call.requireApiUser(users)
val mood = ownedMood(call.parameters["id"], user) val mood = ownedMood(call.parameters["id"], user)
val limit = (call.request.queryParameters["limit"]?.toIntOrNull() ?: 20).coerceIn(1, 50) val limit = (call.request.queryParameters["limit"]?.toIntOrNull() ?: 20).coerceIn(1, 50)
call.respond(SearchResponse(radioService.preview(mood, user, library, limit).map { it.toDto() })) call.respond(TracksResponse(radioService.preview(mood, user, limit).map { it.toDto() }))
} }
get("/moods/{id}/recommendations") { get("/moods/{id}/recommendations") {
val user = call.requireApiUser(users) val user = call.requireApiUser(users)
val mood = ownedMood(call.parameters["id"], user) val mood = ownedMood(call.parameters["id"], user)
call.respond(recommendations(mood, user, library)) call.respond(RecommendationsResponse(recommendations(mood)))
} }
/** Runs a discovery pass now, bypassing the mood's refresh interval. */ /** Runs a discovery pass now, bypassing the mood's refresh interval. */
post("/moods/{id}/recommendations/refresh") { post("/moods/{id}/recommendations/refresh") {
val user = call.requireApiUser(users) val user = call.requireApiUser(users)
val mood = ownedMood(call.parameters["id"], user) val mood = ownedMood(call.parameters["id"], user)
if (!mood.ytEnabled) conflict("YT discovery is disabled for this mood") if (mood.recommendationCount <= 0) conflict("Recommendations are disabled for this mood")
val outcome = withContext(Dispatchers.IO) { val outcome = withContext(Dispatchers.IO) {
discoveryService.refresh(mood, user, library, force = true) discoveryService.refresh(mood, force = true)
} }
outcome.error?.let { message -> outcome.error?.let { message ->
throw ApiException(HttpStatusCode.BadGateway, "discovery_failed", message) throw ApiException(HttpStatusCode.BadGateway, "discovery_failed", message)
} }
topUpService.topUp(mood, force = false)
val refreshed = MoodRepository.get(mood.id, user.name) ?: mood val refreshed = MoodRepository.get(mood.id, user.name) ?: mood
call.respond(detail(refreshed, user, library)) call.respond(detail(refreshed))
} }
post("/moods/{id}/recommendations/{videoId}/accept") { post("/moods/{id}/recommendations/{videoId}/add") {
val user = call.requireApiUser(users) val user = call.requireApiUser(users)
val mood = ownedMood(call.parameters["id"], user) val mood = ownedMood(call.parameters["id"], user)
val videoId = call.parameters["videoId"] ?: badRequest("Video id is required") val videoId = call.parameters["videoId"] ?: badRequest("Video id is required")
val recommendation = MoodRepository.recommendation(mood.id, videoId) val recommendation = MoodRepository.recommendation(mood.id, videoId)
?: notFound("Recommendation") ?: notFound("Recommendation")
if (recommendation.state == "failed") {
val matched = recommendation.matchedSongId?.let { songId -> conflict("Retry the download before adding this track")
LibraryRepository.song(songId, user.musicFolderIds(library), user.name)
} ?: DownloadRepository.completedSongForVideo(recommendation.videoId)?.let { songId ->
// The same video may already have been downloaded for another mood.
LibraryRepository.song(songId, user.musicFolderIds(library), user.name)
} }
val seedAdded = matched != null &&
MoodRepository.addSeed(mood.id, user.name, matched.id, "recommendation") MoodRepository.markAdded(mood.id, videoId)
val alreadyAccepted = recommendation.state == "accepted" topUpService.trigger(mood)
MoodRepository.setRecommendationState(mood.id, videoId, "accepted") val refreshed = MoodRepository.get(mood.id, user.name) ?: mood
if (!alreadyAccepted) MoodRepository.incrementRecommendationAdds(mood.id, videoId) call.respond(
if (matched == null && downloadsEnabled) { RecommendationDecisionDto(
DownloadRepository.enqueue( detail = detail(refreshed),
userName = user.name, recommendation = MoodRepository.recommendation(mood.id, videoId)
moodId = mood.id, ?.toDto(song = recommendationSong(mood.id, videoId))
videoId = recommendation.videoId, ?: recommendation.toDto(song = recommendationSong(mood.id, videoId)),
title = recommendation.title, ),
artist = recommendation.artist,
) )
} }
call.respond(decision(mood.id, videoId, user, library, seedAdded))
}
post("/moods/{id}/recommendations/{videoId}/reject") { post("/moods/{id}/recommendations/{videoId}/reject") {
val user = call.requireApiUser(users) val user = call.requireApiUser(users)
val mood = ownedMood(call.parameters["id"], user) val mood = ownedMood(call.parameters["id"], user)
val videoId = call.parameters["videoId"] ?: badRequest("Video id is required") val videoId = call.parameters["videoId"] ?: badRequest("Video id is required")
MoodRepository.recommendation(mood.id, videoId) ?: notFound("Recommendation") val recommendation = MoodRepository.recommendation(mood.id, videoId)
MoodRepository.setRecommendationState(mood.id, videoId, "rejected") ?: notFound("Recommendation")
DownloadRepository.cancel(mood.id, videoId, "Recommendation rejected")
call.respond(decision(mood.id, videoId, user, library, seedAdded = false)) MoodRepository.markRejected(mood.id, videoId)
DownloadRepository.cancelForMoodVideo(mood.id, videoId, "Recommendation rejected")
recommendation.songId?.let(gc::collect)
topUpService.trigger(mood)
val refreshed = MoodRepository.get(mood.id, user.name) ?: mood
call.respond(
RecommendationDecisionDto(
detail = detail(refreshed),
recommendation = recommendation.copy(state = "rejected").toDto(),
),
)
} }
post("/moods/{id}/recommendations/{videoId}/retry") {
val user = call.requireApiUser(users)
val mood = ownedMood(call.parameters["id"], user)
val videoId = call.parameters["videoId"] ?: badRequest("Video id is required")
val recommendation = MoodRepository.recommendation(mood.id, videoId)
?: notFound("Recommendation")
if (recommendation.state != "failed") conflict("Only failed downloads can be retried")
MoodRepository.markRecommendationDownloading(mood.id, videoId)
DownloadRepository.enqueue(
videoId = videoId,
moodId = mood.id,
kind = DownloadRepository.KIND_RECOMMENDATION,
)
val refreshed = MoodRepository.get(mood.id, user.name) ?: mood
call.respond(detail(refreshed))
}
post("/moods/{id}/songs") {
val user = call.requireApiUser(users)
val mood = ownedMood(call.parameters["id"], user)
val request = call.receiveJson<AddSongRequest>()
val normalized = normalizeAdd(request, resolver)
val bySongId = normalized.songId?.let(SongRepository::byId)
if (normalized.songId != null && bySongId == null) notFound("Song")
val existing = (bySongId ?: normalized.videoId?.let(SongRepository::byVideoId))
?.takeIf { mediaStore.file(it)?.let(Files::isRegularFile) == true }
if (existing != null) {
val added = MoodRepository.addSong(mood.id, existing.id, normalized.origin)
val refreshed = MoodRepository.get(mood.id, user.name) ?: mood
call.respond(
AddSongResponse(
detail = detail(refreshed),
added = added,
downloading = false,
),
)
return@post
}
val videoId = normalized.videoId ?: badRequest("No song or video id was supplied")
DownloadRepository.enqueue(
videoId = videoId,
moodId = mood.id,
kind = DownloadRepository.KIND_SONG,
)
val refreshed = MoodRepository.get(mood.id, user.name) ?: mood
call.respond(
HttpStatusCode.Accepted,
AddSongResponse(detail = detail(refreshed), added = false, downloading = true),
)
}
delete("/moods/{id}/songs/{songId}") {
val user = call.requireApiUser(users)
val mood = ownedMood(call.parameters["id"], user)
val songId = call.parameters["songId"] ?: badRequest("Song id is required")
MoodRepository.removeSong(mood.id, songId)
gc.collect(songId)
call.respond(HttpStatusCode.NoContent)
}
}
private data class NormalizedAdd(
val songId: String?,
val videoId: String?,
val origin: String,
)
private fun normalizeAdd(request: AddSongRequest, resolver: YtResolver): NormalizedAdd = when {
!request.songId.isNullOrBlank() -> NormalizedAdd(request.songId, null, "search")
!request.videoId.isNullOrBlank() -> NormalizedAdd(null, request.videoId.trim(), "search")
!request.url.isNullOrBlank() -> {
val candidate = runCatching { resolver.resolve(request.url) }
.getOrElse { error ->
throw ApiException(
HttpStatusCode.BadGateway,
"resolve_failed",
error.message ?: "Could not resolve the URL",
)
}
?: throw ApiException(
HttpStatusCode.BadGateway,
"resolve_failed",
"The URL did not resolve to a video",
)
if (!isMusicLength(candidate.duration)) {
badRequest("Only tracks up to 10 minutes can be added")
}
NormalizedAdd(null, candidate.videoId, "url")
}
else -> badRequest("Supply songId, videoId or url")
} }
private fun ownedMood(id: String?, user: UserAccount): MoodRecord { private fun ownedMood(id: String?, user: UserAccount): MoodRecord {
@@ -221,54 +287,29 @@ private fun ownedMood(id: String?, user: UserAccount): MoodRecord {
return MoodRepository.get(id, user.name) ?: notFound("Mood") return MoodRepository.get(id, user.name) ?: notFound("Mood")
} }
private fun requireMusicSong(songId: String, user: UserAccount, library: LibraryConfig) = private fun detail(mood: MoodRecord): MoodDetailDto {
LibraryRepository.song(songId, user.musicFolderIds(library), user.name) val songRecords = MoodRepository.songRecords(mood.id).associateBy { it.id }
?: notFound("Song") val songs = MoodRepository.songs(mood.id).mapNotNull { record ->
songRecords[record.songId]?.let { record.toDto(it) }
private fun detail(mood: MoodRecord, user: UserAccount, library: LibraryConfig): MoodDetailDto {
val folders = user.musicFolderIds(library)
val seedRecords = MoodRepository.seeds(mood.id)
val seedSongs = LibraryRepository
.songsByIds(seedRecords.map { it.songId }, folders, user.name)
.associateBy { it.id }
val seeds = seedRecords.mapNotNull { seed ->
seedSongs[seed.songId]?.let { seedDto(seed, it) }
} }
return MoodDetailDto( return MoodDetailDto(
mood = mood.toDto(), mood = mood.toDto(),
seeds = seeds, songs = songs,
recommendations = recommendations(mood, user, library).recommendations, recommendations = recommendations(mood),
) )
} }
private fun recommendations( private fun recommendations(mood: MoodRecord): List<RecommendationDto> {
mood: MoodRecord,
user: UserAccount,
library: LibraryConfig,
): RecommendationsResponse {
val records = MoodRepository.recommendations(mood.id) val records = MoodRepository.recommendations(mood.id)
val matchedIds = records.mapNotNull { it.matchedSongId } val songs = SongRepository.byIds(records.mapNotNull { it.songId }).associateBy { it.id }
val matchedSongs = LibraryRepository val jobs = DownloadRepository.forMood(mood.id)
.songsByIds(matchedIds, user.visibleFolderIds(library), user.name) .filter { it.kind == DownloadRepository.KIND_RECOMMENDATION }
.associateBy { it.id } .associateBy { it.videoId }
val downloads = DownloadRepository.forMood(user.name, mood.id)
return RecommendationsResponse( return records.map { record ->
recommendations = records.map { record -> record.toDto(
RecommendationDto( song = record.songId?.let { songs[it] },
videoId = record.videoId, download = jobs[record.videoId]?.let { job ->
title = record.title,
artist = record.artist,
album = record.album,
duration = record.duration,
thumbnail = record.thumbnail,
state = record.state,
matchedSong = record.matchedSongId?.let { matchedSongs[it] }?.toDto(),
timesAdded = record.timesAdded,
firstSeen = record.firstSeen,
lastSeen = record.lastSeen,
download = downloads[record.videoId]?.let { job ->
DownloadDto( DownloadDto(
state = job.state, state = job.state,
attempts = job.attempts, attempts = job.attempts,
@@ -277,27 +318,8 @@ private fun recommendations(
) )
}, },
) )
}, }
)
} }
private fun decision( private fun recommendationSong(moodId: String, videoId: String): SongRecord? =
moodId: String, MoodRepository.recommendation(moodId, videoId)?.songId?.let(SongRepository::byId)
videoId: String,
user: UserAccount,
library: LibraryConfig,
seedAdded: Boolean,
): RecommendationDecisionDto {
val mood = MoodRepository.get(moodId, user.name) ?: notFound("Mood")
val detail = detail(mood, user, library)
val recommendation = detail.recommendations.firstOrNull { it.videoId == videoId }
?: notFound("Recommendation")
return RecommendationDecisionDto(
detail = detail,
recommendation = recommendation,
seedAdded = seedAdded,
)
}
@Serializable
data class RecommendationsResponse(val recommendations: List<RecommendationDto>)
@@ -7,41 +7,35 @@ import io.ktor.server.routing.Route
import io.ktor.server.routing.delete import io.ktor.server.routing.delete
import io.ktor.server.routing.get import io.ktor.server.routing.get
import io.ktor.server.routing.post import io.ktor.server.routing.post
import kotlinx.serialization.Serializable
import win.taptap.musicmanager.api.AddToMoodRequest
import win.taptap.musicmanager.api.AddToMoodResponse
import win.taptap.musicmanager.api.NextTracksRequest
import win.taptap.musicmanager.api.PlaybackEventsRequest
import win.taptap.musicmanager.api.PlaybackEventsResponse
import win.taptap.musicmanager.api.SessionDto
import win.taptap.musicmanager.api.SongDto
import win.taptap.musicmanager.api.StartSessionRequest
import win.taptap.musicmanager.api.toDto
import win.taptap.musicmanager.config.LibraryConfig
import win.taptap.musicmanager.config.UserAccount import win.taptap.musicmanager.config.UserAccount
import win.taptap.musicmanager.config.UsersConfig import win.taptap.musicmanager.config.UsersConfig
import win.taptap.musicmanager.db.LibraryRepository
import win.taptap.musicmanager.db.MoodRecord import win.taptap.musicmanager.db.MoodRecord
import win.taptap.musicmanager.db.MoodRepository import win.taptap.musicmanager.db.MoodRepository
import win.taptap.musicmanager.db.PlaybackSessionRecord import win.taptap.musicmanager.db.PlaybackSessionRecord
import win.taptap.musicmanager.db.RadioRepository import win.taptap.musicmanager.db.RadioRepository
import win.taptap.musicmanager.db.SongRepository
import win.taptap.musicmanager.media.MediaGcService
import win.taptap.musicmanager.radio.RadioService import win.taptap.musicmanager.radio.RadioService
@Serializable
data class TracksResponse(val songs: List<SongDto>)
fun Route.sessionRoutes( fun Route.sessionRoutes(
users: UsersConfig, users: UsersConfig,
library: LibraryConfig,
radioService: RadioService, radioService: RadioService,
gc: MediaGcService,
) { ) {
post("/moods/{id}/sessions") { post("/moods/{id}/sessions") {
val user = call.requireApiUser(users) val user = call.requireApiUser(users)
val mood = ownedMoodFor(call.parameters["id"], user) val mood = ownedMoodFor(call.parameters["id"], user)
if (MoodRepository.songIds(mood.id).isEmpty()) {
throw ApiException(
HttpStatusCode.BadRequest,
"empty_mood",
"Add at least one song before playing this mood",
)
}
val request = call.receiveNullable<StartSessionRequest>() ?: StartSessionRequest() val request = call.receiveNullable<StartSessionRequest>() ?: StartSessionRequest()
val limit = request.limit ?: DEFAULT_BATCH val limit = request.limit ?: DEFAULT_BATCH
val started = radioService.start(mood, user, library, limit) val started = radioService.start(mood, user, limit)
call.respond( call.respond(
HttpStatusCode.Created, HttpStatusCode.Created,
SessionDto( SessionDto(
@@ -58,11 +52,8 @@ fun Route.sessionRoutes(
val user = call.requireApiUser(users) val user = call.requireApiUser(users)
val session = ownedSession(call.parameters["id"], user) val session = ownedSession(call.parameters["id"], user)
val mood = ownedMoodFor(session.moodId, user) val mood = ownedMoodFor(session.moodId, user)
val folders = user.musicFolderIds(library)
val items = RadioRepository.items(session.id) val items = RadioRepository.items(session.id)
val songs = LibraryRepository val songs = SongRepository.byIds(items.map { it.songId }).associateBy { it.id }
.songsByIds(items.map { it.songId }, folders, user.name)
.associateBy { it.id }
call.respond( call.respond(
SessionDto( SessionDto(
@@ -86,7 +77,6 @@ fun Route.sessionRoutes(
val songs = radioService.next( val songs = radioService.next(
mood = mood, mood = mood,
user = user, user = user,
library = library,
sessionId = session.id, sessionId = session.id,
excluded = request.excludeSongIds.toSet(), excluded = request.excludeSongIds.toSet(),
limit = request.limit ?: DEFAULT_BATCH, limit = request.limit ?: DEFAULT_BATCH,
@@ -111,9 +101,7 @@ fun Route.sessionRoutes(
val user = call.requireApiUser(users) val user = call.requireApiUser(users)
val session = ownedSession(call.parameters["id"], user) val session = ownedSession(call.parameters["id"], user)
val request = call.receiveJson<AddToMoodRequest>() val request = call.receiveJson<AddToMoodRequest>()
val folders = user.musicFolderIds(library) val song = SongRepository.byId(request.songId) ?: notFound("Song")
val song = LibraryRepository.song(request.songId, folders, user.name)
?: notFound("Song")
val target = if (request.moodId.isNullOrBlank()) { val target = if (request.moodId.isNullOrBlank()) {
ownedMoodFor(session.moodId, user) ownedMoodFor(session.moodId, user)
@@ -121,7 +109,7 @@ fun Route.sessionRoutes(
ownedMoodFor(request.moodId, user) ownedMoodFor(request.moodId, user)
} }
val added = MoodRepository.addSeed(target.id, user.name, song.id, "manual") val added = MoodRepository.addSong(target.id, song.id, "search")
val refreshed = MoodRepository.get(target.id, user.name) ?: target val refreshed = MoodRepository.get(target.id, user.name) ?: target
call.respond( call.respond(
AddToMoodResponse( AddToMoodResponse(
@@ -135,6 +123,7 @@ fun Route.sessionRoutes(
val user = call.requireApiUser(users) val user = call.requireApiUser(users)
val session = ownedSession(call.parameters["id"], user) val session = ownedSession(call.parameters["id"], user)
RadioRepository.close(session.id, user.name) RadioRepository.close(session.id, user.name)
gc.collectGarbage()
call.respond(HttpStatusCode.NoContent) call.respond(HttpStatusCode.NoContent)
} }
} }
@@ -0,0 +1,162 @@
package win.taptap.musicmanager.api
import io.ktor.http.ContentType
import io.ktor.http.HttpHeaders
import io.ktor.http.HttpStatusCode
import io.ktor.server.response.header
import io.ktor.server.response.respond
import io.ktor.server.response.respondFile
import io.ktor.server.response.respondOutputStream
import io.ktor.server.routing.Route
import io.ktor.server.routing.get
import java.net.URLEncoder
import java.nio.file.Files
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.withContext
import org.slf4j.LoggerFactory
import win.taptap.musicmanager.config.UsersConfig
import win.taptap.musicmanager.db.SongRecord
import win.taptap.musicmanager.db.SongRepository
import win.taptap.musicmanager.discovery.YtSearch
import win.taptap.musicmanager.discovery.isMusicLength
import win.taptap.musicmanager.media.MediaStore
import win.taptap.musicmanager.scanner.CoverArtService
import win.taptap.musicmanager.transcoding.FfmpegTranscoder
private val log = LoggerFactory.getLogger("win.taptap.musicmanager.api.SongRoutes")
/**
* YouTube search plus the shared media endpoints. Songs are server-wide, so
* any authenticated user can stream a song id they received from their own
* moods or from search.
*/
fun Route.songRoutes(
users: UsersConfig,
search: YtSearch,
mediaStore: MediaStore,
coverArtService: CoverArtService,
transcoder: FfmpegTranscoder,
) {
get("/search") {
call.requireApiUser(users)
val query = call.request.queryParameters["q"]?.trim().orEmpty()
if (query.isEmpty()) {
call.respond(SearchResponse())
return@get
}
val limit = (call.request.queryParameters["limit"]?.toIntOrNull() ?: 20).coerceIn(1, 25)
val results = withContext(Dispatchers.IO) {
runCatching { search.search(query, limit) }
.getOrElse { error ->
throw ApiException(
HttpStatusCode.BadGateway,
"search_failed",
error.message ?: "YouTube search failed",
)
}
}
call.respond(
SearchResponse(
results = results
.filter { isMusicLength(it.duration) }
.map { candidate ->
val song = SongRepository.byVideoId(candidate.videoId)
val downloaded = song != null && isPresent(mediaStore, song)
SearchResultDto(
videoId = candidate.videoId,
title = candidate.title,
artist = candidate.artist,
album = candidate.album,
duration = candidate.duration,
thumbnail = candidate.thumbnail,
songId = song?.takeIf { downloaded }?.id,
downloaded = downloaded,
)
},
),
)
}
get("/songs/{id}/stream") {
call.requireApiUser(users)
val id = call.parameters["id"] ?: badRequest("Song id is required")
val song = SongRepository.byId(id) ?: notFound("Song")
val file = mediaStore.file(song)?.takeIf { Files.isRegularFile(it) } ?: notFound("Song file")
val parameters = call.request.queryParameters
val rawMaxBitRate = parameters["maxBitRate"]
val maxBitRate = rawMaxBitRate?.toIntOrNull()
if (rawMaxBitRate != null && (maxBitRate == null || maxBitRate < 0)) {
badRequest("Invalid maxBitRate")
}
val rawOffset = parameters["timeOffset"]
val offset = rawOffset?.toIntOrNull()
if (rawOffset != null && (offset == null || offset < 0)) {
badRequest("Invalid timeOffset")
}
if (!transcoder.shouldTranscode(song.bitRate, maxBitRate, parameters["format"], offset ?: 0)) {
call.response.header(HttpHeaders.ContentType, song.contentType)
if (parameters["download"] == "true") {
call.response.header(
HttpHeaders.ContentDisposition,
contentDisposition(file.fileName.toString()),
)
}
call.respondFile(file.toFile())
return@get
}
val process = try {
transcoder.start(file, maxBitRate, offset ?: 0)
} catch (e: Exception) {
log.error("Failed to start ffmpeg for {}", file, e)
throw ApiException(HttpStatusCode.InternalServerError, "transcode_failed", "Transcoding failed")
}
try {
call.response.headers.append(HttpHeaders.CacheControl, "no-store")
call.respondOutputStream(ContentType.parse("audio/mpeg")) {
val output = this
withContext(Dispatchers.IO) {
process.inputStream.use { input -> input.copyTo(output) }
}
}
} finally {
transcoder.stop(process)
}
}
get("/songs/{id}/cover") {
call.requireApiUser(users)
val id = call.parameters["id"] ?: badRequest("Song id is required")
val rawSize = call.request.queryParameters["size"]
val size = rawSize?.toIntOrNull()
if (rawSize != null && (size == null || size !in 1..2048)) {
badRequest("Cover size must be between 1 and 2048")
}
SongRepository.byId(id) ?: notFound("Song")
val cover = if (size == null) {
coverArtService.coverFile(id)
} else {
coverArtService.resizedCover(id, size)
} ?: notFound("Cover")
call.respondFile(cover.toFile())
}
}
private fun isPresent(mediaStore: MediaStore, song: SongRecord): Boolean =
mediaStore.file(song)?.let { Files.isRegularFile(it) } == true
private fun contentDisposition(fileName: String): String {
val encoded = URLEncoder.encode(fileName, Charsets.UTF_8).replace("+", "%20")
val ascii = fileName.map { character ->
if (character.code in 32..126 && character != '"' && character != '\\') character else '_'
}.joinToString("")
return "attachment; filename=\"$ascii\"; filename*=UTF-8''$encoded"
}
@@ -5,19 +5,17 @@ import java.nio.file.Path
/** /**
* Application-level configuration, sourced exclusively from environment variables. * Application-level configuration, sourced exclusively from environment variables.
* *
* Library folders and users are not part of this class: they live in the YAML files * There is no mounted library: the server owns [mediaDir] and fills it by
* referenced here so they can be managed by ConfigMaps/Vault. * downloading from YouTube. Users still live in a Vault-managed YAML file.
*/ */
data class AppConfig( data class AppConfig(
val port: Int, val port: Int,
val dataDir: Path, val dataDir: Path,
val libraryConfigPath: Path,
val usersFile: Path, val usersFile: Path,
val ffmpegPath: String, val ffmpegPath: String,
val ytDlpPath: String = "yt-dlp", val ytDlpPath: String = "yt-dlp",
val downloadsEnabled: Boolean = true, val mediaDir: Path = dataDir.resolve("media"),
val downloadDir: Path = dataDir.resolve("downloads"), val downloadDir: Path = dataDir.resolve("downloads"),
val downloadFolderId: Int? = null,
) { ) {
val databasePath: Path get() = dataDir.resolve("musicmanager.db") val databasePath: Path get() = dataDir.resolve("musicmanager.db")
@@ -36,13 +34,11 @@ data class AppConfig(
return AppConfig( return AppConfig(
port = port, port = port,
dataDir = dataDir, dataDir = dataDir,
libraryConfigPath = Path.of(value("LIBRARY_CONFIG", "./config/library.yaml")),
usersFile = Path.of(value("USERS_FILE", "./config/users.yaml")), usersFile = Path.of(value("USERS_FILE", "./config/users.yaml")),
ffmpegPath = value("FFMPEG_PATH", "ffmpeg"), ffmpegPath = value("FFMPEG_PATH", "ffmpeg"),
ytDlpPath = value("YTDLP_PATH", "yt-dlp"), ytDlpPath = value("YTDLP_PATH", "yt-dlp"),
downloadsEnabled = value("DOWNLOADS_ENABLED", "true").toBooleanStrictOrNull() ?: true, mediaDir = Path.of(value("MEDIA_DIR", dataDir.resolve("media").toString())),
downloadDir = Path.of(value("DOWNLOAD_DIR", dataDir.resolve("downloads").toString())), downloadDir = Path.of(value("DOWNLOAD_DIR", dataDir.resolve("downloads").toString())),
downloadFolderId = env["DOWNLOAD_FOLDER_ID"]?.takeIf { it.isNotBlank() }?.toIntOrNull(),
) )
} }
} }
@@ -1,138 +0,0 @@
package win.taptap.musicmanager.config
import java.nio.file.Files
import java.nio.file.Path
import org.yaml.snakeyaml.LoaderOptions
import org.yaml.snakeyaml.Yaml
import org.yaml.snakeyaml.constructor.SafeConstructor
enum class FolderType {
MUSIC,
LONGFORM,
}
data class MusicFolder(
val id: Int,
val name: String,
val type: FolderType,
val path: Path,
)
/**
* Library folders from `library.yaml`. Folder IDs are stable and referenced by songs,
* so they must never be reused for a different path.
*/
data class LibraryConfig(val folders: List<MusicFolder>) {
val longformFolderIds: Set<Int> = folders
.filter { it.type == FolderType.LONGFORM }
.map { it.id }
.toSet()
fun byId(id: Int): MusicFolder? = folders.firstOrNull { it.id == id }
/** Fails fast at startup if a configured root is missing or unreadable. */
fun validatePaths() {
folders.forEach { folder ->
require(Files.isDirectory(folder.path)) {
"Library folder ${folder.id} (${folder.name}) is not a directory: ${folder.path}. " +
"Create the directory or fix the path in the library config (LIBRARY_CONFIG). " +
"For local development, run `./gradlew setupLocal` and copy " +
"config/library.example.yaml to config/library.yaml. That file is " +
"git-ignored, so `git pull` never updates it."
}
require(Files.isReadable(folder.path)) {
"Library folder ${folder.id} (${folder.name}) is not readable: ${folder.path}. " +
"Check the directory permissions."
}
}
}
companion object {
fun load(path: Path): LibraryConfig {
require(Files.isRegularFile(path)) {
"Library config not found: $path. Copy config/library.example.yaml to " +
"config/library.yaml or point LIBRARY_CONFIG at an existing file."
}
val text = Files.readString(path)
return parse(text)
}
fun parse(text: String): LibraryConfig {
val root = parseYaml(text)
val library = root["library"].asMapping("library")
val rawFolders = library["folders"].asList("library.folders")
val folders = rawFolders.mapIndexed { index, raw ->
val item = raw.asMapping("library.folders[$index]")
val id = item["id"].asInt("library.folders[$index].id")
require(id > 0) { "library.folders[$index].id must be positive" }
val name = item["name"].asString("library.folders[$index].name").trim()
require(name.isNotEmpty()) { "library.folders[$index].name must not be blank" }
val rawType = item["type"].asString("library.folders[$index].type")
val type = runCatching { FolderType.valueOf(rawType.trim().uppercase()) }
.getOrElse {
throw IllegalArgumentException(
"library.folders[$index].type must be 'music' or 'longform', got: $rawType",
)
}
val rawPath = item["path"].asString("library.folders[$index].path")
val path = Path.of(rawPath).toAbsolutePath().normalize()
MusicFolder(id = id, name = name, type = type, path = path)
}
require(folders.isNotEmpty()) { "library.folders must not be empty" }
folders.groupBy { it.id }.forEach { (id, duplicates) ->
require(duplicates.size == 1) { "Duplicate library folder id: $id" }
}
folders.groupBy { it.path }.forEach { (path, duplicates) ->
require(duplicates.size == 1) { "Duplicate library folder path: $path" }
}
return LibraryConfig(folders)
}
private fun parseYaml(text: String): Map<String, Any?> {
val loaded = Yaml(SafeConstructor(LoaderOptions())).load<Any?>(text)
?: throw IllegalArgumentException("Config must not be empty")
@Suppress("UNCHECKED_CAST")
return loaded as? Map<String, Any?>
?: throw IllegalArgumentException("Config root must be a YAML mapping")
}
}
}
internal fun Any?.asMapping(what: String): Map<String, Any?> {
@Suppress("UNCHECKED_CAST")
return this as? Map<String, Any?>
?: throw IllegalArgumentException("$what must be a mapping")
}
internal fun Any?.asList(what: String): List<Any?> =
this as? List<Any?> ?: throw IllegalArgumentException("$what must be a list")
internal fun Any?.asString(what: String): String =
this as? String ?: throw IllegalArgumentException("$what must be a string")
internal fun Any?.asInt(what: String): Int = when (this) {
is Int -> this
is Long -> toInt()
is String -> toIntOrNull()
?: throw IllegalArgumentException("$what must be an integer")
else -> throw IllegalArgumentException("$what must be an integer")
}
internal fun Any?.asBoolean(what: String): Boolean = when (this) {
is Boolean -> this
is String -> when (trim().lowercase()) {
"true", "yes", "1" -> true
"false", "no", "0" -> false
else -> throw IllegalArgumentException("$what must be a boolean")
}
else -> throw IllegalArgumentException("$what must be a boolean")
}
@@ -10,19 +10,17 @@ import org.yaml.snakeyaml.constructor.SafeConstructor
* User accounts. Passwords are supplied by a Vault-managed secret and are never * User accounts. Passwords are supplied by a Vault-managed secret and are never
* persisted in SQLite. The `toString` implementation redacts the password so it * persisted in SQLite. The `toString` implementation redacts the password so it
* cannot leak through logs or error messages. * cannot leak through logs or error messages.
*
* Moods are strictly per-user, so there are no folder assignments anymore. An
* old `folders:` key in users.yaml is accepted and ignored.
*/ */
data class UserAccount( data class UserAccount(
val name: String, val name: String,
val password: String, val password: String,
val admin: Boolean, val admin: Boolean,
val folders: Set<Int>,
) { ) {
/** Music folders assigned to the user plus every shared long-form folder. */
fun visibleFolderIds(library: LibraryConfig): Set<Int> =
folders + library.longformFolderIds
override fun toString(): String = override fun toString(): String =
"UserAccount(name=$name, admin=$admin, folders=$folders, password=***)" "UserAccount(name=$name, admin=$admin, password=***)"
} }
data class UsersConfig(val users: List<UserAccount>) { data class UsersConfig(val users: List<UserAccount>) {
@@ -36,19 +34,6 @@ data class UsersConfig(val users: List<UserAccount>) {
fun byName(name: String): UserAccount? = users.firstOrNull { it.name == name } fun byName(name: String): UserAccount? = users.firstOrNull { it.name == name }
fun validateAgainst(library: LibraryConfig) {
val knownFolderIds = library.folders.map { it.id }.toSet()
users.forEach { user ->
val unknown = user.folders - knownFolderIds
require(unknown.isEmpty()) {
"User '${user.name}' references unknown library folders: ${unknown.sorted()}"
}
require(user.visibleFolderIds(library).isNotEmpty()) {
"User '${user.name}' has no visible library folders"
}
}
}
companion object { companion object {
private val NAME_PATTERN = Regex("[A-Za-z0-9._-]{1,64}") private val NAME_PATTERN = Regex("[A-Za-z0-9._-]{1,64}")
@@ -81,16 +66,7 @@ data class UsersConfig(val users: List<UserAccount>) {
val admin = item["admin"]?.asBoolean("users[$index].admin") ?: false val admin = item["admin"]?.asBoolean("users[$index].admin") ?: false
val folders = when (val rawFolders = item["folders"]) { UserAccount(name = name, password = password, admin = admin)
null -> emptySet()
else -> rawFolders.asList("users[$index].folders")
.mapIndexed { folderIndex, value ->
value.asInt("users[$index].folders[$folderIndex]")
}
.toSet()
}
UserAccount(name = name, password = password, admin = admin, folders = folders)
} }
return UsersConfig(users) return UsersConfig(users)
@@ -0,0 +1,33 @@
package win.taptap.musicmanager.config
/** Small YAML coercion helpers shared by the config parsers. */
internal fun Any?.asMapping(what: String): Map<String, Any?> {
@Suppress("UNCHECKED_CAST")
return this as? Map<String, Any?>
?: throw IllegalArgumentException("$what must be a mapping")
}
internal fun Any?.asList(what: String): List<Any?> =
this as? List<Any?> ?: throw IllegalArgumentException("$what must be a list")
internal fun Any?.asString(what: String): String =
this as? String ?: throw IllegalArgumentException("$what must be a string")
internal fun Any?.asInt(what: String): Int = when (this) {
is Int -> this
is Long -> toInt()
is String -> toIntOrNull()
?: throw IllegalArgumentException("$what must be an integer")
else -> throw IllegalArgumentException("$what must be an integer")
}
internal fun Any?.asBoolean(what: String): Boolean = when (this) {
is Boolean -> this
is String -> when (trim().lowercase()) {
"true", "yes", "1" -> true
"false", "no", "0" -> false
else -> throw IllegalArgumentException("$what must be a boolean")
}
else -> throw IllegalArgumentException("$what must be a boolean")
}
@@ -6,45 +6,20 @@ import org.jetbrains.exposed.sql.insertIgnore
import org.jetbrains.exposed.sql.selectAll import org.jetbrains.exposed.sql.selectAll
import org.jetbrains.exposed.sql.transactions.transaction import org.jetbrains.exposed.sql.transactions.transaction
import org.jetbrains.exposed.sql.update import org.jetbrains.exposed.sql.update
import win.taptap.musicmanager.config.LibraryConfig
import win.taptap.musicmanager.config.UsersConfig import win.taptap.musicmanager.config.UsersConfig
/** /**
* Reconciles the config/Vault-managed identity tables with SQLite. * Reconciles the config/Vault-managed identity table with SQLite.
* *
* Removing a folder cascades to its songs; removing a user cascades to that * Removing a user cascades to that user's moods, sessions and affinity. Songs
* user's play history and mood ownership. * are shared server-wide and are not tied to users.
*/ */
object ConfigRepository { object ConfigRepository {
fun sync(library: LibraryConfig, users: UsersConfig) { fun sync(users: UsersConfig) {
val now = Instant.now().toString() val now = Instant.now().toString()
transaction { transaction {
val folderIds = library.folders.map { it.id }.toSet()
val obsoleteFolderIds = MusicFoldersTable.selectAll()
.map { it[MusicFoldersTable.id] }
.filterNot { it in folderIds }
obsoleteFolderIds.forEach { folderId ->
MusicFoldersTable.deleteWhere {
it.run { MusicFoldersTable.id eq folderId }
}
}
library.folders.forEach { folder ->
MusicFoldersTable.insertIgnore {
it[id] = folder.id
it[name] = folder.name
it[type] = folder.type.name.lowercase()
it[rootPath] = folder.path.toString()
}
MusicFoldersTable.update({ MusicFoldersTable.id eq folder.id }) {
it[name] = folder.name
it[type] = folder.type.name.lowercase()
it[rootPath] = folder.path.toString()
}
}
val userNames = users.users.map { it.name }.toSet() val userNames = users.users.map { it.name }.toSet()
val obsoleteUserNames = UsersTable.selectAll() val obsoleteUserNames = UsersTable.selectAll()
.map { it[UsersTable.name] } .map { it[UsersTable.name] }
@@ -3,6 +3,7 @@ package win.taptap.musicmanager.db
import java.time.Instant import java.time.Instant
import org.jetbrains.exposed.sql.SortOrder import org.jetbrains.exposed.sql.SortOrder
import org.jetbrains.exposed.sql.SqlExpressionBuilder.eq import org.jetbrains.exposed.sql.SqlExpressionBuilder.eq
import org.jetbrains.exposed.sql.SqlExpressionBuilder.inList
import org.jetbrains.exposed.sql.SqlExpressionBuilder.less import org.jetbrains.exposed.sql.SqlExpressionBuilder.less
import org.jetbrains.exposed.sql.and import org.jetbrains.exposed.sql.and
import org.jetbrains.exposed.sql.insertIgnore import org.jetbrains.exposed.sql.insertIgnore
@@ -12,12 +13,11 @@ import org.jetbrains.exposed.sql.update
data class DownloadJobRecord( data class DownloadJobRecord(
val id: String, val id: String,
val userName: String,
val moodId: String,
val videoId: String, val videoId: String,
val title: String, val moodId: String,
val artist: String, val kind: String,
val state: String, val state: String,
val priority: Int,
val attempts: Int, val attempts: Int,
val error: String?, val error: String?,
val songId: String?, val songId: String?,
@@ -26,58 +26,71 @@ data class DownloadJobRecord(
) )
/** /**
* Jobs created when a user accepts a recommendation that the library does not * A single global download queue. Manual song adds use a higher priority than
* have yet. A background worker downloads, scans and promotes them. * automatic recommendation downloads so a user's explicit action never waits
* behind a background refill.
*/ */
object DownloadRepository { object DownloadRepository {
const val KIND_SONG = "song"
const val KIND_RECOMMENDATION = "recommendation"
const val PRIORITY_SONG = 100
const val PRIORITY_RECOMMENDATION = 0
fun enqueue( fun enqueue(
userName: String,
moodId: String,
videoId: String, videoId: String,
title: String, moodId: String,
artist: String, kind: String,
): Boolean = transaction { priority: Int = if (kind == KIND_SONG) PRIORITY_SONG else PRIORITY_RECOMMENDATION,
): DownloadJobRecord = transaction {
val now = Instant.now().toString() val now = Instant.now().toString()
val id = "dl-" + ApiTokenRepository.hash("$moodId|$kind|$videoId").take(24)
val existing = DownloadJobsTable.selectAll() val existing = DownloadJobsTable.selectAll()
.where { .where { DownloadJobsTable.id eq id }
(DownloadJobsTable.userName eq userName) and
(DownloadJobsTable.moodId eq moodId) and
(DownloadJobsTable.videoId eq videoId)
}
.singleOrNull() .singleOrNull()
if (existing == null) { if (existing == null) {
val id = "dl-" + ApiTokenRepository.hash("$userName|$moodId|$videoId").take(24)
DownloadJobsTable.insertIgnore { DownloadJobsTable.insertIgnore {
it[DownloadJobsTable.id] = id it[DownloadJobsTable.id] = id
it[DownloadJobsTable.userName] = userName it[this.videoId] = videoId
it[DownloadJobsTable.moodId] = moodId it[this.moodId] = moodId
it[DownloadJobsTable.videoId] = videoId it[this.kind] = kind
it[DownloadJobsTable.title] = title.take(500)
it[DownloadJobsTable.artist] = artist.take(300)
it[state] = "queued" it[state] = "queued"
it[this.priority] = priority
it[attempts] = 0 it[attempts] = 0
it[createdAt] = now it[createdAt] = now
it[updatedAt] = now it[updatedAt] = now
} }
} else if (existing[DownloadJobsTable.state] == "failed") { } else if (existing[DownloadJobsTable.state] != "queued" &&
DownloadJobsTable.update({ DownloadJobsTable.id eq existing[DownloadJobsTable.id] }) { existing[DownloadJobsTable.state] != "downloading"
) {
// A previous attempt failed, or a finished job no longer has a file
// (the song was removed and garbage-collected). Start it again.
DownloadJobsTable.update({ DownloadJobsTable.id eq id }) {
it[state] = "queued" it[state] = "queued"
it[attempts] = 0 it[attempts] = 0
it[error] = null it[error] = null
it[this.songId] = null
it[this.priority] = priority
it[updatedAt] = now it[updatedAt] = now
} }
} }
true
DownloadJobsTable.selectAll()
.where { DownloadJobsTable.id eq id }
.single()
.toRecord()
} }
/** Takes the oldest queued job and marks it as downloading. */ /** Takes the highest-priority queued job and marks it as downloading. */
fun claimNext(): DownloadJobRecord? = transaction { fun claimNext(): DownloadJobRecord? = transaction {
recoverStale() recoverStale()
val row = DownloadJobsTable.selectAll() val row = DownloadJobsTable.selectAll()
.where { DownloadJobsTable.state eq "queued" } .where { DownloadJobsTable.state eq "queued" }
.orderBy(DownloadJobsTable.createdAt to SortOrder.ASC) .orderBy(
DownloadJobsTable.priority to SortOrder.DESC,
DownloadJobsTable.createdAt to SortOrder.ASC,
)
.limit(1) .limit(1)
.singleOrNull() .singleOrNull()
?: return@transaction null ?: return@transaction null
@@ -107,53 +120,6 @@ object DownloadRepository {
} }
} }
/** Rejects must stop a queued or in-flight download. */
fun cancel(moodId: String, videoId: String, reason: String) {
transaction {
DownloadJobsTable.update({
(DownloadJobsTable.moodId eq moodId) and
(DownloadJobsTable.videoId eq videoId) and
(DownloadJobsTable.state inList listOf("queued", "downloading"))
}) {
it[state] = "failed"
it[error] = reason.take(1000)
it[updatedAt] = Instant.now().toString()
}
}
}
/**
* A pod restart or an uncaught bookkeeping failure can leave jobs in
* `downloading`. Requeue them after a generous timeout so they are not lost.
*/
private fun recoverStale(now: Instant = Instant.now()) {
val cutoff = now.minusSeconds(STALE_DOWNLOAD_SECONDS).toString()
val stale = DownloadJobsTable.selectAll()
.where {
(DownloadJobsTable.state eq "downloading") and
(DownloadJobsTable.updatedAt less cutoff)
}
.toList()
stale.forEach { row ->
val id = row[DownloadJobsTable.id]
val attempts = row[DownloadJobsTable.attempts]
if (attempts >= MAX_ATTEMPTS) {
DownloadJobsTable.update({ DownloadJobsTable.id eq id }) {
it[state] = "failed"
it[error] = "Download was interrupted and could not be resumed"
it[updatedAt] = now.toString()
}
} else {
DownloadJobsTable.update({ DownloadJobsTable.id eq id }) {
it[state] = "queued"
it[error] = "Download was interrupted; retrying"
it[updatedAt] = now.toString()
}
}
}
}
fun markFailed(id: String, error: String) { fun markFailed(id: String, error: String) {
transaction { transaction {
DownloadJobsTable.update({ DownloadJobsTable.id eq id }) { DownloadJobsTable.update({ DownloadJobsTable.id eq id }) {
@@ -174,58 +140,77 @@ object DownloadRepository {
} }
} }
fun forUser(userName: String): List<DownloadJobRecord> = transaction { /** A reject must stop a queued or in-flight download. */
DownloadJobsTable.selectAll() fun cancelForMoodVideo(moodId: String, videoId: String, reason: String) {
.where { DownloadJobsTable.userName eq userName }
.orderBy(DownloadJobsTable.createdAt to SortOrder.DESC)
.map { it.toRecord() }
}
fun forMood(userName: String, moodId: String): Map<String, DownloadJobRecord> = transaction {
DownloadJobsTable.selectAll()
.where {
(DownloadJobsTable.userName eq userName) and
(DownloadJobsTable.moodId eq moodId)
}
.associate { it[DownloadJobsTable.videoId] to it.toRecord() }
}
/** A video already downloaded for any mood, if the current user can see it. */
fun completedSongForVideo(videoId: String): String? = transaction {
DownloadJobsTable.selectAll()
.where {
(DownloadJobsTable.videoId eq videoId) and
(DownloadJobsTable.state eq "done")
}
.orderBy(DownloadJobsTable.updatedAt to SortOrder.DESC)
.limit(1)
.singleOrNull()
?.get(DownloadJobsTable.songId)
}
fun forRecommendation(userName: String, moodId: String, videoId: String): DownloadJobRecord? =
transaction { transaction {
DownloadJobsTable.update({
(DownloadJobsTable.moodId eq moodId) and
(DownloadJobsTable.videoId eq videoId) and
(DownloadJobsTable.state inList listOf("queued", "downloading"))
}) {
it[state] = "failed"
it[error] = reason.take(1000)
it[updatedAt] = Instant.now().toString()
}
}
}
fun find(moodId: String, videoId: String, kind: String): DownloadJobRecord? = transaction {
DownloadJobsTable.selectAll() DownloadJobsTable.selectAll()
.where { .where {
(DownloadJobsTable.userName eq userName) and
(DownloadJobsTable.moodId eq moodId) and (DownloadJobsTable.moodId eq moodId) and
(DownloadJobsTable.videoId eq videoId) (DownloadJobsTable.videoId eq videoId) and
(DownloadJobsTable.kind eq kind)
} }
.singleOrNull() .singleOrNull()
?.toRecord() ?.toRecord()
} }
fun forMood(moodId: String): List<DownloadJobRecord> = transaction {
DownloadJobsTable.selectAll()
.where { DownloadJobsTable.moodId eq moodId }
.orderBy(DownloadJobsTable.createdAt to SortOrder.DESC)
.map { it.toRecord() }
}
fun pendingCount(): Long = transaction {
DownloadJobsTable.selectAll()
.where { DownloadJobsTable.state inList listOf("queued", "downloading") }
.count()
}
/**
* A pod restart or an uncaught bookkeeping failure can leave jobs in
* `downloading`. Requeue them after a generous timeout so they are not lost.
*/
private fun recoverStale(now: Instant = Instant.now()) {
val cutoff = now.minusSeconds(STALE_DOWNLOAD_SECONDS).toString()
val stale = DownloadJobsTable.selectAll()
.where {
(DownloadJobsTable.state eq "downloading") and
(DownloadJobsTable.updatedAt less cutoff)
}
.toList()
stale.forEach { row ->
val id = row[DownloadJobsTable.id]
DownloadJobsTable.update({ DownloadJobsTable.id eq id }) {
it[state] = "queued"
it[error] = "Download was interrupted; retrying"
it[updatedAt] = now.toString()
}
}
}
private const val STALE_DOWNLOAD_SECONDS = 30 * 60L private const val STALE_DOWNLOAD_SECONDS = 30 * 60L
private const val MAX_ATTEMPTS = 3
private fun org.jetbrains.exposed.sql.ResultRow.toRecord() = DownloadJobRecord( private fun org.jetbrains.exposed.sql.ResultRow.toRecord() = DownloadJobRecord(
id = this[DownloadJobsTable.id], id = this[DownloadJobsTable.id],
userName = this[DownloadJobsTable.userName],
moodId = this[DownloadJobsTable.moodId],
videoId = this[DownloadJobsTable.videoId], videoId = this[DownloadJobsTable.videoId],
title = this[DownloadJobsTable.title], moodId = this[DownloadJobsTable.moodId],
artist = this[DownloadJobsTable.artist], kind = this[DownloadJobsTable.kind],
state = this[DownloadJobsTable.state], state = this[DownloadJobsTable.state],
priority = this[DownloadJobsTable.priority],
attempts = this[DownloadJobsTable.attempts], attempts = this[DownloadJobsTable.attempts],
error = this[DownloadJobsTable.error], error = this[DownloadJobsTable.error],
songId = this[DownloadJobsTable.songId], songId = this[DownloadJobsTable.songId],
@@ -233,69 +218,3 @@ object DownloadRepository {
updatedAt = this[DownloadJobsTable.updatedAt], updatedAt = this[DownloadJobsTable.updatedAt],
) )
} }
data class LongformProgressRecord(
val songId: String,
val positionSeconds: Int,
val durationSeconds: Int,
val completed: Boolean,
val updatedAt: String,
)
/** Resume points for audiobooks and podcasts. Music never uses this table. */
object LongformRepository {
fun save(
userName: String,
songId: String,
positionSeconds: Int,
durationSeconds: Int,
completed: Boolean,
updatedAt: String = Instant.now().toString(),
) {
transaction {
val exists = LongformProgressTable.selectAll()
.where {
(LongformProgressTable.userName eq userName) and
(LongformProgressTable.songId eq songId)
}
.any()
if (exists) {
LongformProgressTable.update({
(LongformProgressTable.userName eq userName) and
(LongformProgressTable.songId eq songId)
}) {
it[this.positionSeconds] = positionSeconds.coerceAtLeast(0)
it[this.durationSeconds] = durationSeconds.coerceAtLeast(0)
it[this.completed] = completed
it[this.updatedAt] = updatedAt
}
} else {
LongformProgressTable.insertIgnore {
it[this.userName] = userName
it[this.songId] = songId
it[this.positionSeconds] = positionSeconds.coerceAtLeast(0)
it[this.durationSeconds] = durationSeconds.coerceAtLeast(0)
it[this.completed] = completed
it[this.updatedAt] = updatedAt
}
}
}
}
fun forUser(userName: String, limit: Int = 100): List<LongformProgressRecord> = transaction {
LongformProgressTable.selectAll()
.where { LongformProgressTable.userName eq userName }
.orderBy(LongformProgressTable.updatedAt to SortOrder.DESC)
.limit(limit.coerceIn(1, 500))
.map {
LongformProgressRecord(
songId = it[LongformProgressTable.songId],
positionSeconds = it[LongformProgressTable.positionSeconds],
durationSeconds = it[LongformProgressTable.durationSeconds],
completed = it[LongformProgressTable.completed],
updatedAt = it[LongformProgressTable.updatedAt],
)
}
}
}
@@ -1,879 +0,0 @@
package win.taptap.musicmanager.db
import java.nio.file.Files
import java.nio.file.Path
import org.jetbrains.exposed.sql.JoinType
import org.jetbrains.exposed.sql.Op
import org.jetbrains.exposed.sql.SqlExpressionBuilder.greater
import org.jetbrains.exposed.sql.ResultRow
import org.jetbrains.exposed.sql.SortOrder
import org.jetbrains.exposed.sql.Transaction
import org.jetbrains.exposed.sql.and
import org.jetbrains.exposed.sql.lowerCase
import org.jetbrains.exposed.sql.or
import org.jetbrains.exposed.sql.selectAll
import org.jetbrains.exposed.sql.transactions.transaction
import org.jetbrains.exposed.sql.union
import win.taptap.musicmanager.config.LibraryConfig
import win.taptap.musicmanager.util.IdGenerator
data class ArtistRefRecord(val id: String, val name: String)
data class UserItemState(val starredAt: String?, val rating: Int?)
data class UserSongState(
val playCount: Int,
val lastPlayedAt: String?,
val starredAt: String?,
val rating: Int?,
)
data class GenreRecord(val value: String, val songCount: Int, val albumCount: Int)
data class StarredRecords(
val artists: List<ArtistRecord>,
val albums: List<AlbumRecord>,
val songs: List<SongRecord>,
)
data class ArtistRecord(
val id: String,
val name: String,
val sortName: String,
val albumCount: Int,
val starredAt: String?,
val rating: Int?,
)
data class AlbumRecord(
val id: String,
val name: String,
val artistId: String?,
val artistName: String?,
val year: Int?,
val coverArt: String?,
val songCount: Int,
val duration: Int,
val playCount: Int,
val lastPlayedAt: String?,
val created: String,
val starredAt: String?,
val rating: Int?,
val genres: List<String>,
)
data class SongRecord(
val id: String,
val albumId: String,
val albumName: String,
val artistId: String?,
val artistName: String,
val displayArtist: String,
val title: String,
val trackNumber: Int?,
val discNumber: Int,
val year: Int?,
val duration: Int,
val bitRate: Int,
val size: Long,
val suffix: String,
val contentType: String,
val coverArt: String?,
val folderId: Int,
val relativePath: String,
val created: String,
val updatedAt: String,
val starredAt: String?,
val rating: Int?,
val playCount: Int,
val lastPlayedAt: String?,
val artists: List<ArtistRefRecord>,
val genres: List<String>,
)
data class ResolvedSong(val song: SongRecord, val file: Path)
data class TombstoneRecord(val entityId: String, val deletedAt: String)
/**
* Read-side queries for browsing and media. Every method takes the set of folder
* IDs visible to the requesting user; an album or artist without a visible song
* is never returned.
*/
object LibraryRepository {
fun artists(folderIds: Set<Int>, userName: String): List<ArtistRecord> = read {
if (folderIds.isEmpty()) return@read emptyList()
val visibleSongs = visibleSongs(folderIds)
val visibleAlbums = visibleAlbums(folderIds)
val trackArtists = SongArtistsTable
.select(SongArtistsTable.artistId)
.where { SongArtistsTable.songId inSubQuery visibleSongs }
val albumArtists = AlbumArtistsTable
.select(AlbumArtistsTable.artistId)
.where { AlbumArtistsTable.albumId inSubQuery visibleAlbums }
val albumCounts = AlbumArtistsTable
.select(AlbumArtistsTable.artistId, AlbumArtistsTable.albumId)
.withDistinct()
.where { AlbumArtistsTable.albumId inSubQuery visibleAlbums }
.groupingBy { it[AlbumArtistsTable.artistId] }
.eachCount()
val states = itemStates(userName, "artist")
return@read ArtistsTable
.selectAll()
.where { ArtistsTable.id inSubQuery trackArtists.union(albumArtists) }
.map { row ->
val id = row[ArtistsTable.id]
val state = states[id]
ArtistRecord(
id = id,
name = row[ArtistsTable.name],
sortName = row[ArtistsTable.sortName],
albumCount = albumCounts[id] ?: 0,
starredAt = state?.starredAt,
rating = state?.rating,
)
}
.sortedBy { IdGenerator.normalize(it.sortName) }
}
fun artist(artistId: String, folderIds: Set<Int>, userName: String): ArtistRecord? = read {
if (folderIds.isEmpty()) return@read null
val row = ArtistsTable.selectAll()
.where { ArtistsTable.id eq artistId }
.singleOrNull() ?: return@read null
val albumCount = AlbumArtistsTable
.select(AlbumArtistsTable.albumId)
.withDistinct()
.where {
(AlbumArtistsTable.artistId eq artistId) and
(AlbumArtistsTable.albumId inSubQuery visibleAlbums(folderIds))
}
.count()
.toInt()
val hasVisibleSong = SongArtistsTable
.select(SongArtistsTable.songId)
.where {
(SongArtistsTable.artistId eq artistId) and
(SongArtistsTable.songId inSubQuery visibleSongs(folderIds))
}
.any()
if (albumCount == 0 && !hasVisibleSong) return@read null
val state = itemStates(userName, "artist")[artistId]
ArtistRecord(
id = artistId,
name = row[ArtistsTable.name],
sortName = row[ArtistsTable.sortName],
albumCount = albumCount,
starredAt = state?.starredAt,
rating = state?.rating,
)
}
fun albumsForArtist(artistId: String, folderIds: Set<Int>, userName: String): List<AlbumRecord> = read {
if (folderIds.isEmpty()) return@read emptyList()
val albumIds = AlbumArtistsTable
.select(AlbumArtistsTable.albumId)
.withDistinct()
.where {
(AlbumArtistsTable.artistId eq artistId) and
(AlbumArtistsTable.albumId inSubQuery visibleAlbums(folderIds))
}
.map { it[AlbumArtistsTable.albumId] }
albumIds.mapNotNull { albumRecord(it, folderIds, userName) }
.sortedBy { it.name.lowercase() }
}
fun album(albumId: String, folderIds: Set<Int>, userName: String): AlbumRecord? = read {
albumRecord(albumId, folderIds, userName)
}
fun albumSongs(albumId: String, folderIds: Set<Int>, userName: String): List<SongRecord> = read {
if (folderIds.isEmpty()) return@read emptyList()
val rows = SongsTable.selectAll()
.where {
(SongsTable.albumId eq albumId) and
(SongsTable.folderId inList folderIds)
}
.orderBy(
SongsTable.discNumber to SortOrder.ASC,
SongsTable.trackNumber to SortOrder.ASC,
SongsTable.title to SortOrder.ASC,
)
.toList()
enrichSongs(rows, userName)
}
fun song(songId: String, folderIds: Set<Int>, userName: String): SongRecord? = read {
if (folderIds.isEmpty()) return@read null
val row = SongsTable.selectAll()
.where {
(SongsTable.id eq songId) and
(SongsTable.folderId inList folderIds)
}
.singleOrNull() ?: return@read null
enrichSongs(listOf(row), userName).single()
}
fun allAlbums(limit: Int, offset: Int, folderIds: Set<Int>, userName: String): List<AlbumRecord> = read {
if (folderIds.isEmpty()) return@read emptyList()
val ids = AlbumsTable
.select(AlbumsTable.id)
.where { AlbumsTable.id inSubQuery visibleAlbums(folderIds) }
.orderBy(AlbumsTable.name to SortOrder.ASC)
.limit(limit)
.offset(offset.toLong())
.map { it[AlbumsTable.id] }
ids.mapNotNull { albumRecord(it, folderIds, userName) }
}
/** Stable page of the library snapshot, ordered by song ID. */
fun syncSongs(
folderIds: Set<Int>,
afterId: String,
limit: Int,
userName: String,
): List<SongRecord> = read {
if (folderIds.isEmpty()) return@read emptyList()
val rows = SongsTable.selectAll()
.where {
(SongsTable.folderId inList folderIds) and
(SongsTable.id greater afterId)
}
.orderBy(SongsTable.id to SortOrder.ASC)
.limit(limit)
.toList()
enrichSongs(rows, userName)
}
/** Songs whose metadata changed after the client's last revision. */
fun changedSongs(
since: String,
folderIds: Set<Int>,
limit: Int,
userName: String,
): List<SongRecord> = read {
if (folderIds.isEmpty()) return@read emptyList()
val rows = SongsTable.selectAll()
.where {
(SongsTable.folderId inList folderIds) and
(SongsTable.updatedAt greater since)
}
.orderBy(
SongsTable.updatedAt to SortOrder.ASC,
SongsTable.id to SortOrder.ASC,
)
.limit(limit)
.toList()
enrichSongs(rows, userName)
}
fun tombstones(since: String, limit: Int): List<TombstoneRecord> = read {
LibraryTombstonesTable.selectAll()
.where {
(LibraryTombstonesTable.entity eq "song") and
(LibraryTombstonesTable.deletedAt greater since)
}
.orderBy(LibraryTombstonesTable.deletedAt to SortOrder.ASC)
.limit(limit)
.map {
TombstoneRecord(
entityId = it[LibraryTombstonesTable.entityId],
deletedAt = it[LibraryTombstonesTable.deletedAt],
)
}
}
fun allSongs(limit: Int, offset: Int, folderIds: Set<Int>, userName: String): List<SongRecord> = read {
if (folderIds.isEmpty()) return@read emptyList()
val rows = SongsTable.selectAll()
.where { SongsTable.folderId inList folderIds }
.orderBy(SongsTable.title to SortOrder.ASC)
.limit(limit)
.offset(offset.toLong())
.toList()
enrichSongs(rows, userName)
}
/**
* Album feed used by getAlbumList2. Sorting happens after visibility filtering so
* per-user play counts and stars are correct.
*/
fun albumList(
type: String,
size: Int,
offset: Int,
fromYear: Int?,
toYear: Int?,
genre: String?,
folderIds: Set<Int>,
userName: String,
): List<AlbumRecord> = read {
if (folderIds.isEmpty()) return@read emptyList()
val ids = AlbumsTable
.select(AlbumsTable.id)
.where { AlbumsTable.id inSubQuery visibleAlbums(folderIds) }
.map { it[AlbumsTable.id] }
// Clients may send the range inverted (e.g. descending by year), so normalize it.
val lowerYear = when {
fromYear != null && toYear != null && fromYear > toYear -> toYear
else -> fromYear
}
val upperYear = when {
fromYear != null && toYear != null && fromYear > toYear -> fromYear
else -> toYear
}
var albums = ids.mapNotNull { albumRecord(it, folderIds, userName) }
if (lowerYear != null) albums = albums.filter { (it.year ?: Int.MIN_VALUE) >= lowerYear }
if (upperYear != null) albums = albums.filter { (it.year ?: Int.MAX_VALUE) <= upperYear }
if (!genre.isNullOrBlank()) {
albums = albums.filter { album -> album.genres.any { it.equals(genre, ignoreCase = true) } }
}
val sorted = when (type) {
"newest" -> albums.sortedByDescending { it.created }
"recent" -> albums.sortedWith(
compareByDescending<AlbumRecord> { it.lastPlayedAt ?: "" }
.thenByDescending { it.created },
)
"frequent" -> albums.sortedWith(
compareByDescending<AlbumRecord> { it.playCount }
.thenBy { IdGenerator.normalize(it.name) },
)
"random" -> albums.shuffled()
"alphabeticalByName" -> albums.sortedBy { IdGenerator.normalize(it.name) }
"alphabeticalByArtist" -> albums.sortedWith(
compareBy(
{ IdGenerator.normalize(it.artistName.orEmpty()) },
{ IdGenerator.normalize(it.name) },
),
)
"starred" -> albums
.filter { it.starredAt != null }
.sortedByDescending { it.starredAt }
"byYear" -> albums.sortedWith(
compareByDescending<AlbumRecord> { it.year ?: 0 }
.thenBy { IdGenerator.normalize(it.name) },
)
"byGenre" -> albums.sortedBy { IdGenerator.normalize(it.name) }
else -> albums.sortedBy { IdGenerator.normalize(it.name) }
}
sorted.drop(offset).take(size)
}
fun randomSongs(
size: Int,
genre: String?,
fromYear: Int?,
toYear: Int?,
folderIds: Set<Int>,
userName: String,
): List<SongRecord> = read {
if (folderIds.isEmpty()) return@read emptyList()
val lowerYear = when {
fromYear != null && toYear != null && fromYear > toYear -> toYear
else -> fromYear
}
val upperYear = when {
fromYear != null && toYear != null && fromYear > toYear -> fromYear
else -> toYear
}
val rows = SongsTable.selectAll()
.where {
(SongsTable.folderId inList folderIds) and
(lowerYear?.let { SongsTable.year greaterEq it } ?: Op.TRUE) and
(upperYear?.let { SongsTable.year lessEq it } ?: Op.TRUE)
}
.toList()
var songs = enrichSongs(rows, userName)
if (!genre.isNullOrBlank()) {
songs = songs.filter { song -> song.genres.any { it.equals(genre, ignoreCase = true) } }
}
songs.shuffled().take(size)
}
fun genres(folderIds: Set<Int>): List<GenreRecord> = read {
if (folderIds.isEmpty()) return@read emptyList()
val rows = SongGenresTable
.join(SongsTable, JoinType.INNER, SongGenresTable.songId, SongsTable.id)
.select(SongGenresTable.genre, SongsTable.id, SongsTable.albumId)
.where { SongsTable.folderId inList folderIds }
.toList()
rows.groupBy { it[SongGenresTable.genre] }
.map { (genre, genreRows) ->
GenreRecord(
value = genre,
songCount = genreRows.map { it[SongsTable.id] }.distinct().size,
albumCount = genreRows.map { it[SongsTable.albumId] }.distinct().size,
)
}
.sortedBy { IdGenerator.normalize(it.value) }
}
fun songsByGenre(
genre: String,
size: Int,
offset: Int,
folderIds: Set<Int>,
userName: String,
): List<SongRecord> = read {
if (folderIds.isEmpty()) return@read emptyList()
val songIds = SongGenresTable
.select(SongGenresTable.songId)
.where { SongGenresTable.genre.lowerCase() eq genre.lowercase() }
val rows = SongsTable.selectAll()
.where {
(SongsTable.id inSubQuery songIds) and
(SongsTable.folderId inList folderIds)
}
.orderBy(SongsTable.title to SortOrder.ASC)
.limit(size)
.offset(offset.toLong())
.toList()
enrichSongs(rows, userName)
}
/** Bulk genre lookup used by mood generation to avoid one query per seed genre. */
fun songsByGenres(
genres: Collection<String>,
limit: Int,
folderIds: Set<Int>,
userName: String,
): List<SongRecord> = read {
if (genres.isEmpty() || folderIds.isEmpty()) return@read emptyList()
val normalized = genres.map { it.lowercase() }.distinct()
val songIds = SongGenresTable
.select(SongGenresTable.songId)
.where { SongGenresTable.genre.lowerCase() inList normalized }
val rows = SongsTable.selectAll()
.where {
(SongsTable.id inSubQuery songIds) and
(SongsTable.folderId inList folderIds)
}
.limit(limit)
.toList()
enrichSongs(rows, userName)
}
/** Songs by track artist or album artist, used as related-mood candidates. */
fun songsByArtists(
artistIds: Collection<String>,
limit: Int,
folderIds: Set<Int>,
userName: String,
): List<SongRecord> = read {
if (artistIds.isEmpty() || folderIds.isEmpty()) return@read emptyList()
val ids = artistIds.toList()
val songArtists = SongArtistsTable
.select(SongArtistsTable.songId)
.where { SongArtistsTable.artistId inList ids }
val albumArtists = AlbumArtistsTable
.select(AlbumArtistsTable.albumId)
.where { AlbumArtistsTable.artistId inList ids }
val rows = SongsTable.selectAll()
.where {
(SongsTable.folderId inList folderIds) and
((SongsTable.id inSubQuery songArtists) or (SongsTable.albumId inSubQuery albumArtists))
}
.limit(limit)
.toList()
enrichSongs(rows, userName)
}
fun starred(userName: String, folderIds: Set<Int>): StarredRecords = read {
if (folderIds.isEmpty()) {
return@read StarredRecords(emptyList(), emptyList(), emptyList())
}
val albumStates = itemStates(userName, "album")
val albums = albumStates
.filterValues { it.starredAt != null }
.keys
.mapNotNull { albumRecord(it, folderIds, userName) }
.sortedByDescending { it.starredAt }
val artistStates = itemStates(userName, "artist")
val artists = artistStates
.filterValues { it.starredAt != null }
.keys
.mapNotNull { artist(it, folderIds, userName) }
.sortedByDescending { it.starredAt }
val songStates = UserSongStateTable.selectAll()
.where {
(UserSongStateTable.userName eq userName) and
UserSongStateTable.starredAt.isNotNull()
}
.toList()
val songRows = if (songStates.isEmpty()) {
emptyList()
} else {
SongsTable.selectAll()
.where {
(SongsTable.id inList songStates.map { it[UserSongStateTable.songId] }) and
(SongsTable.folderId inList folderIds)
}
.toList()
}
val songs = enrichSongs(songRows, userName)
.sortedByDescending { it.starredAt }
StarredRecords(artists, albums, songs)
}
fun lyrics(songId: String, folderIds: Set<Int>): String? = read {
if (folderIds.isEmpty()) return@read null
SongsTable
.select(SongsTable.lyrics)
.where {
(SongsTable.id eq songId) and
(SongsTable.folderId inList folderIds)
}
.singleOrNull()
?.get(SongsTable.lyrics)
}
fun songsByIds(ids: List<String>, folderIds: Set<Int>, userName: String): List<SongRecord> = read {
if (ids.isEmpty() || folderIds.isEmpty()) return@read emptyList()
val rows = SongsTable.selectAll()
.where {
(SongsTable.id inList ids) and
(SongsTable.folderId inList folderIds)
}
.toList()
val byId = enrichSongs(rows, userName).associateBy { it.id }
ids.mapNotNull { byId[it] }
}
fun searchArtists(
query: String,
limit: Int,
offset: Int,
folderIds: Set<Int>,
userName: String,
): List<ArtistRecord> = read {
if (folderIds.isEmpty()) return@read emptyList()
val pattern = likePattern(query)
val visibleSongs = visibleSongs(folderIds)
val visibleAlbums = visibleAlbums(folderIds)
val trackArtists = SongArtistsTable.select(SongArtistsTable.artistId)
.where { SongArtistsTable.songId inSubQuery visibleSongs }
val albumArtists = AlbumArtistsTable.select(AlbumArtistsTable.artistId)
.where { AlbumArtistsTable.albumId inSubQuery visibleAlbums }
val ids = ArtistsTable
.select(ArtistsTable.id)
.where {
(ArtistsTable.searchText like pattern) and
(ArtistsTable.id inSubQuery trackArtists.union(albumArtists))
}
.orderBy(ArtistsTable.sortName to SortOrder.ASC)
.limit(limit)
.offset(offset.toLong())
.map { it[ArtistsTable.id] }
ids.mapNotNull { artist(it, folderIds, userName) }
}
fun searchAlbums(
query: String,
limit: Int,
offset: Int,
folderIds: Set<Int>,
userName: String,
): List<AlbumRecord> = read {
if (folderIds.isEmpty()) return@read emptyList()
val pattern = likePattern(query)
val ids = AlbumsTable
.select(AlbumsTable.id)
.where {
(AlbumsTable.searchText like pattern) and
(AlbumsTable.id inSubQuery visibleAlbums(folderIds))
}
.orderBy(AlbumsTable.name to SortOrder.ASC)
.limit(limit)
.offset(offset.toLong())
.map { it[AlbumsTable.id] }
ids.mapNotNull { albumRecord(it, folderIds, userName) }
}
fun searchSongs(
query: String,
limit: Int,
offset: Int,
folderIds: Set<Int>,
userName: String,
): List<SongRecord> = read {
if (folderIds.isEmpty()) return@read emptyList()
val pattern = likePattern(query)
// "%" or "_" alone normalizes to an empty LIKE needle and would match
// the whole library.
if (pattern == "%%") return@read emptyList()
val rows = SongsTable.selectAll()
.where {
(SongsTable.searchText like pattern) and
(SongsTable.folderId inList folderIds)
}
.orderBy(SongsTable.title to SortOrder.ASC)
.limit(limit)
.offset(offset.toLong())
.toList()
enrichSongs(rows, userName)
}
fun resolveSong(
songId: String,
folderIds: Set<Int>,
userName: String,
library: LibraryConfig,
): ResolvedSong? {
val song = song(songId, folderIds, userName) ?: return null
val folder = library.byId(song.folderId) ?: return null
val file = folder.path.resolve(song.relativePath).normalize()
if (!file.startsWith(folder.path)) return null
if (!Files.isRegularFile(file)) return null
return ResolvedSong(song, file)
}
fun albumIdForSong(songId: String, folderIds: Set<Int>): String? = read {
if (folderIds.isEmpty()) return@read null
SongsTable
.select(SongsTable.albumId)
.where {
(SongsTable.id eq songId) and
(SongsTable.folderId inList folderIds)
}
.singleOrNull()
?.get(SongsTable.albumId)
}
private fun albumRecord(albumId: String, folderIds: Set<Int>, userName: String): AlbumRecord? {
if (folderIds.isEmpty()) return null
val albumRow = AlbumsTable.selectAll()
.where { AlbumsTable.id eq albumId }
.singleOrNull() ?: return null
val songs = SongsTable.selectAll()
.where {
(SongsTable.albumId eq albumId) and
(SongsTable.folderId inList folderIds)
}
.toList()
if (songs.isEmpty()) return null
val primaryArtist = AlbumArtistsTable
.join(ArtistsTable, JoinType.INNER, AlbumArtistsTable.artistId, ArtistsTable.id)
.select(AlbumArtistsTable.albumId, ArtistsTable.id, ArtistsTable.name, AlbumArtistsTable.position)
.where { AlbumArtistsTable.albumId eq albumId }
.orderBy(AlbumArtistsTable.position to SortOrder.ASC)
.firstOrNull()
val states = songStates(userName, songs.map { it[SongsTable.id] })
val playCount = states.values.sumOf { it.playCount }
val lastPlayedAt = states.values.mapNotNull { it.lastPlayedAt }.maxOrNull()
val state = itemStates(userName, "album")[albumId]
return AlbumRecord(
id = albumId,
name = albumRow[AlbumsTable.name],
artistId = primaryArtist?.get(ArtistsTable.id),
artistName = primaryArtist?.get(ArtistsTable.name),
year = albumRow[AlbumsTable.year],
coverArt = albumRow[AlbumsTable.coverArt],
songCount = songs.size,
duration = songs.sumOf { it[SongsTable.duration] },
playCount = playCount,
lastPlayedAt = lastPlayedAt,
created = albumRow[AlbumsTable.createdAt],
starredAt = state?.starredAt,
rating = state?.rating,
genres = AlbumGenresTable
.select(AlbumGenresTable.genre)
.where { AlbumGenresTable.albumId eq albumId }
.map { it[AlbumGenresTable.genre] }
.sorted(),
)
}
private fun enrichSongs(rows: List<ResultRow>, userName: String): List<SongRecord> {
if (rows.isEmpty()) return emptyList()
val songIds = rows.map { it[SongsTable.id] }
val albumIds = rows.map { it[SongsTable.albumId] }.distinct()
val songArtists = SongArtistsTable
.join(ArtistsTable, JoinType.INNER, SongArtistsTable.artistId, ArtistsTable.id)
.select(SongArtistsTable.songId, ArtistsTable.id, ArtistsTable.name, SongArtistsTable.position)
.where { SongArtistsTable.songId inList songIds }
.orderBy(SongArtistsTable.position to SortOrder.ASC)
.groupBy { it[SongArtistsTable.songId] }
.mapValues { (_, artistRows) ->
artistRows.map { ArtistRefRecord(it[ArtistsTable.id], it[ArtistsTable.name]) }
}
val albumArtists = AlbumArtistsTable
.join(ArtistsTable, JoinType.INNER, AlbumArtistsTable.artistId, ArtistsTable.id)
.select(AlbumArtistsTable.albumId, ArtistsTable.id, ArtistsTable.name, AlbumArtistsTable.position)
.where { AlbumArtistsTable.albumId inList albumIds }
.orderBy(AlbumArtistsTable.position to SortOrder.ASC)
.groupBy { it[AlbumArtistsTable.albumId] }
.mapValues { (_, artistRows) ->
ArtistRefRecord(artistRows.first()[ArtistsTable.id], artistRows.first()[ArtistsTable.name])
}
val covers = AlbumsTable.selectAll()
.where { AlbumsTable.id inList albumIds }
.associate { it[AlbumsTable.id] to it[AlbumsTable.coverArt] }
val genres = SongGenresTable.selectAll()
.where { SongGenresTable.songId inList songIds }
.groupBy { it[SongGenresTable.songId] }
.mapValues { (_, genreRows) -> genreRows.map { it[SongGenresTable.genre] }.sorted() }
val states = songStates(userName, songIds)
return rows.map { row ->
val id = row[SongsTable.id]
val albumId = row[SongsTable.albumId]
val primaryArtist = albumArtists[albumId]
val state = states[id]
SongRecord(
id = id,
albumId = albumId,
albumName = row[SongsTable.albumName],
artistId = primaryArtist?.id,
artistName = row[SongsTable.displayArtist],
displayArtist = row[SongsTable.displayArtist],
title = row[SongsTable.title],
trackNumber = row[SongsTable.trackNumber],
discNumber = row[SongsTable.discNumber],
year = row[SongsTable.year],
duration = row[SongsTable.duration],
bitRate = row[SongsTable.bitRate],
size = row[SongsTable.size],
suffix = row[SongsTable.suffix],
contentType = row[SongsTable.contentType],
coverArt = covers[albumId],
folderId = row[SongsTable.folderId],
relativePath = row[SongsTable.relativePath],
created = row[SongsTable.createdAt],
updatedAt = row[SongsTable.updatedAt].ifBlank { row[SongsTable.createdAt] },
starredAt = state?.starredAt,
rating = state?.rating,
playCount = state?.playCount ?: 0,
lastPlayedAt = state?.lastPlayedAt,
artists = songArtists[id].orEmpty(),
genres = genres[id].orEmpty(),
)
}
}
private fun visibleSongs(folderIds: Set<Int>) =
SongsTable
.select(SongsTable.id)
.where { SongsTable.folderId inList folderIds }
private fun visibleAlbums(folderIds: Set<Int>) =
SongsTable
.select(SongsTable.albumId)
.withDistinct()
.where { SongsTable.folderId inList folderIds }
private fun itemStates(userName: String, itemType: String): Map<String, UserItemState> =
UserStarsTable.selectAll()
.where {
(UserStarsTable.userName eq userName) and
(UserStarsTable.itemType eq itemType)
}
.associate {
it[UserStarsTable.itemId] to UserItemState(
starredAt = it[UserStarsTable.starredAt].takeIf { value -> value.isNotBlank() },
rating = it[UserStarsTable.rating],
)
}
private fun songStates(userName: String, songIds: Collection<String>): Map<String, UserSongState> {
if (songIds.isEmpty()) return emptyMap()
return UserSongStateTable.selectAll()
.where {
(UserSongStateTable.userName eq userName) and
(UserSongStateTable.songId inList songIds.toList())
}
.associate {
it[UserSongStateTable.songId] to UserSongState(
playCount = it[UserSongStateTable.playCount],
lastPlayedAt = it[UserSongStateTable.lastPlayedAt],
starredAt = it[UserSongStateTable.starredAt],
rating = it[UserSongStateTable.rating],
)
}
}
/** Literal substring matching; `%` and `_` are removed instead of acting as wildcards. */
private fun likePattern(query: String): String {
val needle = IdGenerator.normalize(query).replace("%", "").replace("_", "")
return "%$needle%"
}
private fun <T> read(block: Transaction.() -> T): T = transaction { block() }
}
@@ -1,261 +0,0 @@
package win.taptap.musicmanager.db
import java.time.Instant
import org.jetbrains.exposed.sql.SqlExpressionBuilder.eq
import org.jetbrains.exposed.sql.and
import org.jetbrains.exposed.sql.deleteWhere
import org.jetbrains.exposed.sql.insert
import org.jetbrains.exposed.sql.insertIgnore
import org.jetbrains.exposed.sql.selectAll
import org.jetbrains.exposed.sql.transactions.transaction
import org.jetbrains.exposed.sql.update
import win.taptap.musicmanager.scanner.ArtistRef
import win.taptap.musicmanager.scanner.ExtractedTrack
import win.taptap.musicmanager.util.IdGenerator
import win.taptap.musicmanager.util.SortNames
data class ExistingSong(
val id: String,
val fileMtime: Long,
val size: Long,
)
/** Persistence for the scanner. Per-user state is never touched when metadata is updated. */
object MediaRepository {
fun findExisting(folderId: Int, relativePath: String): ExistingSong? = transaction {
SongsTable.selectAll()
.where {
(SongsTable.folderId eq folderId) and (SongsTable.relativePath eq relativePath)
}
.singleOrNull()
?.let {
ExistingSong(
id = it[SongsTable.id],
fileMtime = it[SongsTable.fileMtime],
size = it[SongsTable.size],
)
}
}
fun upsertTrack(track: ExtractedTrack) {
val now = Instant.now().toString()
transaction {
(track.albumArtists + track.artists).distinctBy { it.id }.forEach(::upsertArtist)
val albumSearch = IdGenerator.normalize(
"${track.albumTitle} ${track.albumArtists.joinToString(" ") { it.name }}",
)
val albumExists = AlbumsTable.selectAll()
.where { AlbumsTable.id eq track.albumId }
.any()
if (albumExists) {
AlbumsTable.update({ AlbumsTable.id eq track.albumId }) {
it[name] = track.albumTitle
if (track.year != null) it[year] = track.year
it[searchText] = albumSearch
}
} else {
AlbumsTable.insert {
it[id] = track.albumId
it[name] = track.albumTitle
it[year] = track.year
it[createdAt] = now
it[searchText] = albumSearch
}
}
AlbumArtistsTable.deleteWhere {
it.run { AlbumArtistsTable.albumId eq track.albumId }
}
track.albumArtists.forEachIndexed { index, artist ->
AlbumArtistsTable.insertIgnore {
it[albumId] = track.albumId
it[artistId] = artist.id
it[position] = index
}
}
val existingByPath = SongsTable.selectAll()
.where {
(SongsTable.folderId eq track.folderId) and
(SongsTable.relativePath eq track.relativePath)
}
.singleOrNull()
val existingId = existingByPath?.get(SongsTable.id)
if (existingId != null && existingId != track.songId) {
SongsTable.deleteWhere { it.run { SongsTable.id eq existingId } }
}
val songExists = existingId == track.songId ||
SongsTable.selectAll().where { SongsTable.id eq track.songId }.any()
if (songExists) {
SongsTable.update({ SongsTable.id eq track.songId }) {
it[albumId] = track.albumId
it[folderId] = track.folderId
it[title] = track.title
it[displayArtist] = track.displayArtist
it[albumName] = track.albumTitle
it[trackNumber] = track.trackNumber
it[discNumber] = track.discNumber
it[year] = track.year
it[duration] = track.durationSeconds
it[bitRate] = track.bitRate
it[size] = track.size
it[suffix] = track.suffix
it[contentType] = track.contentType
it[relativePath] = track.relativePath
it[fileMtime] = track.fileMtime
it[searchText] = track.searchText
it[updatedAt] = now
it[lyrics] = track.lyrics
}
} else {
SongsTable.insert {
it[id] = track.songId
it[albumId] = track.albumId
it[folderId] = track.folderId
it[title] = track.title
it[displayArtist] = track.displayArtist
it[albumName] = track.albumTitle
it[trackNumber] = track.trackNumber
it[discNumber] = track.discNumber
it[year] = track.year
it[duration] = track.durationSeconds
it[bitRate] = track.bitRate
it[size] = track.size
it[suffix] = track.suffix
it[contentType] = track.contentType
it[relativePath] = track.relativePath
it[fileMtime] = track.fileMtime
it[createdAt] = now
it[searchText] = track.searchText
it[updatedAt] = now
it[lyrics] = track.lyrics
}
}
// A re-added song must not stay tombstoned for clients that still
// remember the older revision.
LibraryTombstonesTable.deleteWhere {
it.run { (entity eq "song") and (entityId eq track.songId) }
}
SongArtistsTable.deleteWhere { it.run { SongArtistsTable.songId eq track.songId } }
track.artists.forEachIndexed { index, artist ->
SongArtistsTable.insertIgnore {
it[songId] = track.songId
it[artistId] = artist.id
it[position] = index
}
}
SongGenresTable.deleteWhere { it.run { SongGenresTable.songId eq track.songId } }
track.genres.forEach { genre ->
SongGenresTable.insertIgnore {
it[songId] = track.songId
it[this.genre] = genre
}
}
}
}
fun setAlbumCover(albumId: String) {
transaction {
AlbumsTable.update({ AlbumsTable.id eq albumId }) {
it[coverArt] = albumId
}
}
}
fun deleteSongsNotIn(seenSongIds: Set<String>) {
transaction {
val obsolete = SongsTable.selectAll()
.map { it[SongsTable.id] }
.filterNot { it in seenSongIds }
val base = Instant.now()
obsolete.forEachIndexed { index, songId ->
// Unique timestamps let timestamp cursors advance even when a
// scan deletes hundreds of songs at once.
val timestamp = base.plusNanos(index.toLong()).toString()
LibraryTombstonesTable.insertIgnore {
it[entity] = "song"
it[entityId] = songId
it[deletedAt] = timestamp
}
// insertIgnore keeps the old timestamp on a re-deletion, which
// would make the second deletion invisible to synced clients.
LibraryTombstonesTable.update({
(LibraryTombstonesTable.entity eq "song") and
(LibraryTombstonesTable.entityId eq songId)
}) {
it[deletedAt] = timestamp
}
SongsTable.deleteWhere { it.run { SongsTable.id eq songId } }
}
exec("DELETE FROM albums WHERE id NOT IN (SELECT DISTINCT album_id FROM songs)")
exec(
"""
DELETE FROM artists WHERE id NOT IN (
SELECT artist_id FROM song_artists
UNION
SELECT artist_id FROM album_artists
)
""".trimIndent(),
)
}
}
/** Recomputes album/artist genre aggregates from the song-level tags. */
fun rebuildAggregates() {
transaction {
exec("DELETE FROM album_genres")
exec(
"""
INSERT INTO album_genres (album_id, genre)
SELECT DISTINCT s.album_id, sg.genre
FROM song_genres sg
JOIN songs s ON s.id = sg.song_id
""".trimIndent(),
)
exec("DELETE FROM artist_genres")
exec(
"""
INSERT INTO artist_genres (artist_id, genre)
SELECT DISTINCT sa.artist_id, sg.genre
FROM song_artists sa
JOIN song_genres sg ON sg.song_id = sa.song_id
UNION
SELECT DISTINCT aa.artist_id, ag.genre
FROM album_artists aa
JOIN album_genres ag ON ag.album_id = aa.album_id
""".trimIndent(),
)
}
}
fun countSongs(): Long = transaction {
SongsTable.selectAll().count()
}
private fun upsertArtist(artist: ArtistRef) {
ArtistsTable.insertIgnore {
it[id] = artist.id
it[name] = artist.name
it[sortName] = SortNames.of(artist.name)
it[searchText] = IdGenerator.normalize(artist.name)
}
ArtistsTable.update({ ArtistsTable.id eq artist.id }) {
it[name] = artist.name
it[sortName] = SortNames.of(artist.name)
it[searchText] = IdGenerator.normalize(artist.name)
}
}
}
@@ -9,6 +9,8 @@ import org.jetbrains.exposed.sql.ResultRow
import org.jetbrains.exposed.sql.SortOrder import org.jetbrains.exposed.sql.SortOrder
import org.jetbrains.exposed.sql.SqlExpressionBuilder.eq import org.jetbrains.exposed.sql.SqlExpressionBuilder.eq
import org.jetbrains.exposed.sql.SqlExpressionBuilder.inList import org.jetbrains.exposed.sql.SqlExpressionBuilder.inList
import org.jetbrains.exposed.sql.SqlExpressionBuilder.inSubQuery
import org.jetbrains.exposed.sql.SqlExpressionBuilder.notInList
import org.jetbrains.exposed.sql.and import org.jetbrains.exposed.sql.and
import org.jetbrains.exposed.sql.deleteWhere import org.jetbrains.exposed.sql.deleteWhere
import org.jetbrains.exposed.sql.insert import org.jetbrains.exposed.sql.insert
@@ -25,19 +27,20 @@ data class MoodRecord(
val description: String?, val description: String?,
val query: String?, val query: String?,
val genres: List<String>, val genres: List<String>,
val targetSize: Int, val collectionPercent: Int,
val seedPercent: Int = 50, val recommendationCount: Int,
val refreshHours: Int, val refreshHours: Int,
val ytEnabled: Boolean,
val lastDiscoveryAt: String?, val lastDiscoveryAt: String?,
val createdAt: String, val createdAt: String,
val changedAt: String, val changedAt: String,
val lastError: String?, val lastError: String?,
val seedCount: Int, val songCount: Int,
val newRecommendations: Int, val recommendationReady: Int,
val recommendationDownloading: Int,
val recommendationFailed: Int,
) )
data class MoodSeedRecord( data class MoodSongRecord(
val songId: String, val songId: String,
val origin: String, val origin: String,
val addedAt: String, val addedAt: String,
@@ -50,7 +53,6 @@ data class RecommendationUpsert(
val album: String?, val album: String?,
val duration: Int?, val duration: Int?,
val thumbnail: String?, val thumbnail: String?,
val matchedSongId: String?,
) )
data class RecommendationRecord( data class RecommendationRecord(
@@ -61,21 +63,31 @@ data class RecommendationRecord(
val duration: Int?, val duration: Int?,
val thumbnail: String?, val thumbnail: String?,
val state: String, val state: String,
val matchedSongId: String?, val songId: String?,
val timesAdded: Int, val attempts: Int,
val error: String?,
val rank: Int,
val firstSeen: String, val firstSeen: String,
val lastSeen: String, val lastSeen: String,
) )
/** Result of applying a finished download to a recommendation. */
enum class RecommendationCompletion { READY, ALREADY_ADDED, REJECTED, MISSING }
/** /**
* Moods are the product's only collection. Seeds are permanent anchors; the * Moods are the product's only collection. Songs are permanent entries; the
* play queue is generated by [win.taptap.musicmanager.radio.RadioService] for * recommendation table also holds hidden `new` candidates and the lifecycle of
* every playback session instead of being stored as a playlist. * the active slots. The play queue is generated by [win.taptap.musicmanager.radio.RadioService]
* for every playback session instead of being stored as a playlist.
*/ */
object MoodRepository { object MoodRepository {
private val genreSerializer = ListSerializer(String.serializer()) private val genreSerializer = ListSerializer(String.serializer())
private val ACTIVE_STATES = listOf("downloading", "ready", "failed")
private val VISIBLE_STATES = ACTIVE_STATES
private val HIDDEN_DECISIONS = listOf("added", "rejected")
fun list(owner: String): List<MoodRecord> = transaction { fun list(owner: String): List<MoodRecord> = transaction {
MoodsTable.selectAll() MoodsTable.selectAll()
.where { MoodsTable.owner eq owner } .where { MoodsTable.owner eq owner }
@@ -83,6 +95,12 @@ object MoodRepository {
.map { moodRecord(it) } .map { moodRecord(it) }
} }
fun all(): List<MoodRecord> = transaction {
MoodsTable.selectAll()
.orderBy(MoodsTable.changedAt to SortOrder.DESC)
.map { moodRecord(it) }
}
fun get(id: String, owner: String): MoodRecord? = transaction { fun get(id: String, owner: String): MoodRecord? = transaction {
MoodsTable.selectAll() MoodsTable.selectAll()
.where { (MoodsTable.id eq id) and (MoodsTable.owner eq owner) } .where { (MoodsTable.id eq id) and (MoodsTable.owner eq owner) }
@@ -96,11 +114,9 @@ object MoodRepository {
description: String?, description: String?,
query: String?, query: String?,
genres: List<String>, genres: List<String>,
targetSize: Int, collectionPercent: Int,
recommendationCount: Int,
refreshHours: Int, refreshHours: Int,
ytEnabled: Boolean,
seedSongIds: List<String>,
seedPercent: Int = 50,
): MoodRecord = transaction { ): MoodRecord = transaction {
val now = Instant.now().toString() val now = Instant.now().toString()
val id = IdGenerator.moodId(owner, name, now) val id = IdGenerator.moodId(owner, name, now)
@@ -112,19 +128,14 @@ object MoodRepository {
it[MoodsTable.description] = description?.take(2000) it[MoodsTable.description] = description?.take(2000)
it[MoodsTable.query] = query?.trim()?.takeIf(String::isNotEmpty) it[MoodsTable.query] = query?.trim()?.takeIf(String::isNotEmpty)
it[MoodsTable.genres] = encodeGenres(genres) it[MoodsTable.genres] = encodeGenres(genres)
it[MoodsTable.targetSize] = targetSize.coerceIn(5, 500) it[MoodsTable.collectionPercent] = collectionPercent.coerceIn(0, 100)
it[MoodsTable.seedPercent] = seedPercent.coerceIn(0, 100) it[MoodsTable.recommendationCount] = recommendationCount.coerceIn(0, 30)
it[MoodsTable.refreshHours] = refreshHours.coerceIn(1, 720) it[MoodsTable.refreshHours] = refreshHours.coerceIn(1, 720)
it[MoodsTable.ytEnabled] = ytEnabled
it[createdAt] = now it[createdAt] = now
it[changedAt] = now it[changedAt] = now
} }
insertSeeds(id, seedSongIds, "manual") moodRecord(MoodsTable.selectAll().where { MoodsTable.id eq id }.single())
moodRecord(
MoodsTable.selectAll().where { MoodsTable.id eq id }.single(),
)
} }
fun update( fun update(
@@ -134,10 +145,9 @@ object MoodRepository {
description: String?, description: String?,
query: String?, query: String?,
genres: List<String>, genres: List<String>,
targetSize: Int, collectionPercent: Int,
recommendationCount: Int,
refreshHours: Int, refreshHours: Int,
ytEnabled: Boolean,
seedPercent: Int,
): MoodRecord? = transaction { ): MoodRecord? = transaction {
val exists = MoodsTable.selectAll() val exists = MoodsTable.selectAll()
.where { (MoodsTable.id eq id) and (MoodsTable.owner eq owner) } .where { (MoodsTable.id eq id) and (MoodsTable.owner eq owner) }
@@ -149,11 +159,10 @@ object MoodRepository {
it[MoodsTable.description] = description?.take(2000) it[MoodsTable.description] = description?.take(2000)
it[MoodsTable.query] = query?.trim()?.takeIf(String::isNotEmpty) it[MoodsTable.query] = query?.trim()?.takeIf(String::isNotEmpty)
it[MoodsTable.genres] = encodeGenres(genres) it[MoodsTable.genres] = encodeGenres(genres)
it[MoodsTable.targetSize] = targetSize.coerceIn(5, 500) it[MoodsTable.collectionPercent] = collectionPercent.coerceIn(0, 100)
it[MoodsTable.seedPercent] = seedPercent.coerceIn(0, 100) it[MoodsTable.recommendationCount] = recommendationCount.coerceIn(0, 30)
it[MoodsTable.refreshHours] = refreshHours.coerceIn(1, 720) it[MoodsTable.refreshHours] = refreshHours.coerceIn(1, 720)
it[MoodsTable.ytEnabled] = ytEnabled it[MoodsTable.changedAt] = Instant.now().toString()
it[changedAt] = Instant.now().toString()
} }
moodRecord(MoodsTable.selectAll().where { MoodsTable.id eq id }.single()) moodRecord(MoodsTable.selectAll().where { MoodsTable.id eq id }.single())
@@ -165,70 +174,117 @@ object MoodRepository {
} > 0 } > 0
} }
fun seeds(id: String): List<MoodSeedRecord> = transaction { // ---------------------------------------------------------------- songs
MoodSeedsTable.selectAll()
.where { MoodSeedsTable.moodId eq id } fun songs(id: String): List<MoodSongRecord> = transaction {
MoodSongsTable.selectAll()
.where { MoodSongsTable.moodId eq id }
.orderBy( .orderBy(
MoodSeedsTable.addedAt to SortOrder.ASC, MoodSongsTable.addedAt to SortOrder.ASC,
MoodSeedsTable.songId to SortOrder.ASC, MoodSongsTable.songId to SortOrder.ASC,
) )
.map { .map {
MoodSeedRecord( MoodSongRecord(
songId = it[MoodSeedsTable.songId], songId = it[MoodSongsTable.songId],
origin = it[MoodSeedsTable.origin], origin = it[MoodSongsTable.origin],
addedAt = it[MoodSeedsTable.addedAt], addedAt = it[MoodSongsTable.addedAt],
) )
} }
} }
fun seedSongIds(id: String): List<String> = seeds(id).map { it.songId } fun songIds(id: String): List<String> = songs(id).map { it.songId }
/** Returns false when the song is already a seed or the mood is not owned. */ fun songRecords(id: String): List<SongRecord> {
fun addSeed(id: String, owner: String, songId: String, origin: String): Boolean = transaction { val ids = songIds(id)
val owned = MoodsTable.selectAll() return SongRepository.byIds(ids)
.where { (MoodsTable.id eq id) and (MoodsTable.owner eq owner) } }
.any()
if (!owned) return@transaction false
val exists = MoodSeedsTable.selectAll() /** Returns false when the song is already in the mood or the mood does not exist. */
.where { (MoodSeedsTable.moodId eq id) and (MoodSeedsTable.songId eq songId) } fun addSong(id: String, songId: String, origin: String): Boolean = transaction {
val exists = MoodSongsTable.selectAll()
.where { (MoodSongsTable.moodId eq id) and (MoodSongsTable.songId eq songId) }
.any() .any()
if (exists) return@transaction false if (exists) return@transaction false
val moodExists = MoodsTable.selectAll().where { MoodsTable.id eq id }.any()
if (!moodExists) return@transaction false
val now = Instant.now().toString() val now = Instant.now().toString()
MoodSeedsTable.insertIgnore { MoodSongsTable.insertIgnore {
it[MoodSeedsTable.moodId] = id it[moodId] = id
it[MoodSeedsTable.songId] = songId it[this.songId] = songId
it[MoodSeedsTable.origin] = origin.takeIf { value -> value in ORIGINS } ?: "manual" it[this.origin] = origin.takeIf { value -> value in ORIGINS } ?: "search"
it[addedAt] = now it[addedAt] = now
} }
MoodsTable.update({ MoodsTable.id eq id }) { MoodsTable.update({ MoodsTable.id eq id }) {
it[changedAt] = now it[changedAt] = now
} }
// A track that is now permanently in the mood must not also linger as a
// hidden candidate or an active recommendation slot.
val videoId = SongsTable.select(SongsTable.videoId)
.where { SongsTable.id eq songId }
.singleOrNull()
?.get(SongsTable.videoId)
if (videoId != null) {
MoodRecommendationsTable.deleteWhere {
(MoodRecommendationsTable.moodId eq id) and
(MoodRecommendationsTable.videoId eq videoId) and
(MoodRecommendationsTable.state eq "new")
}
MoodRecommendationsTable.update({
(MoodRecommendationsTable.moodId eq id) and
(MoodRecommendationsTable.videoId eq videoId) and
(MoodRecommendationsTable.state inList listOf("downloading", "ready", "failed"))
}) {
it[state] = "added"
it[this.songId] = songId
}
}
true true
} }
fun removeSeed(id: String, owner: String, songId: String): Boolean = transaction { fun removeSong(id: String, songId: String): Boolean = transaction {
val owned = MoodsTable.selectAll() val removed = MoodSongsTable.deleteWhere {
.where { (MoodsTable.id eq id) and (MoodsTable.owner eq owner) } (MoodSongsTable.moodId eq id) and (MoodSongsTable.songId eq songId)
.any()
if (!owned) return@transaction false
val removed = MoodSeedsTable.deleteWhere {
(MoodSeedsTable.moodId eq id) and (MoodSeedsTable.songId eq songId)
} > 0 } > 0
if (removed) { if (removed) {
MoodsTable.update({ MoodsTable.id eq id }) { MoodsTable.update({ MoodsTable.id eq id }) {
it[changedAt] = Instant.now().toString() it[changedAt] = Instant.now().toString()
} }
// Detaching must not keep a permanent `added` decision pinned to the
// file, otherwise the GC could never delete it.
MoodRecommendationsTable.update({
(MoodRecommendationsTable.moodId eq id) and
(MoodRecommendationsTable.songId eq songId) and
(MoodRecommendationsTable.state eq "added")
}) {
it[MoodRecommendationsTable.songId] = null
}
} }
removed removed
} }
// ------------------------------------------------------ recommendations
/** Active recommendations, i.e. the rows the client is allowed to see. */
fun recommendations(id: String): List<RecommendationRecord> = transaction { fun recommendations(id: String): List<RecommendationRecord> = transaction {
MoodRecommendationsTable.selectAll()
.where {
(MoodRecommendationsTable.moodId eq id) and
(MoodRecommendationsTable.state inList VISIBLE_STATES)
}
.orderBy(
MoodRecommendationsTable.rank to SortOrder.ASC,
MoodRecommendationsTable.firstSeen to SortOrder.ASC,
)
.map { it.toRecommendationRecord() }
}
fun allRecommendations(id: String): List<RecommendationRecord> = transaction {
MoodRecommendationsTable.selectAll() MoodRecommendationsTable.selectAll()
.where { MoodRecommendationsTable.moodId eq id } .where { MoodRecommendationsTable.moodId eq id }
.orderBy(MoodRecommendationsTable.lastSeen to SortOrder.DESC) .orderBy(MoodRecommendationsTable.rank to SortOrder.ASC)
.map { it.toRecommendationRecord() } .map { it.toRecommendationRecord() }
} }
@@ -242,22 +298,51 @@ object MoodRepository {
?.toRecommendationRecord() ?.toRecommendationRecord()
} }
fun activeCount(moodId: String): Int = transaction {
MoodRecommendationsTable.selectAll()
.where {
(MoodRecommendationsTable.moodId eq moodId) and
(MoodRecommendationsTable.state inList ACTIVE_STATES)
}
.count()
.toInt()
}
fun videoIdsInMood(moodId: String): Set<String> = transaction {
val songs = MoodSongsTable.select(MoodSongsTable.songId)
.where { MoodSongsTable.moodId eq moodId }
SongsTable.select(SongsTable.videoId)
.where { SongsTable.id inSubQuery songs }
.map { it[SongsTable.videoId] }
.toSet()
}
fun decidedVideoIds(moodId: String): Set<String> = transaction {
MoodRecommendationsTable.selectAll()
.where {
(MoodRecommendationsTable.moodId eq moodId) and
(MoodRecommendationsTable.state inList HIDDEN_DECISIONS)
}
.map { it[MoodRecommendationsTable.videoId] }
.toSet()
}
/** /**
* Inserts new candidates and refreshes the metadata of known ones. Decisions * Stores a sorted discovery batch as hidden `new` candidates. Existing rows
* (accepted/rejected) and the add counter survive a refresh; stale undecided * keep their state: active slots and decisions must survive a refresh.
* candidates are removed so the review list does not grow without bound.
*/ */
fun storeRecommendations( fun storeCandidates(
moodId: String, moodId: String,
items: List<RecommendationUpsert>, items: List<RecommendationUpsert>,
seenAt: String, seenAt: String,
poolLimit: Int = CANDIDATE_POOL,
): Int = transaction { ): Int = transaction {
val existing = MoodRecommendationsTable.selectAll() val existing = MoodRecommendationsTable.selectAll()
.where { MoodRecommendationsTable.moodId eq moodId } .where { MoodRecommendationsTable.moodId eq moodId }
.associateBy { it[MoodRecommendationsTable.videoId] } .associateBy { it[MoodRecommendationsTable.videoId] }
var stored = 0 var stored = 0
items.forEach { item -> items.take(poolLimit.coerceAtLeast(1)).forEachIndexed { index, item ->
val row = existing[item.videoId] val row = existing[item.videoId]
if (row == null) { if (row == null) {
MoodRecommendationsTable.insert { MoodRecommendationsTable.insert {
@@ -269,12 +354,11 @@ object MoodRepository {
it[duration] = item.duration it[duration] = item.duration
it[thumbnail] = item.thumbnail?.take(500) it[thumbnail] = item.thumbnail?.take(500)
it[state] = "new" it[state] = "new"
it[matchedSongId] = item.matchedSongId it[rank] = index
it[timesAdded] = 0
it[firstSeen] = seenAt it[firstSeen] = seenAt
it[lastSeen] = seenAt it[lastSeen] = seenAt
} }
} else { } else if (row[MoodRecommendationsTable.state] == "new") {
MoodRecommendationsTable.update({ MoodRecommendationsTable.update({
(MoodRecommendationsTable.moodId eq moodId) and (MoodRecommendationsTable.moodId eq moodId) and
(MoodRecommendationsTable.videoId eq item.videoId) (MoodRecommendationsTable.videoId eq item.videoId)
@@ -284,61 +368,201 @@ object MoodRepository {
it[album] = item.album?.take(300) it[album] = item.album?.take(300)
it[duration] = item.duration it[duration] = item.duration
it[thumbnail] = item.thumbnail?.take(500) it[thumbnail] = item.thumbnail?.take(500)
it[matchedSongId] = item.matchedSongId it[rank] = index
?: row[MoodRecommendationsTable.matchedSongId]
it[lastSeen] = seenAt it[lastSeen] = seenAt
} }
} }
stored += 1 stored += 1
} }
val keep = items.map { it.videoId }.toSet() // Candidates that were not returned again are no longer useful.
val stale = existing.keys - keep val keep = items.take(poolLimit.coerceAtLeast(1)).map { it.videoId }.toSet()
if (stale.isNotEmpty()) { if (keep.isNotEmpty()) {
MoodRecommendationsTable.deleteWhere { MoodRecommendationsTable.deleteWhere {
(MoodRecommendationsTable.moodId eq moodId) and (MoodRecommendationsTable.moodId eq moodId) and
(MoodRecommendationsTable.videoId inList stale.toList()) and (MoodRecommendationsTable.state eq "new") and
(MoodRecommendationsTable.state eq "new") (MoodRecommendationsTable.videoId notInList keep.toList())
} }
} }
stored stored
} }
fun setRecommendationState(moodId: String, videoId: String, state: String): Boolean = transaction { /** Moves up to [limit] hidden candidates into the downloading state. */
fun promoteCandidates(moodId: String, limit: Int): List<RecommendationRecord> = transaction {
if (limit <= 0) return@transaction emptyList()
val rows = MoodRecommendationsTable.selectAll()
.where {
(MoodRecommendationsTable.moodId eq moodId) and
(MoodRecommendationsTable.state eq "new")
}
.orderBy(
MoodRecommendationsTable.rank to SortOrder.ASC,
MoodRecommendationsTable.firstSeen to SortOrder.ASC,
)
.limit(limit)
.toList()
rows.forEach { row ->
MoodRecommendationsTable.update({
(MoodRecommendationsTable.moodId eq moodId) and
(MoodRecommendationsTable.videoId eq row[MoodRecommendationsTable.videoId])
}) {
it[state] = "downloading"
it[error] = null
}
}
rows.map { it.toRecommendationRecord().copy(state = "downloading", error = null) }
}
fun setRecommendationState(moodId: String, videoId: String, state: String, error: String? = null) {
transaction {
MoodRecommendationsTable.update({ MoodRecommendationsTable.update({
(MoodRecommendationsTable.moodId eq moodId) and (MoodRecommendationsTable.moodId eq moodId) and
(MoodRecommendationsTable.videoId eq videoId) (MoodRecommendationsTable.videoId eq videoId)
}) { }) {
it[MoodRecommendationsTable.state] = state it[this.state] = state
} > 0 it[this.error] = error?.take(1000)
}
}
} }
fun incrementRecommendationAdds(moodId: String, videoId: String) { fun markRecommendationDownloading(moodId: String, videoId: String) {
transaction { transaction {
val current = MoodRecommendationsTable.selectAll() MoodRecommendationsTable.update({
(MoodRecommendationsTable.moodId eq moodId) and
(MoodRecommendationsTable.videoId eq videoId)
}) {
it[state] = "downloading"
it[attempts] = 0
it[error] = null
}
}
}
fun failRecommendation(moodId: String, videoId: String, error: String, attempts: Int) {
transaction {
MoodRecommendationsTable.update({
(MoodRecommendationsTable.moodId eq moodId) and
(MoodRecommendationsTable.videoId eq videoId) and
(MoodRecommendationsTable.state eq "downloading")
}) {
it[state] = "failed"
it[this.error] = error.take(1000)
it[this.attempts] = attempts
}
}
}
/**
* Applies a finished download. A rejection that happened while the download
* was running wins: the file is left for the GC to delete.
*/
fun completeDownload(moodId: String, videoId: String, songId: String): RecommendationCompletion =
transaction {
val row = MoodRecommendationsTable.selectAll()
.where { .where {
(MoodRecommendationsTable.moodId eq moodId) and (MoodRecommendationsTable.moodId eq moodId) and
(MoodRecommendationsTable.videoId eq videoId) (MoodRecommendationsTable.videoId eq videoId)
} }
.singleOrNull() .singleOrNull()
?.get(MoodRecommendationsTable.timesAdded) ?: return@transaction RecommendationCompletion.MISSING
?: return@transaction
when (row[MoodRecommendationsTable.state]) {
"rejected" -> RecommendationCompletion.REJECTED
"added" -> {
insertRecommendationSong(moodId, songId)
MoodRecommendationsTable.update({ MoodRecommendationsTable.update({
(MoodRecommendationsTable.moodId eq moodId) and (MoodRecommendationsTable.moodId eq moodId) and
(MoodRecommendationsTable.videoId eq videoId) (MoodRecommendationsTable.videoId eq videoId)
}) { }) {
it[timesAdded] = current + 1 it[this.songId] = songId
it[state] = "added"
}
RecommendationCompletion.ALREADY_ADDED
}
else -> {
MoodRecommendationsTable.update({
(MoodRecommendationsTable.moodId eq moodId) and
(MoodRecommendationsTable.videoId eq videoId)
}) {
it[this.songId] = songId
it[state] = "ready"
it[error] = null
it[lastSeen] = Instant.now().toString()
}
RecommendationCompletion.READY
} }
} }
} }
/** Moods whose YT discovery is due, oldest first. */ /** Called when the user presses Add. Marks the row and attaches a ready song. */
fun markAdded(moodId: String, videoId: String): RecommendationCompletion = transaction {
val row = MoodRecommendationsTable.selectAll()
.where {
(MoodRecommendationsTable.moodId eq moodId) and
(MoodRecommendationsTable.videoId eq videoId)
}
.singleOrNull()
?: return@transaction RecommendationCompletion.MISSING
val songId = row[MoodRecommendationsTable.songId]
MoodRecommendationsTable.update({
(MoodRecommendationsTable.moodId eq moodId) and
(MoodRecommendationsTable.videoId eq videoId)
}) {
it[state] = "added"
}
if (songId != null && row[MoodRecommendationsTable.state] != "added") {
insertRecommendationSong(moodId, songId)
return@transaction RecommendationCompletion.ALREADY_ADDED
}
RecommendationCompletion.READY
}
fun markRejected(moodId: String, videoId: String) {
setRecommendationState(moodId, videoId, "rejected")
}
fun readySongs(moodId: String): List<SongRecord> = transaction {
val songIds = MoodRecommendationsTable
.select(MoodRecommendationsTable.songId)
.where {
(MoodRecommendationsTable.moodId eq moodId) and
(MoodRecommendationsTable.state eq "ready")
}
.mapNotNull { it[MoodRecommendationsTable.songId] }
if (songIds.isEmpty()) emptyList()
else SongsTable.selectAll()
.where { SongsTable.id inList songIds }
.map {
SongRecord(
id = it[SongsTable.id],
videoId = it[SongsTable.videoId],
title = it[SongsTable.title],
artist = it[SongsTable.artist],
album = it[SongsTable.album],
duration = it[SongsTable.duration],
bitRate = it[SongsTable.bitRate],
size = it[SongsTable.size],
suffix = it[SongsTable.suffix],
contentType = it[SongsTable.contentType],
relativePath = it[SongsTable.relativePath],
fileMtime = it[SongsTable.fileMtime],
thumbnail = it[SongsTable.thumbnail],
createdAt = it[SongsTable.createdAt],
updatedAt = it[SongsTable.updatedAt],
)
}
}
// ---------------------------------------------------------- discovery
/** Moods whose automatic discovery is due, oldest first. */
fun staleDiscovery(now: Instant, limit: Int): List<MoodRecord> = transaction { fun staleDiscovery(now: Instant, limit: Int): List<MoodRecord> = transaction {
MoodsTable.selectAll() MoodsTable.selectAll()
.where { MoodsTable.ytEnabled eq true }
.orderBy(MoodsTable.lastDiscoveryAt to SortOrder.ASC) .orderBy(MoodsTable.lastDiscoveryAt to SortOrder.ASC)
.map { moodRecord(it) } .map { moodRecord(it) }
.filter { mood -> isStale(mood.lastDiscoveryAt, mood.refreshHours, now) } .filter { mood -> mood.recommendationCount > 0 && isStale(mood.lastDiscoveryAt, mood.refreshHours, now) }
.take(limit.coerceAtLeast(1)) .take(limit.coerceAtLeast(1))
} }
@@ -350,31 +574,6 @@ object MoodRepository {
} }
} }
/** Links a downloaded recommendation to the scanned song and accepts it. */
fun attachDownloadedSong(moodId: String, videoId: String, songId: String): Boolean = transaction {
val row = MoodRecommendationsTable.selectAll()
.where {
(MoodRecommendationsTable.moodId eq moodId) and
(MoodRecommendationsTable.videoId eq videoId)
}
.singleOrNull()
?: return@transaction false
// The user may have rejected the track while it was downloading.
if (row[MoodRecommendationsTable.state] == "rejected") {
return@transaction false
}
MoodRecommendationsTable.update({
(MoodRecommendationsTable.moodId eq moodId) and
(MoodRecommendationsTable.videoId eq videoId)
}) {
it[matchedSongId] = songId
it[state] = "accepted"
it[timesAdded] = row[MoodRecommendationsTable.timesAdded] + 1
it[lastSeen] = Instant.now().toString()
}
true
}
fun markDiscovered(id: String) { fun markDiscovered(id: String) {
transaction { transaction {
MoodsTable.update({ MoodsTable.id eq id }) { MoodsTable.update({ MoodsTable.id eq id }) {
@@ -384,15 +583,20 @@ object MoodRepository {
} }
} }
private fun insertSeeds(id: String, songIds: List<String>, origin: String) { fun isDiscoveryDue(mood: MoodRecord, now: Instant = Instant.now()): Boolean =
val now = Instant.now().toString() isStale(mood.lastDiscoveryAt, mood.refreshHours, now)
songIds.distinct().forEach { songId ->
MoodSeedsTable.insertIgnore { // ------------------------------------------------------------ internal
it[moodId] = id
private fun insertRecommendationSong(moodId: String, songId: String) {
MoodSongsTable.insertIgnore {
it[this.moodId] = moodId
it[this.songId] = songId it[this.songId] = songId
it[this.origin] = origin it[origin] = "recommendation"
it[addedAt] = now it[addedAt] = Instant.now().toString()
} }
MoodsTable.update({ MoodsTable.id eq moodId }) {
it[changedAt] = Instant.now().toString()
} }
} }
@@ -410,25 +614,23 @@ object MoodRepository {
duration = this[MoodRecommendationsTable.duration], duration = this[MoodRecommendationsTable.duration],
thumbnail = this[MoodRecommendationsTable.thumbnail], thumbnail = this[MoodRecommendationsTable.thumbnail],
state = this[MoodRecommendationsTable.state], state = this[MoodRecommendationsTable.state],
matchedSongId = this[MoodRecommendationsTable.matchedSongId], songId = this[MoodRecommendationsTable.songId],
timesAdded = this[MoodRecommendationsTable.timesAdded], attempts = this[MoodRecommendationsTable.attempts],
error = this[MoodRecommendationsTable.error],
rank = this[MoodRecommendationsTable.rank],
firstSeen = this[MoodRecommendationsTable.firstSeen], firstSeen = this[MoodRecommendationsTable.firstSeen],
lastSeen = this[MoodRecommendationsTable.lastSeen], lastSeen = this[MoodRecommendationsTable.lastSeen],
) )
private fun moodRecord(row: ResultRow): MoodRecord { private fun moodRecord(row: ResultRow): MoodRecord {
val id = row[MoodsTable.id] val id = row[MoodsTable.id]
val seedCount = MoodSeedsTable.selectAll() val songCount = MoodSongsTable.selectAll()
.where { MoodSeedsTable.moodId eq id } .where { MoodSongsTable.moodId eq id }
.count()
.toInt()
val newRecommendations = MoodRecommendationsTable.selectAll()
.where {
(MoodRecommendationsTable.moodId eq id) and
(MoodRecommendationsTable.state eq "new")
}
.count() .count()
.toInt() .toInt()
val ready = countState(id, "ready")
val downloading = countState(id, "downloading")
val failed = countState(id, "failed")
return MoodRecord( return MoodRecord(
id = id, id = id,
@@ -437,19 +639,29 @@ object MoodRepository {
description = row[MoodsTable.description], description = row[MoodsTable.description],
query = row[MoodsTable.query], query = row[MoodsTable.query],
genres = decodeGenres(row[MoodsTable.genres]), genres = decodeGenres(row[MoodsTable.genres]),
targetSize = row[MoodsTable.targetSize], collectionPercent = row[MoodsTable.collectionPercent],
seedPercent = row[MoodsTable.seedPercent], recommendationCount = row[MoodsTable.recommendationCount],
refreshHours = row[MoodsTable.refreshHours], refreshHours = row[MoodsTable.refreshHours],
ytEnabled = row[MoodsTable.ytEnabled],
lastDiscoveryAt = row[MoodsTable.lastDiscoveryAt], lastDiscoveryAt = row[MoodsTable.lastDiscoveryAt],
createdAt = row[MoodsTable.createdAt], createdAt = row[MoodsTable.createdAt],
changedAt = row[MoodsTable.changedAt], changedAt = row[MoodsTable.changedAt],
lastError = row[MoodsTable.lastError], lastError = row[MoodsTable.lastError],
seedCount = seedCount, songCount = songCount,
newRecommendations = newRecommendations, recommendationReady = ready,
recommendationDownloading = downloading,
recommendationFailed = failed,
) )
} }
private fun countState(moodId: String, state: String): Int =
MoodRecommendationsTable.selectAll()
.where {
(MoodRecommendationsTable.moodId eq moodId) and
(MoodRecommendationsTable.state eq state)
}
.count()
.toInt()
private fun encodeGenres(genres: List<String>): String = private fun encodeGenres(genres: List<String>): String =
Json.encodeToString( Json.encodeToString(
genreSerializer, genreSerializer,
@@ -459,5 +671,6 @@ object MoodRepository {
private fun decodeGenres(value: String): List<String> = private fun decodeGenres(value: String): List<String> =
runCatching { Json.decodeFromString(genreSerializer, value) }.getOrDefault(emptyList()) runCatching { Json.decodeFromString(genreSerializer, value) }.getOrDefault(emptyList())
private val ORIGINS = setOf("manual", "recommendation", "yt") private val ORIGINS = setOf("search", "recommendation", "url")
private const val CANDIDATE_POOL = 60
} }
@@ -0,0 +1,189 @@
package win.taptap.musicmanager.db
import java.time.Instant
import org.jetbrains.exposed.sql.ResultRow
import org.jetbrains.exposed.sql.SortOrder
import org.jetbrains.exposed.sql.SqlExpressionBuilder.eq
import org.jetbrains.exposed.sql.SqlExpressionBuilder.inList
import org.jetbrains.exposed.sql.and
import org.jetbrains.exposed.sql.deleteWhere
import org.jetbrains.exposed.sql.insert
import org.jetbrains.exposed.sql.selectAll
import org.jetbrains.exposed.sql.transactions.transaction
import org.jetbrains.exposed.sql.update
import win.taptap.musicmanager.util.IdGenerator
data class SongRecord(
val id: String,
val videoId: String,
val title: String,
val artist: String,
val album: String?,
val duration: Int,
val bitRate: Int,
val size: Long,
val suffix: String,
val contentType: String,
val relativePath: String,
val fileMtime: Long,
val thumbnail: String?,
val createdAt: String,
val updatedAt: String,
)
data class SongUpsert(
val videoId: String,
val title: String,
val artist: String,
val album: String?,
val duration: Int,
val bitRate: Int,
val size: Long,
val suffix: String = "mp3",
val contentType: String = "audio/mpeg",
val relativePath: String,
val fileMtime: Long,
val thumbnail: String?,
)
/**
* The shared, server-wide song store. Rows are keyed by YouTube video id, so
* downloading the same video for two moods or two users produces one file and
* one row.
*/
object SongRepository {
fun upsert(input: SongUpsert): SongRecord = transaction {
val now = Instant.now().toString()
val existing = SongsTable.selectAll()
.where { SongsTable.videoId eq input.videoId }
.singleOrNull()
if (existing == null) {
val id = IdGenerator.songId(input.videoId)
SongsTable.insert {
it[this.id] = id
it[videoId] = input.videoId
it[title] = input.title
it[artist] = input.artist
it[album] = input.album
it[duration] = input.duration
it[bitRate] = input.bitRate
it[size] = input.size
it[suffix] = input.suffix
it[contentType] = input.contentType
it[relativePath] = input.relativePath
it[fileMtime] = input.fileMtime
it[thumbnail] = input.thumbnail
it[createdAt] = now
it[updatedAt] = now
}
SongRecord(
id = id,
videoId = input.videoId,
title = input.title,
artist = input.artist,
album = input.album,
duration = input.duration,
bitRate = input.bitRate,
size = input.size,
suffix = input.suffix,
contentType = input.contentType,
relativePath = input.relativePath,
fileMtime = input.fileMtime,
thumbnail = input.thumbnail,
createdAt = now,
updatedAt = now,
)
} else {
val current = existing.toRecord()
val id = current.id
SongsTable.update({ SongsTable.id eq id }) {
it[title] = input.title
it[artist] = input.artist
it[album] = input.album
it[duration] = input.duration
it[bitRate] = input.bitRate
it[size] = input.size
it[suffix] = input.suffix
it[contentType] = input.contentType
it[relativePath] = input.relativePath
it[fileMtime] = input.fileMtime
it[thumbnail] = input.thumbnail ?: current.thumbnail
it[updatedAt] = now
}
current.copy(
title = input.title,
artist = input.artist,
album = input.album,
duration = input.duration,
bitRate = input.bitRate,
size = input.size,
suffix = input.suffix,
contentType = input.contentType,
relativePath = input.relativePath,
fileMtime = input.fileMtime,
thumbnail = input.thumbnail ?: current.thumbnail,
updatedAt = now,
)
}
}
fun byId(id: String): SongRecord? = transaction {
SongsTable.selectAll()
.where { SongsTable.id eq id }
.singleOrNull()
?.toRecord()
}
fun byVideoId(videoId: String): SongRecord? = transaction {
SongsTable.selectAll()
.where { SongsTable.videoId eq videoId }
.singleOrNull()
?.toRecord()
}
fun byIds(ids: Collection<String>): List<SongRecord> {
if (ids.isEmpty()) return emptyList()
return transaction {
// Stay below SQLite's variable limit for very large moods.
ids.toList().chunked(SQL_CHUNK).flatMap { chunk ->
SongsTable.selectAll()
.where { SongsTable.id inList chunk }
.map { it.toRecord() }
}
}
}
fun all(): List<SongRecord> = transaction {
SongsTable.selectAll()
.orderBy(SongsTable.createdAt to SortOrder.ASC)
.map { it.toRecord() }
}
fun count(): Long = transaction { SongsTable.selectAll().count() }
fun delete(id: String): Boolean = transaction {
SongsTable.deleteWhere { SongsTable.id eq id } > 0
}
private fun org.jetbrains.exposed.sql.ResultRow.toRecord(): SongRecord = SongRecord(
id = this[SongsTable.id],
videoId = this[SongsTable.videoId],
title = this[SongsTable.title],
artist = this[SongsTable.artist],
album = this[SongsTable.album],
duration = this[SongsTable.duration],
bitRate = this[SongsTable.bitRate],
size = this[SongsTable.size],
suffix = this[SongsTable.suffix],
contentType = this[SongsTable.contentType],
relativePath = this[SongsTable.relativePath],
fileMtime = this[SongsTable.fileMtime],
thumbnail = this[SongsTable.thumbnail],
createdAt = this[SongsTable.createdAt],
updatedAt = this[SongsTable.updatedAt],
)
private const val SQL_CHUNK = 500
}
@@ -12,132 +12,34 @@ object UsersTable : Table("users") {
override val primaryKey = PrimaryKey(name) override val primaryKey = PrimaryKey(name)
} }
object MusicFoldersTable : Table("music_folders") { /**
val id = integer("id") * The shared song store. One row per YouTube video id; the same file can be
val name = varchar("name", 255) * referenced by any number of users' moods. There is no per-song owner.
val type = varchar("type", 16) */
val rootPath = text("root_path")
override val primaryKey = PrimaryKey(id)
}
object ArtistsTable : Table("artists") {
val id = varchar("id", 32)
val name = text("name")
val sortName = text("sort_name")
val coverArt = varchar("cover_art", 32).nullable()
val searchText = text("search_text")
override val primaryKey = PrimaryKey(id)
}
object AlbumsTable : Table("albums") {
val id = varchar("id", 32)
val name = text("name")
val year = integer("year").nullable()
val coverArt = varchar("cover_art", 32).nullable()
val createdAt = varchar("created_at", 40)
val searchText = text("search_text")
override val primaryKey = PrimaryKey(id)
}
object AlbumArtistsTable : Table("album_artists") {
val albumId = reference("album_id", AlbumsTable.id, onDelete = ReferenceOption.CASCADE)
val artistId = reference("artist_id", ArtistsTable.id, onDelete = ReferenceOption.CASCADE)
val position = integer("position")
override val primaryKey = PrimaryKey(albumId, position)
}
object SongsTable : Table("songs") { object SongsTable : Table("songs") {
val id = varchar("id", 32) val id = varchar("id", 32)
val albumId = reference("album_id", AlbumsTable.id, onDelete = ReferenceOption.CASCADE) val videoId = varchar("video_id", 64).uniqueIndex()
val folderId = reference("folder_id", MusicFoldersTable.id, onDelete = ReferenceOption.CASCADE)
val title = text("title") val title = text("title")
val displayArtist = text("display_artist") val artist = text("artist")
val albumName = text("album_name") val album = text("album").nullable()
val trackNumber = integer("track_number").nullable() val duration = integer("duration").default(0)
val discNumber = integer("disc_number").default(1) val bitRate = integer("bit_rate").default(0)
val year = integer("year").nullable() val size = long("size").default(0)
val duration = integer("duration") val suffix = varchar("suffix", 16).default("mp3")
val bitRate = integer("bit_rate") val contentType = varchar("content_type", 64).default("audio/mpeg")
val size = long("size")
val suffix = varchar("suffix", 16)
val contentType = varchar("content_type", 64)
val relativePath = text("relative_path") val relativePath = text("relative_path")
val fileMtime = long("file_mtime") val fileMtime = long("file_mtime").default(0)
val thumbnail = text("thumbnail").nullable()
val createdAt = varchar("created_at", 40) val createdAt = varchar("created_at", 40)
val updatedAt = varchar("updated_at", 40).default("")
val searchText = text("search_text")
val lyrics = text("lyrics").nullable()
override val primaryKey = PrimaryKey(id)
}
object SongArtistsTable : Table("song_artists") {
val songId = reference("song_id", SongsTable.id, onDelete = ReferenceOption.CASCADE)
val artistId = reference("artist_id", ArtistsTable.id, onDelete = ReferenceOption.CASCADE)
val position = integer("position")
override val primaryKey = PrimaryKey(songId, position)
}
object SongGenresTable : Table("song_genres") {
val songId = reference("song_id", SongsTable.id, onDelete = ReferenceOption.CASCADE)
val genre = varchar("genre", 255)
override val primaryKey = PrimaryKey(songId, genre)
}
object AlbumGenresTable : Table("album_genres") {
val albumId = reference("album_id", AlbumsTable.id, onDelete = ReferenceOption.CASCADE)
val genre = varchar("genre", 255)
override val primaryKey = PrimaryKey(albumId, genre)
}
object ArtistGenresTable : Table("artist_genres") {
val artistId = reference("artist_id", ArtistsTable.id, onDelete = ReferenceOption.CASCADE)
val genre = varchar("genre", 255)
override val primaryKey = PrimaryKey(artistId, genre)
}
object UserSongStateTable : Table("user_song_state") {
val userName = reference("user_name", UsersTable.name, onDelete = ReferenceOption.CASCADE)
val songId = reference("song_id", SongsTable.id, onDelete = ReferenceOption.CASCADE)
val playCount = integer("play_count").default(0)
val lastPlayedAt = varchar("last_played_at", 40).nullable()
val starredAt = varchar("starred_at", 40).nullable()
val rating = integer("rating").nullable()
override val primaryKey = PrimaryKey(userName, songId)
}
object UserStarsTable : Table("user_stars") {
val userName = reference("user_name", UsersTable.name, onDelete = ReferenceOption.CASCADE)
val itemType = varchar("item_type", 16)
val itemId = varchar("item_id", 32)
val starredAt = varchar("starred_at", 40)
val rating = integer("rating").nullable()
override val primaryKey = PrimaryKey(userName, itemType, itemId)
}
object BookmarksTable : Table("bookmarks") {
val userName = reference("user_name", UsersTable.name, onDelete = ReferenceOption.CASCADE)
val songId = reference("song_id", SongsTable.id, onDelete = ReferenceOption.CASCADE)
val positionMs = long("position_ms")
val updatedAt = varchar("updated_at", 40) val updatedAt = varchar("updated_at", 40)
override val primaryKey = PrimaryKey(userName, songId) override val primaryKey = PrimaryKey(id)
} }
/** /**
* Moods are the only collection type. A mood owns seeds (permanent anchors) and * Moods are the only collection type. A mood owns permanent songs and an
* recommendations (YT Music candidates); playback queues are generated live and * active recommendation pool; playback queues are generated live and never
* never materialized here. * materialized as playlists.
*/ */
object MoodsTable : Table("moods") { object MoodsTable : Table("moods") {
val id = varchar("id", 32) val id = varchar("id", 32)
@@ -146,10 +48,9 @@ object MoodsTable : Table("moods") {
val description = text("description").nullable() val description = text("description").nullable()
val query = text("query").nullable() val query = text("query").nullable()
val genres = text("genres").default("[]") val genres = text("genres").default("[]")
val targetSize = integer("target_size").default(30) val collectionPercent = integer("collection_percent").default(50)
val seedPercent = integer("seed_percent").default(50) val recommendationCount = integer("recommendation_count").default(10)
val refreshHours = integer("refresh_hours").default(24) val refreshHours = integer("refresh_hours").default(24)
val ytEnabled = bool("yt_enabled").default(true)
val lastDiscoveryAt = varchar("last_discovery_at", 40).nullable() val lastDiscoveryAt = varchar("last_discovery_at", 40).nullable()
val createdAt = varchar("created_at", 40) val createdAt = varchar("created_at", 40)
val changedAt = varchar("changed_at", 40) val changedAt = varchar("changed_at", 40)
@@ -158,10 +59,10 @@ object MoodsTable : Table("moods") {
override val primaryKey = PrimaryKey(id) override val primaryKey = PrimaryKey(id)
} }
object MoodSeedsTable : Table("mood_seeds") { object MoodSongsTable : Table("mood_songs") {
val moodId = reference("mood_id", MoodsTable.id, onDelete = ReferenceOption.CASCADE) val moodId = reference("mood_id", MoodsTable.id, onDelete = ReferenceOption.CASCADE)
val songId = reference("song_id", SongsTable.id, onDelete = ReferenceOption.CASCADE) val songId = reference("song_id", SongsTable.id, onDelete = ReferenceOption.CASCADE)
val origin = varchar("origin", 16).default("manual") val origin = varchar("origin", 16).default("search")
val addedAt = varchar("added_at", 40) val addedAt = varchar("added_at", 40)
override val primaryKey = PrimaryKey(moodId, songId) override val primaryKey = PrimaryKey(moodId, songId)
@@ -176,12 +77,14 @@ object MoodRecommendationsTable : Table("mood_recommendations") {
val duration = integer("duration").nullable() val duration = integer("duration").nullable()
val thumbnail = text("thumbnail").nullable() val thumbnail = text("thumbnail").nullable()
val state = varchar("state", 16).default("new") val state = varchar("state", 16).default("new")
val matchedSongId = reference( val songId = reference(
"matched_song_id", "song_id",
SongsTable.id, SongsTable.id,
onDelete = ReferenceOption.SET_NULL, onDelete = ReferenceOption.SET_NULL,
).nullable() ).nullable()
val timesAdded = integer("times_added").default(0) val attempts = integer("attempts").default(0)
val error = text("error").nullable()
val rank = integer("rank").default(0)
val firstSeen = varchar("first_seen", 40) val firstSeen = varchar("first_seen", 40)
val lastSeen = varchar("last_seen", 40) val lastSeen = varchar("last_seen", 40)
@@ -202,6 +105,22 @@ object ApiTokensTable : Table("api_tokens") {
override val primaryKey = PrimaryKey(id) override val primaryKey = PrimaryKey(id)
} }
object DownloadJobsTable : Table("download_jobs") {
val id = varchar("id", 32)
val videoId = varchar("video_id", 64)
val moodId = reference("mood_id", MoodsTable.id, onDelete = ReferenceOption.CASCADE)
val kind = varchar("kind", 16).default("recommendation")
val state = varchar("state", 16).default("queued")
val priority = integer("priority").default(0)
val attempts = integer("attempts").default(0)
val error = text("error").nullable()
val songId = reference("song_id", SongsTable.id, onDelete = ReferenceOption.SET_NULL).nullable()
val createdAt = varchar("created_at", 40)
val updatedAt = varchar("updated_at", 40)
override val primaryKey = PrimaryKey(id)
}
object PlaybackSessionsTable : Table("playback_sessions") { object PlaybackSessionsTable : Table("playback_sessions") {
val id = varchar("id", 32) val id = varchar("id", 32)
val userName = reference("user_name", UsersTable.name, onDelete = ReferenceOption.CASCADE) val userName = reference("user_name", UsersTable.name, onDelete = ReferenceOption.CASCADE)
@@ -234,39 +153,3 @@ object SongAffinityTable : Table("song_affinity") {
override val primaryKey = PrimaryKey(userName, songId) override val primaryKey = PrimaryKey(userName, songId)
} }
object DownloadJobsTable : Table("download_jobs") {
val id = varchar("id", 32)
val userName = reference("user_name", UsersTable.name, onDelete = ReferenceOption.CASCADE)
val moodId = reference("mood_id", MoodsTable.id, onDelete = ReferenceOption.CASCADE)
val videoId = varchar("video_id", 64)
val title = text("title")
val artist = text("artist")
val state = varchar("state", 16).default("queued")
val attempts = integer("attempts").default(0)
val error = text("error").nullable()
val songId = reference("song_id", SongsTable.id, onDelete = ReferenceOption.SET_NULL).nullable()
val createdAt = varchar("created_at", 40)
val updatedAt = varchar("updated_at", 40)
override val primaryKey = PrimaryKey(id)
}
object LongformProgressTable : Table("longform_progress") {
val userName = reference("user_name", UsersTable.name, onDelete = ReferenceOption.CASCADE)
val songId = reference("song_id", SongsTable.id, onDelete = ReferenceOption.CASCADE)
val positionSeconds = integer("position_seconds").default(0)
val durationSeconds = integer("duration_seconds").default(0)
val completed = bool("completed").default(false)
val updatedAt = varchar("updated_at", 40)
override val primaryKey = PrimaryKey(userName, songId)
}
object LibraryTombstonesTable : Table("library_tombstones") {
val entity = varchar("entity", 20)
val entityId = varchar("entity_id", 32)
val deletedAt = varchar("deleted_at", 40)
override val primaryKey = PrimaryKey(entity, entityId)
}
@@ -7,10 +7,6 @@ import kotlinx.coroutines.async
import kotlinx.coroutines.awaitAll import kotlinx.coroutines.awaitAll
import kotlinx.coroutines.coroutineScope import kotlinx.coroutines.coroutineScope
import org.slf4j.LoggerFactory import org.slf4j.LoggerFactory
import win.taptap.musicmanager.config.FolderType
import win.taptap.musicmanager.config.LibraryConfig
import win.taptap.musicmanager.config.UserAccount
import win.taptap.musicmanager.db.LibraryRepository
import win.taptap.musicmanager.db.MoodRecord import win.taptap.musicmanager.db.MoodRecord
import win.taptap.musicmanager.db.MoodRepository import win.taptap.musicmanager.db.MoodRepository
import win.taptap.musicmanager.db.RecommendationUpsert import win.taptap.musicmanager.db.RecommendationUpsert
@@ -18,15 +14,14 @@ import win.taptap.musicmanager.util.IdGenerator
data class DiscoveryOutcome( data class DiscoveryOutcome(
val stored: Int, val stored: Int,
val matched: Int,
val skipped: Boolean = false, val skipped: Boolean = false,
val error: String? = null, val error: String? = null,
) )
/** /**
* Turns mood seeds, genres and free text into YouTube Music candidates. New * Turns a mood's collection, genres and free text into YouTube candidates.
* candidates are stored for review; accepted ones either become seeds (when the * Candidates are stored as hidden `new` rows; [win.taptap.musicmanager.discovery.TopUpService]
* library already has the track) or wait for the future downloader. * promotes them into active download slots.
*/ */
class DiscoveryService( class DiscoveryService(
private val search: YtSearch, private val search: YtSearch,
@@ -34,36 +29,20 @@ class DiscoveryService(
) { ) {
private val log = LoggerFactory.getLogger(DiscoveryService::class.java) private val log = LoggerFactory.getLogger(DiscoveryService::class.java)
suspend fun refresh( suspend fun refresh(mood: MoodRecord, force: Boolean = false): DiscoveryOutcome {
mood: MoodRecord, if (mood.recommendationCount <= 0) return DiscoveryOutcome(0, skipped = true)
user: UserAccount,
library: LibraryConfig,
force: Boolean = false,
): DiscoveryOutcome {
if (!mood.ytEnabled) return DiscoveryOutcome(0, 0, skipped = true)
val now = clock() val now = clock()
if (!force && isFresh(mood.lastDiscoveryAt, mood.refreshHours, now)) { if (!force && isFresh(mood.lastDiscoveryAt, mood.refreshHours, now)) {
return DiscoveryOutcome(0, 0, skipped = true) return DiscoveryOutcome(0, skipped = true)
} }
val folders = library.folders val collection = MoodRepository.songRecords(mood.id)
.asSequence() val seedArtists = collection.map { it.artist }.filter { it.isNotBlank() }
.filter { it.type == FolderType.MUSIC && it.id in user.folders }
.map { it.id }
.toSet()
if (folders.isEmpty()) {
MoodRepository.markError(mood.id, "No music folder is visible to this user")
return DiscoveryOutcome(0, 0, error = "No music folder is visible to this user")
}
val seeds = LibraryRepository.songsByIds(MoodRepository.seedSongIds(mood.id), folders, user.name)
val seedIds = seeds.map { it.id }.toSet()
val seedArtists = seeds.map { it.displayArtist }.filter { it.isNotBlank() }
val queries = buildQueries(mood, seedArtists) val queries = buildQueries(mood, seedArtists)
if (queries.isEmpty()) { if (queries.isEmpty()) {
MoodRepository.markDiscovered(mood.id) MoodRepository.markDiscovered(mood.id)
return DiscoveryOutcome(0, 0) return DiscoveryOutcome(0)
} }
// Queries are independent; running them together keeps a refresh within // Queries are independent; running them together keeps a refresh within
@@ -81,7 +60,7 @@ class DiscoveryService(
?: "Search failed" ?: "Search failed"
log.warn("YT discovery failed for mood {}: {}", mood.name, message) log.warn("YT discovery failed for mood {}: {}", mood.name, message)
MoodRepository.markError(mood.id, message) MoodRepository.markError(mood.id, message)
return DiscoveryOutcome(0, 0, error = message) return DiscoveryOutcome(0, error = message)
} }
val candidates = LinkedHashMap<String, YtCandidate>() val candidates = LinkedHashMap<String, YtCandidate>()
@@ -94,71 +73,52 @@ class DiscoveryService(
if (candidates.isEmpty()) { if (candidates.isEmpty()) {
val message = "YouTube returned no usable results" val message = "YouTube returned no usable results"
MoodRepository.markError(mood.id, message) MoodRepository.markError(mood.id, message)
return DiscoveryOutcome(0, 0, error = message) return DiscoveryOutcome(0, error = message)
} }
val known = MoodRepository.recommendations(mood.id).associateBy { it.videoId } val excluded = MoodRepository.videoIdsInMood(mood.id) + MoodRepository.decidedVideoIds(mood.id) +
val moodText = listOfNotNull(mood.query, mood.description, mood.name) MoodRepository.allRecommendations(mood.id)
.joinToString(" ") .filter { it.state != "new" }
.map { it.videoId }
.toSet()
val moodText = listOfNotNull(mood.query, mood.description, mood.name).joinToString(" ")
val upserts = candidates.values val upserts = candidates.values
.mapNotNull { candidate -> .filterNot { it.videoId in excluded }
val existing = known[candidate.videoId] .map { candidate ->
// Accepted or rejected tracks keep their decision and are never candidate to scoreCandidate(candidate, seedArtists, moodText)
// offered again; only last_seen is refreshed by the upsert.
val matched = existing?.matchedSongId ?: matchSong(candidate, folders, user.name)
if (existing == null && matched != null && matched in seedIds) {
return@mapNotNull null
} }
UpsertCandidate( .sortedWith(
candidate = candidate, compareByDescending<Pair<YtCandidate, Double>> { it.second }
matchedSongId = matched, .thenBy { it.first.title },
score = scoreCandidate(candidate, seedArtists, moodText, matched != null),
) )
} .take(CANDIDATE_POOL)
.sortedWith(compareByDescending<UpsertCandidate> { it.score }.thenBy { it.candidate.title }) .map { (candidate, _) ->
.take(mood.targetSize.coerceIn(10, 60))
.map {
RecommendationUpsert( RecommendationUpsert(
videoId = it.candidate.videoId, videoId = candidate.videoId,
title = it.candidate.title, title = candidate.title,
artist = it.candidate.artist, artist = candidate.artist,
album = it.candidate.album, album = candidate.album,
duration = it.candidate.duration, duration = candidate.duration,
thumbnail = it.candidate.thumbnail, thumbnail = candidate.thumbnail,
matchedSongId = it.matchedSongId,
) )
} }
val stored = MoodRepository.storeRecommendations(mood.id, upserts, now.toString()) if (upserts.isEmpty()) {
MoodRepository.markDiscovered(mood.id) MoodRepository.markDiscovered(mood.id)
log.info("YT discovery for mood {} stored {} recommendations", mood.name, stored) return DiscoveryOutcome(0)
return DiscoveryOutcome(
stored = stored,
matched = upserts.count { it.matchedSongId != null },
)
} }
private fun matchSong(candidate: YtCandidate, folders: Set<Int>, userName: String): String? { val stored = MoodRepository.storeCandidates(mood.id, upserts, now.toString(), CANDIDATE_POOL)
val titleKey = matchKey(candidate.title) MoodRepository.markDiscovered(mood.id)
if (titleKey.isEmpty()) return null log.info("YT discovery for mood {} stored {} candidates", mood.name, stored)
return LibraryRepository return DiscoveryOutcome(stored = stored)
.searchSongs(candidate.title, MATCH_LIMIT, 0, folders, userName)
.firstOrNull { song ->
matchKey(song.title) == titleKey && artistMatches(candidate.artist, song.displayArtist)
} }
?.id
}
private data class UpsertCandidate(
val candidate: YtCandidate,
val matchedSongId: String?,
val score: Double,
)
private companion object { private companion object {
const val QUERY_LIMIT = 10 const val QUERY_LIMIT = 10
const val MATCH_LIMIT = 25 const val CANDIDATE_POOL = 60
} }
} }
@@ -168,8 +128,9 @@ internal fun isFresh(lastDiscoveryAt: String?, refreshHours: Int, now: Instant):
return at.plus(refreshHours.toLong().coerceAtLeast(1), ChronoUnit.HOURS).isAfter(now) return at.plus(refreshHours.toLong().coerceAtLeast(1), ChronoUnit.HOURS).isAfter(now)
} }
/** Search and discovery only accept music-length tracks: 30 seconds to 10 minutes. */
internal fun isMusicLength(seconds: Int?): Boolean = internal fun isMusicLength(seconds: Int?): Boolean =
seconds == null || seconds in 45..(20 * 60) seconds == null || seconds in MIN_TRACK_SECONDS..MAX_TRACK_SECONDS
internal fun buildQueries(mood: MoodRecord, seedArtists: List<String>): List<String> { internal fun buildQueries(mood: MoodRecord, seedArtists: List<String>): List<String> {
val queries = LinkedHashSet<String>() val queries = LinkedHashSet<String>()
@@ -188,7 +149,6 @@ internal fun scoreCandidate(
candidate: YtCandidate, candidate: YtCandidate,
seedArtists: List<String>, seedArtists: List<String>,
moodText: String, moodText: String,
matched: Boolean,
): Double { ): Double {
var score = 0.0 var score = 0.0
val artist = matchKey(candidate.artist) val artist = matchKey(candidate.artist)
@@ -199,7 +159,6 @@ internal fun scoreCandidate(
} }
val text = matchKey(moodText) val text = matchKey(moodText)
if (text.isNotEmpty() && matchKey(candidate.title).contains(text)) score += 1.0 if (text.isNotEmpty() && matchKey(candidate.title).contains(text)) score += 1.0
if (matched) score += 1.5
if ((candidate.duration ?: 0) > 8 * 60) score -= 1.0 if ((candidate.duration ?: 0) > 8 * 60) score -= 1.0
return score return score
} }
@@ -209,7 +168,7 @@ private val NON_WORD = Regex("[^\\p{L}\\p{N} ]+")
private val SPACES = Regex("\\s+") private val SPACES = Regex("\\s+")
private val FEATURING = Regex("\\b(feat|ft|featuring)\\b.*$", RegexOption.IGNORE_CASE) private val FEATURING = Regex("\\b(feat|ft|featuring)\\b.*$", RegexOption.IGNORE_CASE)
/** Loose key for comparing YT titles and artist names with library tags. */ /** Loose key for comparing YT titles and artist names with downloaded tags. */
internal fun matchKey(value: String): String = internal fun matchKey(value: String): String =
IdGenerator.normalize(value) IdGenerator.normalize(value)
.replace(BRACKETS, " ") .replace(BRACKETS, " ")
@@ -227,3 +186,6 @@ internal fun artistMatches(a: String, b: String): Boolean {
if (left.length < 3 || right.length < 3) return false if (left.length < 3 || right.length < 3) return false
return " $left ".contains(" $right ") || " $right ".contains(" $left ") return " $left ".contains(" $right ") || " $right ".contains(" $left ")
} }
internal const val MIN_TRACK_SECONDS = 30
internal const val MAX_TRACK_SECONDS = 600
@@ -0,0 +1,77 @@
package win.taptap.musicmanager.discovery
import kotlinx.coroutines.CoroutineScope
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.launch
import kotlinx.coroutines.sync.Mutex
import kotlinx.coroutines.sync.withLock
import org.slf4j.LoggerFactory
import win.taptap.musicmanager.db.DownloadRepository
import win.taptap.musicmanager.db.MoodRecord
import win.taptap.musicmanager.db.MoodRepository
data class TopUpResult(
val promoted: Int,
val discoveryError: String? = null,
)
/**
* Keeps a mood's active recommendation slots at `recommendationCount`.
*
* Order: promote hidden candidates first; only run (rate-limited) discovery
* when the pool cannot fill the missing slots. User actions pass `force = true`
* so an add/reject/retry refills immediately.
*/
class TopUpService(
private val discovery: DiscoveryService,
private val backgroundScope: CoroutineScope? = null,
) {
private val log = LoggerFactory.getLogger(TopUpService::class.java)
private val mutex = Mutex()
/** Fire-and-forget refill used by API handlers after a user decision. */
fun trigger(mood: MoodRecord) {
val scope = backgroundScope ?: return
scope.launch(Dispatchers.IO) {
runCatching { topUp(mood, force = true) }
.onFailure { log.warn("Recommendation top-up for mood {} failed", mood.name, it) }
}
}
suspend fun topUp(mood: MoodRecord, force: Boolean = false): TopUpResult = mutex.withLock {
var current = MoodRepository.get(mood.id, mood.owner) ?: return@withLock TopUpResult(0)
if (current.recommendationCount <= 0) return@withLock TopUpResult(0)
var promoted = promoteMissing(current)
val stillMissing = current.recommendationCount - MoodRepository.activeCount(current.id)
var discoveryError: String? = null
if (stillMissing > 0 && (force || MoodRepository.isDiscoveryDue(current))) {
val outcome = discovery.refresh(current, force)
discoveryError = outcome.error
current = MoodRepository.get(mood.id, mood.owner) ?: current
promoted += promoteMissing(current)
}
if (promoted > 0) {
log.info("Top-up promoted {} recommendation slots for mood {}", promoted, mood.name)
}
TopUpResult(promoted, discoveryError)
}
private fun promoteMissing(mood: MoodRecord): Int {
val active = MoodRepository.activeCount(mood.id)
val missing = mood.recommendationCount - active
if (missing <= 0) return 0
val promoted = MoodRepository.promoteCandidates(mood.id, missing)
promoted.forEach { recommendation ->
DownloadRepository.enqueue(
videoId = recommendation.videoId,
moodId = mood.id,
kind = DownloadRepository.KIND_RECOMMENDATION,
)
}
return promoted.size
}
}
@@ -23,18 +23,23 @@ fun interface YtSearch {
fun search(query: String, limit: Int): List<YtCandidate> fun search(query: String, limit: Int): List<YtCandidate>
} }
/** Resolves a pasted watch URL into the same metadata shape as a search hit. */
fun interface YtResolver {
fun resolve(url: String): YtCandidate?
}
/** /**
* Searches YouTube through `yt-dlp`. Flat playlist output avoids resolving every * Talks to YouTube through `yt-dlp`. Flat playlist output avoids resolving every
* video, which keeps a discovery run to a single request per query. * video, which keeps a discovery run to a single request per query.
*/ */
class YtDlpSearch( class YtDlpSearch(
private val executable: String, private val executable: String,
private val timeoutSeconds: Long = 30, private val timeoutSeconds: Long = 30,
) : YtSearch { ) : YtSearch, YtResolver {
override fun search(query: String, limit: Int): List<YtCandidate> { override fun search(query: String, limit: Int): List<YtCandidate> {
val process = ProcessBuilder( val output = run(
executable, listOf(
"--flat-playlist", "--flat-playlist",
"--dump-single-json", "--dump-single-json",
"--no-warnings", "--no-warnings",
@@ -43,7 +48,32 @@ class YtDlpSearch(
"--socket-timeout", "--socket-timeout",
"10", "10",
"ytsearch${limit.coerceIn(1, 25)}:$query", "ytsearch${limit.coerceIn(1, 25)}:$query",
),
timeoutSeconds,
) )
return parseYtSearchOutput(output)
}
override fun resolve(url: String): YtCandidate? {
val trimmed = url.trim()
if (trimmed.isEmpty()) return null
val output = run(
listOf(
"--dump-single-json",
"--no-playlist",
"--no-warnings",
"--no-cache-dir",
"--socket-timeout",
"10",
trimmed,
),
timeoutSeconds,
)
return parseYtVideo(output)
}
private fun run(args: List<String>, timeout: Long): String {
val process = ProcessBuilder(listOf(executable) + args)
// Warnings and ERROR lines must not be mixed into stdout: the JSON // Warnings and ERROR lines must not be mixed into stdout: the JSON
// parser would fail and look like an empty result set. // parser would fail and look like an empty result set.
.redirectErrorStream(false) .redirectErrorStream(false)
@@ -51,22 +81,22 @@ class YtDlpSearch(
val output = StringBuilder() val output = StringBuilder()
val errors = StringBuilder() val errors = StringBuilder()
val reader = thread(isDaemon = true, name = "yt-dlp-search-stdout") { val reader = thread(isDaemon = true, name = "yt-dlp-stdout") {
process.inputStream.bufferedReader().useLines { lines -> process.inputStream.bufferedReader().useLines { lines ->
lines.forEach { output.appendLine(it) } lines.forEach { output.appendLine(it) }
} }
} }
val errorReader = thread(isDaemon = true, name = "yt-dlp-search-stderr") { val errorReader = thread(isDaemon = true, name = "yt-dlp-stderr") {
process.errorStream.bufferedReader().useLines { lines -> process.errorStream.bufferedReader().useLines { lines ->
lines.forEach { errors.appendLine(it) } lines.forEach { errors.appendLine(it) }
} }
} }
if (!process.waitFor(timeoutSeconds, TimeUnit.SECONDS)) { if (!process.waitFor(timeout, TimeUnit.SECONDS)) {
process.destroyForcibly() process.destroyForcibly()
reader.join(2_000) reader.join(2_000)
errorReader.join(2_000) errorReader.join(2_000)
throw IllegalStateException("yt-dlp timed out after ${timeoutSeconds}s") throw IllegalStateException("yt-dlp timed out after ${timeout}s")
} }
reader.join(2_000) reader.join(2_000)
errorReader.join(2_000) errorReader.join(2_000)
@@ -74,7 +104,7 @@ class YtDlpSearch(
val detail = (errors.toString().ifBlank { output.toString() }).take(400).trim() val detail = (errors.toString().ifBlank { output.toString() }).take(400).trim()
throw IllegalStateException("yt-dlp exited with ${process.exitValue()}: $detail") throw IllegalStateException("yt-dlp exited with ${process.exitValue()}: $detail")
} }
return parseYtSearchOutput(output.toString()) return output.toString()
} }
} }
@@ -84,22 +114,32 @@ internal fun parseYtSearchOutput(json: String): List<YtCandidate> {
val entries = runCatching { root["entries"]?.jsonArray }.getOrNull() ?: return emptyList() val entries = runCatching { root["entries"]?.jsonArray }.getOrNull() ?: return emptyList()
return entries.mapNotNull { element -> return entries.mapNotNull { element ->
val entry = runCatching { element.jsonObject }.getOrNull() ?: return@mapNotNull null runCatching { element.jsonObject }.getOrNull()?.toCandidate()
val videoId = entry.string("id")?.takeIf { it.isNotBlank() } ?: return@mapNotNull null }
val rawTitle = entry.string("title")?.takeIf { it.isNotBlank() } ?: return@mapNotNull null }
val channel = entry.string("channel") ?: entry.string("uploader").orEmpty()
/** Parses `yt-dlp --dump-single-json <url>` for a pasted URL. */
internal fun parseYtVideo(json: String): YtCandidate? {
if (json.isBlank()) return null
val root = runCatching { Json.parseToJsonElement(json).jsonObject }.getOrNull() ?: return null
return root.toCandidate()
}
private fun JsonObject.toCandidate(): YtCandidate? {
val videoId = string("id")?.takeIf { it.isNotBlank() } ?: return null
val rawTitle = string("title")?.takeIf { it.isNotBlank() } ?: return null
val channel = string("channel") ?: string("uploader").orEmpty()
val (artist, title) = splitYtTitle(rawTitle, channel) val (artist, title) = splitYtTitle(rawTitle, channel)
YtCandidate( return YtCandidate(
videoId = videoId, videoId = videoId,
title = title.ifEmpty { rawTitle }, title = title.ifEmpty { rawTitle },
artist = artist.ifEmpty { "Unknown artist" }, artist = artist.ifEmpty { "Unknown artist" },
album = entry.string("album"), album = string("album"),
duration = entry.number("duration")?.toInt(), duration = number("duration")?.toInt(),
thumbnail = entry.thumbnail(), thumbnail = thumbnail(),
) )
} }
}
private const val NOISE_WORDS = "official|lyrics?|audio|video|visuali[sz]er|mv|hd|4k|remaster(?:ed)?" private const val NOISE_WORDS = "official|lyrics?|audio|video|visuali[sz]er|mv|hd|4k|remaster(?:ed)?"
@@ -7,36 +7,36 @@ import java.nio.file.Path
import java.nio.file.ReadOnlyFileSystemException import java.nio.file.ReadOnlyFileSystemException
import java.nio.file.StandardCopyOption import java.nio.file.StandardCopyOption
import org.slf4j.LoggerFactory import org.slf4j.LoggerFactory
import win.taptap.musicmanager.config.FolderType
import win.taptap.musicmanager.config.LibraryConfig
import win.taptap.musicmanager.config.UsersConfig
import win.taptap.musicmanager.db.DownloadJobRecord import win.taptap.musicmanager.db.DownloadJobRecord
import win.taptap.musicmanager.db.DownloadRepository import win.taptap.musicmanager.db.DownloadRepository
import win.taptap.musicmanager.db.LibraryRepository
import win.taptap.musicmanager.db.MoodRepository import win.taptap.musicmanager.db.MoodRepository
import win.taptap.musicmanager.discovery.artistMatches import win.taptap.musicmanager.db.RecommendationCompletion
import win.taptap.musicmanager.discovery.matchKey import win.taptap.musicmanager.db.SongRecord
import win.taptap.musicmanager.util.IdGenerator import win.taptap.musicmanager.db.SongRepository
import win.taptap.musicmanager.media.MediaGcService
fun interface AudioDownloader { import win.taptap.musicmanager.media.MediaStore
/** Downloads [videoId] and returns the resulting MP3 file. */ import win.taptap.musicmanager.scanner.Mp3TagExtractor
fun download(videoId: String, title: String, artist: String, targetDir: Path): Path import win.taptap.musicmanager.scanner.SongIndexer
}
/** /**
* Downloads accepted recommendations that are not in the library: fetch the * Downloads queued videos, moves the MP3 into the shared media store, indexes it
* MP3, put it into the music folder, run a scan and promote the new song. * immediately and applies the job outcome:
*
* - `kind=song` attaches the indexed song to the requesting mood;
* - `kind=recommendation` promotes the recommendation slot to `ready`, or
* attaches it directly when the user pressed Add while it was downloading.
*
* A rejection that happens mid-download wins; the file is then collected by the
* GC because nothing references it.
*/ */
class DownloadService( class DownloadService(
private val users: UsersConfig,
private val library: LibraryConfig,
private val downloader: AudioDownloader, private val downloader: AudioDownloader,
private val mediaStore: MediaStore,
private val indexer: SongIndexer,
private val gc: MediaGcService,
private val tagExtractor: Mp3TagExtractor = Mp3TagExtractor(),
private val tempDir: Path, private val tempDir: Path,
private val startScan: () -> Boolean,
private val isScanning: () -> Boolean,
private val folderId: Int? = null,
private val maxAttempts: Int = 3, private val maxAttempts: Int = 3,
private val sleepMillis: (Long) -> Unit = { Thread.sleep(it) },
) { ) {
private val log = LoggerFactory.getLogger(DownloadService::class.java) private val log = LoggerFactory.getLogger(DownloadService::class.java)
@@ -47,10 +47,12 @@ class DownloadService(
process(job) process(job)
} catch (t: Throwable) { } catch (t: Throwable) {
val message = describe(t) val message = describe(t)
val attempts = if (isPermanent(t)) maxAttempts else job.attempts log.warn("Download {} failed (attempt {}): {}", job.videoId, job.attempts, message)
log.warn("Download {} failed (attempt {}): {}", job.videoId, attempts, message) if (job.attempts >= maxAttempts || isPermanent(t)) {
if (attempts >= maxAttempts) {
DownloadRepository.markFailed(job.id, message) DownloadRepository.markFailed(job.id, message)
if (job.kind == DownloadRepository.KIND_RECOMMENDATION) {
MoodRepository.failRecommendation(job.moodId, job.videoId, message, job.attempts)
}
} else { } else {
DownloadRepository.retryLater(job.id, message) DownloadRepository.retryLater(job.id, message)
} }
@@ -59,31 +61,24 @@ class DownloadService(
} }
private fun process(job: DownloadJobRecord) { private fun process(job: DownloadJobRecord) {
val user = users.byName(job.userName) // A previous job (other mood or other user) may already have the file.
?: throw IllegalStateException("Unknown user ${job.userName}") val existing = SongRepository.byVideoId(job.videoId)
val folders = library.folders val existingFile = existing?.let { mediaStore.file(it) }
.asSequence() if (existing != null && existingFile != null && Files.isRegularFile(existingFile)) {
.filter { it.type == FolderType.MUSIC && it.id in user.folders } finish(job, existing)
.associateBy { it.id } return
val folder = folderId?.let { folders[it] } }
?: folders.values.firstOrNull()
?: throw IllegalStateException("No writable music folder for ${job.userName}")
val importDir = prepareImportDir(folder.path)
val working = tempDir.resolve(job.id) val working = tempDir.resolve(job.id)
runCatching { working.toFile().deleteRecursively() }
Files.createDirectories(working) Files.createDirectories(working)
var promoted: Path? = null
try { try {
val downloaded = downloader.download(job.videoId, job.title, job.artist, working) val downloaded = downloader.download(job.videoId, working)
val extension = downloaded.fileName.toString().substringAfterLast('.', "mp3") val extracted = tagExtractor.extract(downloaded)
// One folder per artist/title keeps the scanner's album fallback ?: throw IllegalStateException("Downloaded file has no readable MP3 metadata")
// unique, so two downloads never overwrite each other's cover. val target = mediaStore.target(job.videoId, extracted.title)
val targetDir = importDir Files.createDirectories(target.parent)
.resolve(sanitize(job.artist))
.resolve(sanitize(job.title))
Files.createDirectories(targetDir)
val target = uniqueTarget(targetDir, job.title, extension)
try { try {
Files.move(downloaded, target, StandardCopyOption.REPLACE_EXISTING) Files.move(downloaded, target, StandardCopyOption.REPLACE_EXISTING)
} catch (e: IOException) { } catch (e: IOException) {
@@ -92,100 +87,45 @@ class DownloadService(
e, e,
) )
} }
promoted = target
log.info("Downloaded {} to {}", job.videoId, target) log.info("Downloaded {} to {}", job.videoId, target)
awaitScan() // A re-download may follow a title change; keep one file per video.
Files.list(target.parent).use { stream ->
// The scanner derives the song id from the folder and relative stream.filter { it != target && it.fileName.toString().endsWith(".mp3", ignoreCase = true) }
// path. Prefer that exact id: metadata matching could otherwise .forEach { runCatching { Files.deleteIfExists(it) } }
// attach the download to an older track with the same title.
val relativePath = folder.path.relativize(target).joinToString("/") { it.toString() }
val expectedId = IdGenerator.songId(folder.id, relativePath)
val songId = LibraryRepository.song(expectedId, folders.keys, user.name)?.id
?: matchSong(job, folders.keys, user.name)
?: throw IllegalStateException("Downloaded file was not indexed by the scanner")
DownloadRepository.markDone(job.id, songId)
val attached = MoodRepository.attachDownloadedSong(job.moodId, job.videoId, songId)
if (attached) {
MoodRepository.addSeed(job.moodId, job.userName, songId, "yt")
promoted = null
} else {
log.info("Recommendation {} was rejected; keeping the file but not seeding", job.videoId)
promoted = null
DownloadRepository.cancel(job.moodId, job.videoId, "Recommendation rejected")
} }
} catch (t: Throwable) {
// A failed verification must not leave a duplicate for the retry. val song = indexer.index(job.videoId, target)
promoted?.let { target -> runCatching { Files.deleteIfExists(target) } } ?: throw IllegalStateException("Downloaded file could not be indexed")
throw t finish(job, song)
} finally { } finally {
runCatching { working.toFile().deleteRecursively() } runCatching { working.toFile().deleteRecursively() }
} }
} }
private fun prepareImportDir(musicFolder: Path): Path { private fun finish(job: DownloadJobRecord, song: SongRecord) {
val dir = musicFolder.resolve(IMPORT_DIR) when (job.kind) {
try { DownloadRepository.KIND_SONG -> {
Files.createDirectories(dir) MoodRepository.addSong(job.moodId, song.id, "search")
} catch (e: IOException) { DownloadRepository.markDone(job.id, song.id)
throw IllegalStateException(
"Cannot write to the music folder: $dir. The server runs as UID 10001; " +
"make sure the hostPath is mounted read-write and that UID 10001 can write " +
"to it (the chart's prepare-imports init container does this). " +
"Underlying error: ${describe(e)}",
e,
)
}
return dir
} }
else -> {
when (MoodRepository.completeDownload(job.moodId, job.videoId, song.id)) {
RecommendationCompletion.READY,
RecommendationCompletion.ALREADY_ADDED,
-> DownloadRepository.markDone(job.id, song.id)
private fun awaitScan() { RecommendationCompletion.REJECTED,
// The downloaded file must be covered by a scan that starts *after* it RecommendationCompletion.MISSING,
// was moved. If a scan is already running, wait for it and then start a -> {
// fresh one; otherwise a slow scan could finish without seeing the file // The user rejected the track while it was downloading,
// and the job would download the same track again on retry. // or the mood was deleted. Drop the file if unused.
val deadline = System.nanoTime() + SCAN_TIMEOUT_MINUTES * 60_000_000_000L DownloadRepository.markFailed(job.id, "Recommendation is no longer active")
var attempts = 0 gc.collect(song.id)
while (!startScan()) {
attempts += 1
if (attempts >= MAX_SCAN_START_ATTEMPTS) {
throw IllegalStateException("Could not start a library scan")
}
awaitScanToFinish(deadline)
}
awaitScanToFinish(deadline)
}
private fun awaitScanToFinish(deadline: Long) {
while (isScanning()) {
if (System.nanoTime() > deadline) {
throw IllegalStateException("Library scan did not finish in time")
}
sleepMillis(250)
} }
} }
private fun matchSong(job: DownloadJobRecord, folders: Set<Int>, userName: String): String? {
val titleKey = matchKey(job.title)
if (titleKey.isEmpty()) return null
return LibraryRepository.searchSongs(job.title, 25, 0, folders, userName)
.firstOrNull {
artistMatches(job.artist, it.displayArtist) && matchKey(it.title) == titleKey
} }
?.id
} }
private fun uniqueTarget(dir: Path, title: String, extension: String): Path {
val base = sanitize(title)
val suffix = extension.ifBlank { "mp3" }.take(10)
var candidate = dir.resolve("$base.$suffix")
var counter = 2
while (Files.exists(candidate)) {
candidate = dir.resolve("$base ($counter).$suffix")
counter += 1
}
return candidate
} }
/** Includes the exception type and cause, which plain NIO messages omit. */ /** Includes the exception type and cause, which plain NIO messages omit. */
@@ -209,33 +149,4 @@ class DownloadService(
cause.message?.contains("Read-only", ignoreCase = true) == true || cause.message?.contains("Read-only", ignoreCase = true) == true ||
cause.message?.contains("Permission denied", ignoreCase = true) == true cause.message?.contains("Permission denied", ignoreCase = true) == true
} }
private fun sanitize(value: String): String =
truncateUtf8(
value.replace(Regex("[\\\\/:*?\"<>|\\p{Cntrl}]"), "_")
.replace(Regex("\\s+"), " ")
.trim()
.ifEmpty { "track" },
MAX_NAME_BYTES,
)
/** Linux NAME_MAX is 255 bytes, not characters; CJK titles are 3x. */
private fun truncateUtf8(value: String, maxBytes: Int): String {
var bytes = 0
val result = StringBuilder()
for (character in value) {
val length = character.toString().toByteArray(Charsets.UTF_8).size
if (bytes + length > maxBytes) break
result.append(character)
bytes += length
}
return result.toString().ifEmpty { "track" }
}
private companion object {
const val IMPORT_DIR = "AudioManager Imports"
const val SCAN_TIMEOUT_MINUTES = 10L
const val MAX_SCAN_START_ATTEMPTS = 5
const val MAX_NAME_BYTES = 120
}
} }
@@ -7,14 +7,19 @@ import kotlin.concurrent.thread
/** /**
* Downloads a single YouTube video as an MP3 with embedded metadata, so the * Downloads a single YouTube video as an MP3 with embedded metadata, so the
* scanner can derive artist and title from the file itself. * indexer can derive artist and title from the file itself.
*/ */
fun interface AudioDownloader {
/** Downloads [videoId] into [targetDir] and returns the resulting MP3 file. */
fun download(videoId: String, targetDir: Path): Path
}
class YtDlpDownloader( class YtDlpDownloader(
private val executable: String, private val executable: String,
private val timeoutSeconds: Long = 600, private val timeoutSeconds: Long = 600,
) : AudioDownloader { ) : AudioDownloader {
override fun download(videoId: String, title: String, artist: String, targetDir: Path): Path { override fun download(videoId: String, targetDir: Path): Path {
Files.createDirectories(targetDir) Files.createDirectories(targetDir)
val process = ProcessBuilder( val process = ProcessBuilder(
executable, executable,
@@ -62,9 +67,7 @@ class YtDlpDownloader(
errorReader.join(2_000) errorReader.join(2_000)
if (process.exitValue() != 0) { if (process.exitValue() != 0) {
val detail = (errors.toString().ifBlank { output.toString() }).take(400).trim() val detail = (errors.toString().ifBlank { output.toString() }).take(400).trim()
throw IllegalStateException( throw IllegalStateException("yt-dlp exited with ${process.exitValue()}: $detail")
"yt-dlp exited with ${process.exitValue()}: $detail",
)
} }
val printed = output.lines() val printed = output.lines()
@@ -0,0 +1,113 @@
package win.taptap.musicmanager.media
import java.nio.file.Files
import java.nio.file.attribute.FileTime
import java.time.Instant
import java.time.temporal.ChronoUnit
import org.jetbrains.exposed.sql.SqlExpressionBuilder.eq
import org.jetbrains.exposed.sql.SqlExpressionBuilder.inList
import org.jetbrains.exposed.sql.and
import org.jetbrains.exposed.sql.selectAll
import org.jetbrains.exposed.sql.transactions.transaction
import org.slf4j.LoggerFactory
import win.taptap.musicmanager.db.MoodRecommendationsTable
import win.taptap.musicmanager.db.MoodSongsTable
import win.taptap.musicmanager.db.PlaybackSessionsTable
import win.taptap.musicmanager.db.SessionItemsTable
import win.taptap.musicmanager.db.SongRecord
import win.taptap.musicmanager.db.SongRepository
import win.taptap.musicmanager.scanner.CoverArtService
/**
* Reference-counted file cleanup. A song's file is only deleted when no mood
* references it, no recommendation slot still points at it and no open playback
* session has it queued.
*/
class MediaGcService(
private val mediaStore: MediaStore,
private val coverArtService: CoverArtService,
) {
private val log = LoggerFactory.getLogger(MediaGcService::class.java)
/** Deletes every unreferenced song row, file and cover. Returns the count. */
fun collectGarbage(): Int {
var deleted = 0
SongRepository.all().forEach { song ->
if (collect(song)) deleted++
}
collectOrphanDirectories()
return deleted
}
/** Deletes [song] if it has no references. */
fun collect(song: SongRecord): Boolean {
if (isReferenced(song.id)) return false
if (!SongRepository.delete(song.id)) return false
coverArtService.deleteCover(song.id)
mediaStore.deleteSongDirectory(song.videoId)
log.info("GC removed unreferenced song {} ({})", song.id, song.videoId)
return true
}
fun collect(songId: String): Boolean {
val song = SongRepository.byId(songId) ?: return false
return collect(song)
}
fun isReferenced(songId: String): Boolean = transaction {
val inMood = MoodSongsTable.selectAll()
.where { MoodSongsTable.songId eq songId }
.any()
val inRecommendation = MoodRecommendationsTable.selectAll()
.where {
(MoodRecommendationsTable.songId eq songId) and
(MoodRecommendationsTable.state inList listOf("downloading", "ready", "added", "failed"))
}
.any()
val sessionIds = SessionItemsTable.select(SessionItemsTable.sessionId)
.where { SessionItemsTable.songId eq songId }
.map { it[SessionItemsTable.sessionId] }
val inOpenSession = if (sessionIds.isEmpty()) {
false
} else {
PlaybackSessionsTable.selectAll()
.where {
(PlaybackSessionsTable.id inList sessionIds) and
PlaybackSessionsTable.closedAt.isNull()
}
.any()
}
inMood || inRecommendation || inOpenSession
}
/**
* A crash between moving a file and indexing its row leaves a directory that
* no song points at. Only remove directories that have been idle for a while
* so a download that is still in progress is never touched.
*/
private fun collectOrphanDirectories() {
val cutoff = Instant.now().minus(ORPHAN_GRACE_MINUTES, ChronoUnit.MINUTES)
val known = SongRepository.all().map { it.videoId }.toSet()
mediaStore.songDirectories().forEach { directory ->
val videoId = directory.fileName?.toString() ?: return@forEach
if (videoId in known) return@forEach
val modified = runCatching {
Files.getLastModifiedTime(directory)
}.getOrNull() ?: return@forEach
if (modified.toInstant().isAfter(cutoff)) return@forEach
log.info("GC removed orphan media directory {}", directory)
runCatching { directory.toFile().deleteRecursively() }
}
}
private companion object {
const val ORPHAN_GRACE_MINUTES = 30L
}
}
@@ -0,0 +1,82 @@
package win.taptap.musicmanager.media
import java.nio.file.Files
import java.nio.file.Path
import win.taptap.musicmanager.db.SongRecord
/**
* Layout of the server-owned media root:
*
* <media>/<videoId>/<sanitized-title>.mp3
*
* The first path segment is the YouTube video id, which makes startup scans
* deterministic and lets the downloader find its file without waiting for one.
*/
class MediaStore(val root: Path) {
init {
Files.createDirectories(root)
}
fun file(song: SongRecord): Path? = resolveRelative(song.relativePath)
fun resolveRelative(relativePath: String): Path? {
val candidate = root.resolve(relativePath).normalize()
return candidate.takeIf { it.startsWith(root) }
}
/** Target path for a download. The title is sanitised for the filesystem. */
fun target(videoId: String, title: String): Path =
root.resolve(videoId).resolve("${sanitizeName(title)}.mp3")
fun relativePath(videoId: String, title: String): String =
"$videoId/${sanitizeName(title)}.mp3"
fun videoIdFrom(relativePath: String): String? =
relativePath.replace('\\', '/')
.split('/')
.firstOrNull()
?.takeIf { it.isNotBlank() && it != "." && it != ".." }
fun deleteSongDirectory(videoId: String) {
val dir = root.resolve(videoId).normalize()
if (!dir.startsWith(root) || dir == root) return
runCatching { dir.toFile().deleteRecursively() }
}
fun songDirectories(): List<Path> {
if (!Files.isDirectory(root)) return emptyList()
return Files.list(root).use { stream ->
stream.filter { Files.isDirectory(it) }.toList()
}
}
companion object {
private const val MAX_NAME_BYTES = 120
private val INVALID = Regex("[\\\\/:*?\"<>|\\p{Cntrl}]")
private val SPACES = Regex("\\s+")
fun sanitizeName(value: String): String =
truncateUtf8(
value.replace(INVALID, "_")
.replace(SPACES, " ")
.trim()
.trim('.')
.ifEmpty { "track" },
MAX_NAME_BYTES,
)
/** Linux NAME_MAX is 255 bytes, not characters; CJK titles are 3x. */
fun truncateUtf8(value: String, maxBytes: Int): String {
var bytes = 0
val result = StringBuilder()
for (character in value) {
val length = character.toString().toByteArray(Charsets.UTF_8).size
if (bytes + length > maxBytes) break
result.append(character)
bytes += length
}
return result.toString().ifEmpty { "track" }
}
}
}
@@ -6,10 +6,7 @@ import java.util.UUID
import kotlin.math.max import kotlin.math.max
import kotlin.random.Random import kotlin.random.Random
import org.slf4j.LoggerFactory import org.slf4j.LoggerFactory
import win.taptap.musicmanager.api.musicFolderIds
import win.taptap.musicmanager.config.LibraryConfig
import win.taptap.musicmanager.config.UserAccount import win.taptap.musicmanager.config.UserAccount
import win.taptap.musicmanager.db.LibraryRepository
import win.taptap.musicmanager.db.MoodRecord import win.taptap.musicmanager.db.MoodRecord
import win.taptap.musicmanager.db.MoodRepository import win.taptap.musicmanager.db.MoodRepository
import win.taptap.musicmanager.db.PlaybackSessionRecord import win.taptap.musicmanager.db.PlaybackSessionRecord
@@ -24,11 +21,11 @@ data class StartedSession(
/** /**
* Server-driven mood radio. * Server-driven mood radio.
* *
* A batch is a weighted sample of the mood's world: seeds are guaranteed * A batch mixes the mood's permanent collection with its ready recommendation
* candidates, related genres/artists and free-text matches shape the pool, and * songs. Every slot rolls `collectionPercent` independently, so a 70% mood gets
* per-user completions/skips nudge the ranking. Tracks already queued in the * roughly seven collection tracks in ten. Per-user completions/skips nudge the
* session are never repeated. The client asks for more whenever its buffer is * ranking. Tracks already queued in the session are never repeated until the
* low, so the queue is always generated with the latest signals. * pool is exhausted; then the least-recently-heard tracks replay.
*/ */
class RadioService( class RadioService(
private val random: Random = Random.Default, private val random: Random = Random.Default,
@@ -36,14 +33,9 @@ class RadioService(
private val log = LoggerFactory.getLogger(RadioService::class.java) private val log = LoggerFactory.getLogger(RadioService::class.java)
fun start( fun start(mood: MoodRecord, user: UserAccount, limit: Int): StartedSession {
mood: MoodRecord,
user: UserAccount,
library: LibraryConfig,
limit: Int,
): StartedSession {
val session = RadioRepository.createSession(user.name, mood.id, UUID.randomUUID().toString()) val session = RadioRepository.createSession(user.name, mood.id, UUID.randomUUID().toString())
val songs = generate(mood, user, library, session.id, emptySet(), limit, persist = true) val songs = generate(mood, user, session.id, emptySet(), limit, persist = true)
log.info("Radio session {} started for mood {} with {} tracks", session.id, mood.name, songs.size) log.info("Radio session {} started for mood {} with {} tracks", session.id, mood.name, songs.size)
return StartedSession(session, songs) return StartedSession(session, songs)
} }
@@ -51,34 +43,31 @@ class RadioService(
fun next( fun next(
mood: MoodRecord, mood: MoodRecord,
user: UserAccount, user: UserAccount,
library: LibraryConfig,
sessionId: String, sessionId: String,
excluded: Set<String>, excluded: Set<String>,
limit: Int, limit: Int,
): List<SongRecord> = ): List<SongRecord> = generate(mood, user, sessionId, excluded, limit, persist = true)
generate(mood, user, library, sessionId, excluded, limit, persist = true)
/** Queue preview for the mood screen; nothing is persisted. */ /** Queue preview for the mood screen; nothing is persisted. */
fun preview( fun preview(mood: MoodRecord, user: UserAccount, limit: Int): List<SongRecord> =
mood: MoodRecord, generate(mood, user, sessionId = null, excluded = emptySet(), limit, persist = false)
user: UserAccount,
library: LibraryConfig,
limit: Int,
): List<SongRecord> =
generate(mood, user, library, sessionId = null, excluded = emptySet(), limit, persist = false)
private fun generate( private fun generate(
mood: MoodRecord, mood: MoodRecord,
user: UserAccount, user: UserAccount,
library: LibraryConfig,
sessionId: String?, sessionId: String?,
excluded: Set<String>, excluded: Set<String>,
requestedLimit: Int, requestedLimit: Int,
persist: Boolean, persist: Boolean,
): List<SongRecord> { ): List<SongRecord> {
val limit = requestedLimit.coerceIn(1, 50) val limit = requestedLimit.coerceIn(1, 50)
val folders = user.musicFolderIds(library) val collection = MoodRepository.songRecords(mood.id)
if (folders.isEmpty()) return emptyList() // A mood can only play once it has at least one permanent song.
if (collection.isEmpty()) return emptyList()
val collectionIds = collection.map { it.id }.toSet()
val recommendations = MoodRepository.readySongs(mood.id)
.filterNot { it.id in collectionIds }
val sessionSongs = if (sessionId != null && persist) { val sessionSongs = if (sessionId != null && persist) {
RadioRepository.queuedSongIds(sessionId) RadioRepository.queuedSongIds(sessionId)
@@ -86,180 +75,111 @@ class RadioService(
emptySet() emptySet()
} }
val skip = sessionSongs + excluded val skip = sessionSongs + excluded
val seeds = LibraryRepository.songsByIds(
MoodRepository.seedSongIds(mood.id),
folders,
user.name,
)
val seedIds = seeds.map { it.id }.toSet()
val seedGenres = (seeds.flatMap { it.genres } + mood.genres)
.map { it.trim() }
.filter { it.isNotEmpty() }
.distinctBy { it.lowercase() }
val seedArtistIds = seeds.mapNotNull { it.artistId }.distinct()
val query = mood.query?.trim()?.takeIf { it.isNotEmpty() }
val candidates = LinkedHashMap<String, SongRecord>()
seeds.forEach { candidates[it.id] = it }
if (seedGenres.isNotEmpty()) {
LibraryRepository.songsByGenres(seedGenres, POOL_LIMIT, folders, user.name)
.forEach { candidates[it.id] = it }
}
if (seedArtistIds.isNotEmpty()) {
LibraryRepository.songsByArtists(seedArtistIds, POOL_LIMIT, folders, user.name)
.forEach { candidates[it.id] = it }
}
if (query != null) {
LibraryRepository.searchSongs(query, POOL_LIMIT, 0, folders, user.name)
.forEach { candidates[it.id] = it }
}
// A mood with no seeds and no filters still needs a starting point.
LibraryRepository.randomSongs(
size = POOL_LIMIT,
genre = null,
fromYear = null,
toYear = null,
folderIds = folders,
userName = user.name,
).forEach { candidates[it.id] = it }
candidates.keys.removeAll(skip)
// Small libraries can exhaust the mood. Replay tracks the session already
// heard instead of returning silence, but never repeat the current batch.
if (candidates.size < limit) {
refill(candidates, folders, user.name, excluded, sessionSongs, emptySet(), limit)
}
val recent = RadioRepository.recentSongIds( val recent = RadioRepository.recentSongIds(
user.name, user.name,
Instant.now().minus(RECENT_DAYS, ChronoUnit.DAYS), Instant.now().minus(RECENT_DAYS, ChronoUnit.DAYS),
) )
// In a normal-sized library, skip songs from the last week entirely. val affinity = RadioRepository.affinity(
// Small libraries keep them as low-scored fallbacks instead of going silent. user.name,
if (candidates.size > limit * 3) { (collection.map { it.id } + recommendations.map { it.id }).toSet(),
candidates.keys.removeAll(recent - seedIds)
}
// Pruning can empty the pool even though the library has songs. Prefer
// non-recent tracks, then fall back to replay if the library is tiny.
if (candidates.size < limit) {
refill(candidates, folders, user.name, excluded, sessionSongs, recent, limit)
if (candidates.size < limit) {
refill(candidates, folders, user.name, excluded, emptySet(), emptySet(), limit)
}
}
val affinity = RadioRepository.affinity(user.name, candidates.keys)
val scored = candidates.values.map { song ->
song to scoreCandidate(
song = song,
seedIds = seedIds,
seedGenres = seedGenres,
seedArtistIds = seedArtistIds,
query = query,
affinity = affinity[song.id]?.score ?: 0.0,
recent = recent,
jitter = random.nextDouble(0.0, 2.0),
) )
}
val seedScored = scored.filter { (song, _) -> song.id in seedIds } val freshCollection = collection.filterNot { it.id in skip }
val recommendationScored = scored.filter { (song, _) -> song.id !in seedIds } val freshRecommendations = recommendations.filterNot { it.id in skip }
val picks = pickMixed( val picks = pickMixed(
seedScored = seedScored, collectionScored = score(freshCollection, affinity, recent, heard = false),
recommendationScored = recommendationScored, recommendationScored = score(freshRecommendations, affinity, recent, heard = false),
limit = limit, limit = limit,
seedPercent = mood.seedPercent, collectionPercent = mood.collectionPercent,
).toMutableList()
// Small moods exhaust their pool quickly. Replay the least-recently
// queued tracks instead of returning silence, but never duplicate
// within one batch.
if (picks.size < limit) {
val pickedIds = picks.map { it.id }.toSet()
val heardCollection = collection.filter { it.id in skip && it.id !in pickedIds }
val heardRecommendations = recommendations.filter { it.id in skip && it.id !in pickedIds }
picks += pickMixed(
collectionScored = score(heardCollection, affinity, recent, heard = true),
recommendationScored = score(heardRecommendations, affinity, recent, heard = true),
limit = limit - picks.size,
collectionPercent = mood.collectionPercent,
) )
}
if (persist && sessionId != null && picks.isNotEmpty()) { if (persist && sessionId != null && picks.isNotEmpty()) {
RadioRepository.append(sessionId, picks.map { it.id }) RadioRepository.append(sessionId, picks.map { it.id })
} }
return picks return picks
} }
private fun refill( private fun score(
candidates: LinkedHashMap<String, SongRecord>, songs: List<SongRecord>,
folders: Set<Int>, affinity: Map<String, win.taptap.musicmanager.db.AffinityRecord>,
userName: String, recent: Set<String>,
excluded: Set<String>, heard: Boolean,
avoidSession: Set<String>, ): List<Pair<SongRecord, Double>> = songs.map { song ->
avoidRecent: Set<String>, song to scoreCandidate(
limit: Int, song = song,
) { affinity = affinity[song.id]?.score ?: 0.0,
LibraryRepository.randomSongs( recent = song.id in recent,
size = max(limit * 10, 200), jitter = random.nextDouble(0.0, 2.0),
genre = null, heard = heard,
fromYear = null, )
toYear = null,
folderIds = folders,
userName = userName,
).forEach { song ->
if (song.id !in excluded && song.id !in avoidSession && song.id !in avoidRecent) {
candidates.putIfAbsent(song.id, song)
}
}
} }
internal fun scoreCandidate( internal fun scoreCandidate(
song: SongRecord, song: SongRecord,
seedIds: Set<String>,
seedGenres: List<String>,
seedArtistIds: List<String>,
query: String?,
affinity: Double, affinity: Double,
recent: Set<String> = emptySet(), recent: Boolean = false,
jitter: Double = 0.0, jitter: Double = 0.0,
heard: Boolean = false,
): Double { ): Double {
var score = jitter var score = jitter + affinity.coerceIn(-8.0, 8.0)
if (song.id in seedIds) score += 10.0 if (recent) score -= RECENT_PENALTY
if (song.artistId != null && song.artistId in seedArtistIds) score += 3.0 if (heard) score -= HEARD_PENALTY
score += song.genres.count { candidate ->
seedGenres.any { it.equals(candidate, ignoreCase = true) }
} * 1.5
if (query != null && song.title.contains(query, ignoreCase = true)) score += 2.0
score += affinity.coerceIn(-8.0, 8.0)
if (song.id in recent) score -= RECENT_PENALTY
return score return score
} }
/** /**
* Picks a batch with the mood's requested seed share. Each slot rolls the * Picks a batch with the mood's requested collection share. Each slot rolls
* percentage independently, so a 70% mood gets roughly seven seed tracks in * the percentage independently. An exhausted pool falls back to the other
* ten. An exhausted pool falls back to the other one instead of stalling. * one instead of stalling.
*/ */
internal fun pickMixed( internal fun pickMixed(
seedScored: List<Pair<SongRecord, Double>>, collectionScored: List<Pair<SongRecord, Double>>,
recommendationScored: List<Pair<SongRecord, Double>>, recommendationScored: List<Pair<SongRecord, Double>>,
limit: Int, limit: Int,
seedPercent: Int, collectionPercent: Int,
artistSpacing: Int = 2, artistSpacing: Int = 2,
albumSpacing: Int = 3, albumSpacing: Int = 3,
): List<SongRecord> { ): List<SongRecord> {
val seeds = seedScored val collection = collectionScored
.sortedByDescending { it.second } .sortedByDescending { it.second }
.toMutableList() .toMutableList()
val recommendations = recommendationScored val recommendations = recommendationScored
.sortedByDescending { it.second } .sortedByDescending { it.second }
.toMutableList() .toMutableList()
val percent = seedPercent.coerceIn(0, 100) val percent = collectionPercent.coerceIn(0, 100)
val picked = ArrayList<SongRecord>(limit) val picked = ArrayList<SongRecord>(limit)
val artists = ArrayDeque<String>() val artists = ArrayDeque<String>()
val albums = ArrayDeque<String>() val albums = ArrayDeque<String>()
while (picked.size < limit && (seeds.isNotEmpty() || recommendations.isNotEmpty())) { while (picked.size < limit && (collection.isNotEmpty() || recommendations.isNotEmpty())) {
val preferSeed = when { val preferCollection = when {
seeds.isEmpty() -> false collection.isEmpty() -> false
recommendations.isEmpty() -> true recommendations.isEmpty() -> true
else -> random.nextInt(100) < percent else -> random.nextInt(100) < percent
} }
val pool = if (preferSeed) seeds else recommendations val pool = if (preferCollection) collection else recommendations
val song = pool.removeAt(diverseIndex(pool, artists, albums)).first val song = pool.removeAt(diverseIndex(pool, artists, albums)).first
picked += song picked += song
artists.addLast(song.displayArtist) artists.addLast(song.artist)
if (artists.size > artistSpacing) artists.removeFirst() if (artists.size > artistSpacing) artists.removeFirst()
albums.addLast(song.albumId) albums.addLast(albumKey(song))
if (albums.size > albumSpacing) albums.removeFirst() if (albums.size > albumSpacing) albums.removeFirst()
} }
return picked return picked
@@ -271,53 +191,17 @@ class RadioService(
albums: ArrayDeque<String>, albums: ArrayDeque<String>,
): Int { ): Int {
var index = pool.indexOfFirst { (song, _) -> var index = pool.indexOfFirst { (song, _) ->
song.displayArtist !in artists && song.albumId !in albums song.artist !in artists && albumKey(song) !in albums
} }
if (index < 0) index = pool.indexOfFirst { (song, _) -> song.displayArtist !in artists } if (index < 0) index = pool.indexOfFirst { (song, _) -> song.artist !in artists }
return if (index < 0) 0 else index return if (index < 0) 0 else index
} }
/** private fun albumKey(song: SongRecord): String = song.album?.takeIf { it.isNotBlank() } ?: song.id
* Picks the best remaining candidate while keeping artists and albums apart.
* An artist may return after [artistSpacing] other tracks, an album after
* [albumSpacing]. Fallbacks relax the constraints rather than stalling.
*/
internal fun pickDiverse(
scored: List<Pair<SongRecord, Double>>,
limit: Int,
artistSpacing: Int = 2,
albumSpacing: Int = 3,
): List<SongRecord> {
val remaining = scored
.sortedWith(compareByDescending<Pair<SongRecord, Double>> { it.second })
.toMutableList()
val picked = ArrayList<SongRecord>(limit)
val artists = ArrayDeque<String>()
val albums = ArrayDeque<String>()
while (picked.size < limit && remaining.isNotEmpty()) {
var index = remaining.indexOfFirst { (song, _) ->
song.displayArtist !in artists && song.albumId !in albums
}
if (index < 0) {
index = remaining.indexOfFirst { (song, _) -> song.displayArtist !in artists }
}
if (index < 0) index = 0
val song = remaining.removeAt(index).first
picked += song
artists.addLast(song.displayArtist)
if (artists.size > artistSpacing) artists.removeFirst()
albums.addLast(song.albumId)
if (albums.size > albumSpacing) albums.removeFirst()
}
return picked
}
private companion object { private companion object {
const val POOL_LIMIT = 400
const val RECENT_DAYS = 7L const val RECENT_DAYS = 7L
const val RECENT_PENALTY = 6.0 const val RECENT_PENALTY = 6.0
const val HEARD_PENALTY = 20.0
} }
} }
@@ -11,11 +11,10 @@ import net.coobird.thumbnailator.Thumbnails
import org.slf4j.LoggerFactory import org.slf4j.LoggerFactory
/** /**
* Extracts, normalises and caches album covers under `DATA_DIR/covers`. * Extracts, normalises and caches cover art under `DATA_DIR/covers`, keyed by
* * song id. Covers are always stored as JPEG. Original-size files are named
* Covers are always stored as JPEG. Original-size files are named `{albumId}.jpg`, * `{songId}.jpg`, resized variants `{songId}_{size}.jpg`. When a cover changes,
* resized variants `{albumId}_{size}.jpg`. When a cover changes, its variants are * its variants are invalidated.
* invalidated.
*/ */
class CoverArtService(dataDir: Path) { class CoverArtService(dataDir: Path) {
@@ -26,23 +25,23 @@ class CoverArtService(dataDir: Path) {
Files.createDirectories(coversDir) Files.createDirectories(coversDir)
} }
fun coverFile(albumId: String): Path? = fun coverFile(songId: String): Path? =
coversDir.resolve("$albumId.jpg").takeIf { Files.isRegularFile(it) } coversDir.resolve("$songId.jpg").takeIf { Files.isRegularFile(it) }
/** /**
* Stores [bytes] as the album cover. Returns true when a usable cover exists, * Stores [bytes] as the song cover. Returns true when a usable cover exists,
* including when the bytes were unchanged. * including when the bytes were unchanged.
*/ */
@Synchronized @Synchronized
fun saveCover(albumId: String, bytes: ByteArray): Boolean { fun saveCover(songId: String, bytes: ByteArray): Boolean {
if (bytes.isEmpty() || bytes.size > MAX_COVER_BYTES) return false if (bytes.isEmpty() || bytes.size > MAX_COVER_BYTES) return false
val image = runCatching { val image = runCatching {
ImageIO.read(ByteArrayInputStream(bytes)) ImageIO.read(ByteArrayInputStream(bytes))
}.getOrNull() ?: return false }.getOrNull() ?: return false
val target = coversDir.resolve("$albumId.jpg") val target = coversDir.resolve("$songId.jpg")
val temp = Files.createTempFile(coversDir, "$albumId-", ".jpg") val temp = Files.createTempFile(coversDir, "$songId-", ".jpg")
return try { return try {
writeJpeg(image, temp.toFile()) writeJpeg(image, temp.toFile())
@@ -52,10 +51,10 @@ class CoverArtService(dataDir: Path) {
} }
moveReplacing(temp, target) moveReplacing(temp, target)
deleteVariants(albumId) deleteVariants(songId)
true true
} catch (e: Exception) { } catch (e: Exception) {
log.warn("Could not store cover for album {}: {}", albumId, e.message) log.warn("Could not store cover for song {}: {}", songId, e.message)
false false
} finally { } finally {
Files.deleteIfExists(temp) Files.deleteIfExists(temp)
@@ -64,14 +63,14 @@ class CoverArtService(dataDir: Path) {
/** Returns a cached `size x size` variant, generating it on first request. */ /** Returns a cached `size x size` variant, generating it on first request. */
@Synchronized @Synchronized
fun resizedCover(albumId: String, size: Int): Path? { fun resizedCover(songId: String, size: Int): Path? {
require(size in 1..MAX_RESIZE) { "Cover size must be between 1 and $MAX_RESIZE" } require(size in 1..MAX_RESIZE) { "Cover size must be between 1 and $MAX_RESIZE" }
val base = coverFile(albumId) ?: return null val base = coverFile(songId) ?: return null
val target = coversDir.resolve("${albumId}_$size.jpg") val target = coversDir.resolve("${songId}_$size.jpg")
if (Files.isRegularFile(target)) return target if (Files.isRegularFile(target)) return target
val temp = Files.createTempFile(coversDir, "${albumId}_$size-", ".jpg") val temp = Files.createTempFile(coversDir, "${songId}_$size-", ".jpg")
return try { return try {
Thumbnails.of(base.toFile()) Thumbnails.of(base.toFile())
.size(size, size) .size(size, size)
@@ -82,13 +81,18 @@ class CoverArtService(dataDir: Path) {
moveReplacing(temp, target) moveReplacing(temp, target)
target target
} catch (e: Exception) { } catch (e: Exception) {
log.warn("Could not resize cover for album {}: {}", albumId, e.message) log.warn("Could not resize cover for song {}: {}", songId, e.message)
null null
} finally { } finally {
Files.deleteIfExists(temp) Files.deleteIfExists(temp)
} }
} }
fun deleteCover(songId: String) {
Files.deleteIfExists(coversDir.resolve("$songId.jpg"))
deleteVariants(songId)
}
fun findSidecarCover(audioFile: Path): Path? { fun findSidecarCover(audioFile: Path): Path? {
val directory = audioFile.parent ?: return null val directory = audioFile.parent ?: return null
return SIDECAR_NAMES return SIDECAR_NAMES
@@ -116,10 +120,10 @@ class CoverArtService(dataDir: Path) {
} }
} }
private fun deleteVariants(albumId: String) { private fun deleteVariants(songId: String) {
Files.list(coversDir).use { stream -> Files.list(coversDir).use { stream ->
stream stream
.filter { it.name.startsWith("${albumId}_") && it.name.endsWith(".jpg") } .filter { it.name.startsWith("${songId}_") && it.name.endsWith(".jpg") }
.forEach { runCatching { Files.deleteIfExists(it) } } .forEach { runCatching { Files.deleteIfExists(it) } }
} }
} }
@@ -1,26 +1,26 @@
package win.taptap.musicmanager.scanner package win.taptap.musicmanager.scanner
import java.nio.file.Files import java.nio.file.Files
import java.nio.file.Path
import java.nio.file.attribute.BasicFileAttributes
import java.util.concurrent.atomic.AtomicBoolean import java.util.concurrent.atomic.AtomicBoolean
import java.util.concurrent.atomic.AtomicReference import java.util.concurrent.atomic.AtomicReference
import kotlinx.coroutines.CoroutineScope import kotlinx.coroutines.CoroutineScope
import kotlinx.coroutines.launch import kotlinx.coroutines.launch
import org.slf4j.LoggerFactory import org.slf4j.LoggerFactory
import win.taptap.musicmanager.config.LibraryConfig import win.taptap.musicmanager.db.SongRepository
import win.taptap.musicmanager.config.MusicFolder import win.taptap.musicmanager.media.MediaStore
import win.taptap.musicmanager.db.MediaRepository
/** /**
* Recursive MP3 scanner with delta updates and orphan cleanup. * Rebuilds the song index from the media root. The first path segment is the
* YouTube video id, so a file's identity never depends on its tags.
* *
* Only one scan can run at a time. The file's `(mtime, size)` pair decides whether * Only one scan can run at a time. A file's `(mtime, size)` pair decides whether
* tags need to be re-read; unchanged files are skipped. * tags need to be re-read; unchanged files are skipped.
*/ */
class MediaScanner( class MediaScanner(
private val library: LibraryConfig, private val mediaStore: MediaStore,
private val coverArtService: CoverArtService, private val indexer: SongIndexer,
private val tagExtractor: Mp3TagExtractor = Mp3TagExtractor(),
private val repository: MediaRepository = MediaRepository,
private val onScanFinished: (ScanStatus) -> Unit = {}, private val onScanFinished: (ScanStatus) -> Unit = {},
) { ) {
@@ -49,92 +49,82 @@ class MediaScanner(
var count = 0L var count = 0L
val seenSongIds = mutableSetOf<String>() val seenSongIds = mutableSetOf<String>()
var error: String? = null var error: String? = null
var folderFailed = false var rootFailed = false
try { try {
library.folders.forEach { folder -> val root = mediaStore.root
if (!Files.isDirectory(root) || !Files.isReadable(root)) {
rootFailed = true
error = "Media root is not readable: $root"
log.error("Skipping scan: media root is not readable: {}", root)
} else {
Files.walk(root).use { paths ->
paths.filter { Files.isRegularFile(it) }.forEach { file ->
if (!file.fileName.toString().endsWith(".mp3", ignoreCase = true)) {
return@forEach
}
val relativePath = root.relativize(file).joinToString("/") { it.toString() }
val videoId = mediaStore.videoIdFrom(relativePath)
if (videoId == null) {
log.debug("Skipping file outside a video folder: {}", file)
return@forEach
}
try { try {
scanFolder(folder, seenSongIds) { val existing = SongRepository.byVideoId(videoId)
val attributes = Files.readAttributes(
file,
BasicFileAttributes::class.java,
)
val mtime = attributes.lastModifiedTime().toMillis()
val size = attributes.size()
if (existing != null && existing.fileMtime == mtime && existing.size == size) {
seenSongIds += existing.id
count++ count++
status.set(ScanStatus(scanning = true, count = count)) status.set(ScanStatus(scanning = true, count = count))
return@forEach
} }
val song = indexer.index(videoId, file)
if (song == null) {
if (existing != null) seenSongIds += existing.id
log.warn("Could not read MP3 metadata: {}", file)
return@forEach
}
seenSongIds += song.id
count++
status.set(ScanStatus(scanning = true, count = count))
} catch (e: Exception) { } catch (e: Exception) {
folderFailed = true SongRepository.byVideoId(videoId)?.let { seenSongIds += it.id }
if (error == null) { log.warn("Could not index file: $file", e)
error = "Could not scan folder ${folder.id} (${folder.path}): " + }
(e.message ?: e::class.simpleName)
} }
log.error("Could not scan folder ${folder.id} (${folder.path})", e)
} }
} }
if (folderFailed) { if (rootFailed) {
// An unmounted or temporarily unreadable folder must not be // An unmounted or temporarily unreadable root must not be
// mistaken for deleted files. Keep the last known library. // mistaken for deleted files. Keep the last known index.
log.warn("Skipping orphan cleanup because at least one folder failed to scan") log.warn("Skipping orphan cleanup because the media root failed to scan")
} else { } else {
repository.deleteSongsNotIn(seenSongIds) SongRepository.all().forEach { song ->
if (song.id !in seenSongIds) {
SongRepository.delete(song.id)
}
}
} }
repository.rebuildAggregates()
} catch (t: Throwable) { } catch (t: Throwable) {
error = t.message ?: t::class.simpleName error = t.message ?: t::class.simpleName
log.error("Library scan failed after $count files", t) log.error("Media scan failed after $count files", t)
} finally { } finally {
running.set(false) running.set(false)
} }
val finalStatus = ScanStatus(scanning = false, count = count, error = error) val finalStatus = ScanStatus(scanning = false, count = count, error = error)
status.set(finalStatus) status.set(finalStatus)
log.info("Library scan finished: {} files, error={}", count, error) log.info("Media scan finished: {} files, error={}", count, error)
runCatching { onScanFinished(finalStatus) } runCatching { onScanFinished(finalStatus) }
.onFailure { log.warn("Scan completion hook failed", it) } .onFailure { log.warn("Scan completion hook failed", it) }
return finalStatus return finalStatus
} }
private fun scanFolder(folder: MusicFolder, seenSongIds: MutableSet<String>, onFile: () -> Unit) {
Files.walk(folder.path).use { paths ->
paths.filter { Files.isRegularFile(it) }.forEach { file ->
if (!file.fileName.toString().endsWith(".mp3", ignoreCase = true)) {
log.debug("Skipping unsupported file: {}", file)
return@forEach
}
onFile()
val relativePath = folder.path.relativize(file).joinToString("/") { it.toString() }
val existing = repository.findExisting(folder.id, relativePath)
try {
val attributes = Files.readAttributes(file, java.nio.file.attribute.BasicFileAttributes::class.java)
val mtime = attributes.lastModifiedTime().toMillis()
val size = attributes.size()
if (existing != null && existing.fileMtime == mtime && existing.size == size) {
seenSongIds += existing.id
return@forEach
}
val track = tagExtractor.extract(folder.id, folder.path, file)
if (track == null) {
if (existing != null) seenSongIds += existing.id
log.warn("Could not read MP3 metadata: {}", file)
return@forEach
}
val coverBytes = track.coverBytes
?: coverArtService.findSidecarCover(file)?.let(coverArtService::readCover)
repository.upsertTrack(track)
if (coverBytes != null && coverArtService.saveCover(track.albumId, coverBytes)) {
repository.setAlbumCover(track.albumId)
}
seenSongIds += track.songId
} catch (e: Exception) {
if (existing != null) seenSongIds += existing.id
log.warn("Could not index file: $file", e)
}
}
}
}
} }
@@ -8,25 +8,11 @@ import java.util.logging.Logger
import org.jaudiotagger.audio.AudioFileIO import org.jaudiotagger.audio.AudioFileIO
import org.jaudiotagger.tag.FieldKey import org.jaudiotagger.tag.FieldKey
import org.jaudiotagger.tag.Tag import org.jaudiotagger.tag.Tag
import win.taptap.musicmanager.util.IdGenerator
data class ArtistRef(val id: String, val name: String) data class ExtractedSong(
data class ExtractedTrack(
val songId: String,
val albumId: String,
val albumArtistId: String,
val albumArtists: List<ArtistRef>,
val artists: List<ArtistRef>,
val title: String, val title: String,
val displayArtist: String, val artist: String,
val albumTitle: String, val album: String?,
val folderId: Int,
val relativePath: String,
val trackNumber: Int?,
val discNumber: Int,
val year: Int?,
val genres: List<String>,
val durationSeconds: Int, val durationSeconds: Int,
val bitRate: Int, val bitRate: Int,
val size: Long, val size: Long,
@@ -34,15 +20,11 @@ data class ExtractedTrack(
val contentType: String, val contentType: String,
val fileMtime: Long, val fileMtime: Long,
val coverBytes: ByteArray?, val coverBytes: ByteArray?,
val lyrics: String?, )
) {
val searchText: String =
IdGenerator.normalize("$title $displayArtist $albumTitle ${albumArtists.joinToString(" ") { it.name }}")
}
/** /**
* Reads MP3 metadata with jaudiotagger and applies the documented fallbacks: * Reads the tags yt-dlp embeds into downloaded MP3s. Files are always MP3s
* album artist -> artist -> folder name, title -> file name. * inside the media store; anything else is ignored.
*/ */
class Mp3TagExtractor { class Mp3TagExtractor {
@@ -55,7 +37,7 @@ class Mp3TagExtractor {
).forEach { Logger.getLogger(it).level = Level.WARNING } ).forEach { Logger.getLogger(it).level = Level.WARNING }
} }
fun extract(folderId: Int, root: Path, file: Path): ExtractedTrack? { fun extract(file: Path): ExtractedSong? {
if (!file.fileName.toString().endsWith(".mp3", ignoreCase = true)) return null if (!file.fileName.toString().endsWith(".mp3", ignoreCase = true)) return null
val attributes = Files.readAttributes(file, BasicFileAttributes::class.java) val attributes = Files.readAttributes(file, BasicFileAttributes::class.java)
@@ -67,54 +49,20 @@ class Mp3TagExtractor {
val tag = audioFile.tag val tag = audioFile.tag
val header = audioFile.audioHeader val header = audioFile.audioHeader
val relativePath = root.relativize(file).joinToString("/") { it.toString() }
val fallbackArtist = file.parent val fallbackTitle = file.fileName.toString().removeSuffix(".mp3").removeSuffix(".MP3")
?.takeIf { it != root }
?.parent
?.takeIf { it != root }
?.fileName
?.toString()
?.takeIf { it.isNotBlank() }
?: "Unknown Artist"
val fallbackAlbum = file.parent val title = tag.values(FieldKey.TITLE).firstOrNull() ?: fallbackTitle
?.takeIf { it != root } val artists = (tag.values(FieldKey.ARTIST) + tag.values(FieldKey.ARTISTS)).distinct()
?.fileName val albumTitle = tag.values(FieldKey.ALBUM).firstOrNull()
?.toString() val albumArtist = (tag.values(FieldKey.ALBUM_ARTIST) + tag.values(FieldKey.ALBUM_ARTISTS))
?.takeIf { it.isNotBlank() }
?: "Unknown Album"
val tagArtists = (tag.values(FieldKey.ARTIST) + tag.values(FieldKey.ARTISTS)).distinct()
val selectedArtists = tagArtists.ifEmpty { listOf(fallbackArtist) }
val albumArtistNames =
(tag.values(FieldKey.ALBUM_ARTIST) + tag.values(FieldKey.ALBUM_ARTISTS))
.ifEmpty { selectedArtists }
val albumArtist = albumArtistNames.first()
val title = tag.values(FieldKey.TITLE).firstOrNull()
?: file.fileName.toString().removeSuffix(".mp3").removeSuffix(".MP3")
val albumTitle = tag.values(FieldKey.ALBUM).firstOrNull() ?: fallbackAlbum
val year = (tag.values(FieldKey.YEAR) + tag.values(FieldKey.ALBUM_YEAR))
.firstNotNullOfOrNull { YEAR_PATTERN.find(it)?.value?.toIntOrNull() }
val trackNumber = tag.values(FieldKey.TRACK)
.firstOrNull() .firstOrNull()
?.substringBefore('/')
?.trim()
?.toIntOrNull()
val discNumber = tag.values(FieldKey.DISC_NO) val artist = when {
.firstOrNull() artists.isNotEmpty() -> artists.joinToString(", ")
?.substringBefore('/') albumArtist != null -> albumArtist
?.trim() else -> "Unknown artist"
?.toIntOrNull() }
?: 1
val genres = tag.values(FieldKey.GENRE)
val duration = header.trackLength.takeIf { it > 0 } val duration = header.trackLength.takeIf { it > 0 }
?: header.preciseTrackLength.toInt().coerceAtLeast(0) ?: header.preciseTrackLength.toInt().coerceAtLeast(0)
@@ -126,26 +74,10 @@ class Mp3TagExtractor {
.getOrNull() .getOrNull()
?.takeIf { it.isNotEmpty() } ?.takeIf { it.isNotEmpty() }
val lyrics = tag.values(FieldKey.LYRICS).firstOrNull() ?: readSidecarLyrics(file) return ExtractedSong(
val albumArtistRefs = albumArtistNames.map { ArtistRef(IdGenerator.artistId(it), it) }
val artistRefs = selectedArtists.map { ArtistRef(IdGenerator.artistId(it), it) }
return ExtractedTrack(
songId = IdGenerator.songId(folderId, relativePath),
albumId = IdGenerator.albumId(albumArtist, albumTitle),
albumArtistId = IdGenerator.artistId(albumArtist),
albumArtists = albumArtistRefs,
artists = artistRefs,
title = title, title = title,
displayArtist = selectedArtists.joinToString(", "), artist = artist,
albumTitle = albumTitle, album = albumTitle,
folderId = folderId,
relativePath = relativePath,
trackNumber = trackNumber,
discNumber = discNumber,
year = year,
genres = genres,
durationSeconds = duration, durationSeconds = duration,
bitRate = bitRate, bitRate = bitRate,
size = attributes.size(), size = attributes.size(),
@@ -153,7 +85,6 @@ class Mp3TagExtractor {
contentType = "audio/mpeg", contentType = "audio/mpeg",
fileMtime = attributes.lastModifiedTime().toMillis(), fileMtime = attributes.lastModifiedTime().toMillis(),
coverBytes = coverBytes, coverBytes = coverBytes,
lyrics = lyrics,
) )
} }
@@ -164,15 +95,4 @@ class Mp3TagExtractor {
.map { it.trim() } .map { it.trim() }
.filter { it.isNotEmpty() } .filter { it.isNotEmpty() }
.distinct() .distinct()
private fun readSidecarLyrics(file: Path): String? {
val lrc = file.resolveSibling(file.fileName.toString().substringBeforeLast('.') + ".lrc")
return runCatching {
if (Files.isRegularFile(lrc)) Files.readString(lrc) else null
}.getOrNull()
}
private companion object {
val YEAR_PATTERN = Regex("\\d{4}")
}
} }
@@ -0,0 +1,53 @@
package win.taptap.musicmanager.scanner
import java.nio.file.Files
import java.nio.file.Path
import win.taptap.musicmanager.db.SongRecord
import win.taptap.musicmanager.db.SongRepository
import win.taptap.musicmanager.db.SongUpsert
import win.taptap.musicmanager.media.MediaStore
/**
* Single-file indexing shared by the scanner and the downloader. The
* downloader calls this immediately after moving a fresh MP3 into the media
* root, so it does not wait for a full scan.
*/
class SongIndexer(
private val mediaStore: MediaStore,
private val coverArtService: CoverArtService,
private val tagExtractor: Mp3TagExtractor = Mp3TagExtractor(),
) {
fun index(videoId: String, file: Path, thumbnail: String? = null): SongRecord? {
val normalized = file.normalize()
if (!normalized.startsWith(mediaStore.root)) return null
if (!Files.isRegularFile(normalized)) return null
val extracted = tagExtractor.extract(normalized) ?: return null
val relativePath = mediaStore.root.relativize(normalized).joinToString("/") { it.toString() }
val song = SongRepository.upsert(
SongUpsert(
videoId = videoId,
title = extracted.title,
artist = extracted.artist,
album = extracted.album,
duration = extracted.durationSeconds,
bitRate = extracted.bitRate,
size = extracted.size,
suffix = extracted.suffix,
contentType = extracted.contentType,
relativePath = relativePath,
fileMtime = extracted.fileMtime,
thumbnail = thumbnail,
),
)
val coverBytes = extracted.coverBytes
?: coverArtService.findSidecarCover(normalized)?.let(coverArtService::readCover)
if (coverBytes != null) {
coverArtService.saveCover(song.id, coverBytes)
}
return song
}
}
@@ -19,22 +19,12 @@ object IdGenerator {
.replace(WHITESPACE, " ") .replace(WHITESPACE, " ")
.lowercase(Locale.ROOT) .lowercase(Locale.ROOT)
fun artistId(albumArtist: String): String = "ar-" + hash(normalize(albumArtist)) /** Stable across restarts and independent of the downloaded file's title. */
fun songId(videoId: String): String = "sg-" + hash("yt:${normalize(videoId)}")
fun albumId(albumArtist: String, albumTitle: String): String =
"al-" + hash("${normalize(albumArtist)}|${normalize(albumTitle)}")
fun songId(folderId: Int, relativePath: String): String =
"sg-" + hash("$folderId:${normalizePath(relativePath)}")
fun moodId(owner: String, name: String, createdAt: String): String = fun moodId(owner: String, name: String, createdAt: String): String =
"mo-" + hash("${normalize(owner)}|${normalize(name)}|$createdAt") "mo-" + hash("${normalize(owner)}|${normalize(name)}|$createdAt")
private fun normalizePath(path: String): String = path
.replace('\\', '/')
.removePrefix("./")
.trimStart('/')
private fun hash(value: String): String { private fun hash(value: String): String {
val digest = MessageDigest.getInstance("SHA-256").digest(value.toByteArray(Charsets.UTF_8)) val digest = MessageDigest.getInstance("SHA-256").digest(value.toByteArray(Charsets.UTF_8))
val hex = digest.joinToString("") { "%02x".format(it) } val hex = digest.joinToString("") { "%02x".format(it) }
@@ -0,0 +1,162 @@
-- Mood-first redesign: destructive migration.
--
-- The mounted-library model is gone. Songs enter only through downloads, so
-- every content table from the old design is dropped. Users and device tokens
-- are preserved; the media root starts empty and is rebuilt by the downloader.
--
-- Drop order matters because SQLite enforces foreign keys on DROP TABLE.
DROP TABLE IF EXISTS playback_session_items;
DROP TABLE IF EXISTS playback_sessions;
DROP TABLE IF EXISTS song_affinity;
DROP TABLE IF EXISTS download_jobs;
DROP TABLE IF EXISTS mood_recommendations;
DROP TABLE IF EXISTS mood_seeds;
DROP TABLE IF EXISTS moods;
DROP TABLE IF EXISTS song_genres;
DROP TABLE IF EXISTS song_artists;
DROP TABLE IF EXISTS album_genres;
DROP TABLE IF EXISTS artist_genres;
DROP TABLE IF EXISTS album_artists;
DROP TABLE IF EXISTS user_song_state;
DROP TABLE IF EXISTS user_stars;
DROP TABLE IF EXISTS bookmarks;
DROP TABLE IF EXISTS library_tombstones;
DROP TABLE IF EXISTS longform_progress;
DROP TABLE IF EXISTS songs;
DROP TABLE IF EXISTS albums;
DROP TABLE IF EXISTS artists;
DROP TABLE IF EXISTS music_folders;
-- One row per YouTube video. The file lives at <media>/<video_id>/<title>.mp3,
-- so the video id is both the dedupe key and the on-disk folder name.
CREATE TABLE songs (
id TEXT PRIMARY KEY,
video_id TEXT NOT NULL UNIQUE,
title TEXT NOT NULL,
artist TEXT NOT NULL,
album TEXT,
duration INTEGER NOT NULL DEFAULT 0,
bit_rate INTEGER NOT NULL DEFAULT 0,
size INTEGER NOT NULL DEFAULT 0,
suffix TEXT NOT NULL DEFAULT 'mp3',
content_type TEXT NOT NULL DEFAULT 'audio/mpeg',
relative_path TEXT NOT NULL,
file_mtime INTEGER NOT NULL DEFAULT 0,
thumbnail TEXT,
created_at TEXT NOT NULL,
updated_at TEXT NOT NULL
);
CREATE INDEX idx_songs_updated ON songs (updated_at);
CREATE TABLE moods (
id TEXT PRIMARY KEY,
owner TEXT NOT NULL REFERENCES users(name) ON DELETE CASCADE,
name TEXT NOT NULL,
description TEXT,
query TEXT,
genres TEXT NOT NULL DEFAULT '[]',
collection_percent INTEGER NOT NULL DEFAULT 50
CHECK (collection_percent BETWEEN 0 AND 100),
recommendation_count INTEGER NOT NULL DEFAULT 10
CHECK (recommendation_count BETWEEN 0 AND 30),
refresh_hours INTEGER NOT NULL DEFAULT 24,
last_discovery_at TEXT,
created_at TEXT NOT NULL,
changed_at TEXT NOT NULL,
last_error TEXT
);
CREATE INDEX idx_moods_owner ON moods (owner);
CREATE TABLE mood_songs (
mood_id TEXT NOT NULL REFERENCES moods(id) ON DELETE CASCADE,
song_id TEXT NOT NULL REFERENCES songs(id) ON DELETE CASCADE,
origin TEXT NOT NULL DEFAULT 'search'
CHECK (origin IN ('search', 'recommendation', 'url')),
added_at TEXT NOT NULL,
PRIMARY KEY (mood_id, song_id)
);
CREATE INDEX idx_mood_songs_song ON mood_songs (song_id);
-- new hidden candidate pool used to fill active slots
-- downloading an active slot with a queued/running download job
-- ready an active slot whose MP3 is indexed and playable
-- failed an active slot whose download exhausted its retries
-- added the user promoted it to mood_songs
-- rejected the user does not want it in this mood again
CREATE TABLE mood_recommendations (
mood_id TEXT NOT NULL REFERENCES moods(id) ON DELETE CASCADE,
video_id TEXT NOT NULL,
title TEXT NOT NULL,
artist TEXT NOT NULL,
album TEXT,
duration INTEGER,
thumbnail TEXT,
state TEXT NOT NULL DEFAULT 'new'
CHECK (state IN ('new', 'downloading', 'ready', 'failed', 'added', 'rejected')),
song_id TEXT REFERENCES songs(id) ON DELETE SET NULL,
attempts INTEGER NOT NULL DEFAULT 0,
error TEXT,
rank INTEGER NOT NULL DEFAULT 0,
first_seen TEXT NOT NULL,
last_seen TEXT NOT NULL,
PRIMARY KEY (mood_id, video_id)
);
CREATE INDEX idx_mood_recommendations_state ON mood_recommendations (mood_id, state);
CREATE INDEX idx_mood_recommendations_song ON mood_recommendations (song_id);
CREATE TABLE download_jobs (
id TEXT PRIMARY KEY,
video_id TEXT NOT NULL,
mood_id TEXT NOT NULL REFERENCES moods(id) ON DELETE CASCADE,
kind TEXT NOT NULL CHECK (kind IN ('song', 'recommendation')),
state TEXT NOT NULL DEFAULT 'queued'
CHECK (state IN ('queued', 'downloading', 'done', 'failed')),
priority INTEGER NOT NULL DEFAULT 0,
attempts INTEGER NOT NULL DEFAULT 0,
error TEXT,
song_id TEXT REFERENCES songs(id) ON DELETE SET NULL,
created_at TEXT NOT NULL,
updated_at TEXT NOT NULL,
UNIQUE (mood_id, video_id, kind)
);
CREATE INDEX idx_download_jobs_state ON download_jobs (state, priority, created_at);
CREATE TABLE playback_sessions (
id TEXT PRIMARY KEY,
user_name TEXT NOT NULL REFERENCES users(name) ON DELETE CASCADE,
mood_id TEXT NOT NULL REFERENCES moods(id) ON DELETE CASCADE,
next_position INTEGER NOT NULL DEFAULT 0,
created_at TEXT NOT NULL,
last_seen_at TEXT NOT NULL,
closed_at TEXT
);
CREATE INDEX idx_playback_sessions_user ON playback_sessions (user_name, last_seen_at);
CREATE TABLE playback_session_items (
session_id TEXT NOT NULL REFERENCES playback_sessions(id) ON DELETE CASCADE,
position INTEGER NOT NULL,
song_id TEXT NOT NULL REFERENCES songs(id) ON DELETE CASCADE,
state TEXT NOT NULL DEFAULT 'queued'
CHECK (state IN ('queued', 'started', 'completed', 'skipped')),
queued_at TEXT NOT NULL,
updated_at TEXT NOT NULL,
PRIMARY KEY (session_id, position)
);
CREATE INDEX idx_session_items_song ON playback_session_items (song_id);
CREATE TABLE song_affinity (
user_name TEXT NOT NULL REFERENCES users(name) ON DELETE CASCADE,
song_id TEXT NOT NULL REFERENCES songs(id) ON DELETE CASCADE,
completions INTEGER NOT NULL DEFAULT 0,
skips INTEGER NOT NULL DEFAULT 0,
last_event_at TEXT NOT NULL,
PRIMARY KEY (user_name, song_id)
);
File diff suppressed because it is too large. Load diff
@@ -13,9 +13,11 @@ class AppConfigTest {
assertEquals(8080, config.port) assertEquals(8080, config.port)
assertEquals(Path.of("./data"), config.dataDir) assertEquals(Path.of("./data"), config.dataDir)
assertEquals(Path.of("./config/library.yaml"), config.libraryConfigPath)
assertEquals(Path.of("./config/users.yaml"), config.usersFile) assertEquals(Path.of("./config/users.yaml"), config.usersFile)
assertEquals("ffmpeg", config.ffmpegPath) assertEquals("ffmpeg", config.ffmpegPath)
assertEquals("yt-dlp", config.ytDlpPath)
assertEquals(Path.of("./data/media"), config.mediaDir)
assertEquals(Path.of("./data/downloads"), config.downloadDir)
assertEquals(Path.of("./data/musicmanager.db"), config.databasePath) assertEquals(Path.of("./data/musicmanager.db"), config.databasePath)
} }
@@ -25,17 +27,21 @@ class AppConfigTest {
mapOf( mapOf(
"PORT" to "9090", "PORT" to "9090",
"DATA_DIR" to "/var/lib/musicmanager", "DATA_DIR" to "/var/lib/musicmanager",
"LIBRARY_CONFIG" to "/etc/musicmanager/library.yaml",
"USERS_FILE" to "/run/secrets/users.yaml", "USERS_FILE" to "/run/secrets/users.yaml",
"FFMPEG_PATH" to "/usr/bin/ffmpeg", "FFMPEG_PATH" to "/usr/bin/ffmpeg",
"YTDLP_PATH" to "/usr/local/bin/yt-dlp",
"MEDIA_DIR" to "/mnt/media",
"DOWNLOAD_DIR" to "/var/tmp/downloads",
), ),
) )
assertEquals(9090, config.port) assertEquals(9090, config.port)
assertEquals(Path.of("/var/lib/musicmanager"), config.dataDir) assertEquals(Path.of("/var/lib/musicmanager"), config.dataDir)
assertEquals(Path.of("/etc/musicmanager/library.yaml"), config.libraryConfigPath)
assertEquals(Path.of("/run/secrets/users.yaml"), config.usersFile) assertEquals(Path.of("/run/secrets/users.yaml"), config.usersFile)
assertEquals("/usr/bin/ffmpeg", config.ffmpegPath) assertEquals("/usr/bin/ffmpeg", config.ffmpegPath)
assertEquals("/usr/local/bin/yt-dlp", config.ytDlpPath)
assertEquals(Path.of("/mnt/media"), config.mediaDir)
assertEquals(Path.of("/var/tmp/downloads"), config.downloadDir)
} }
@Test @Test
@@ -1,127 +0,0 @@
package win.taptap.musicmanager.config
import java.nio.file.Files
import kotlin.io.path.createTempDirectory
import kotlin.test.Test
import kotlin.test.assertEquals
import kotlin.test.assertFailsWith
import kotlin.test.assertTrue
class LibraryConfigTest {
private val validYaml = """
library:
folders:
- id: 1
name: "Music"
type: music
path: "./media/music"
- id: 10
name: "Long-form"
type: longform
path: "./media/longform"
""".trimIndent()
@Test
fun `parses folders and resolves absolute paths`() {
val config = LibraryConfig.parse(validYaml)
assertEquals(2, config.folders.size)
assertEquals(setOf(10), config.longformFolderIds)
assertEquals(FolderType.MUSIC, config.folders[0].type)
assertEquals(FolderType.LONGFORM, config.folders[1].type)
assertTrue(config.folders[0].path.isAbsolute)
assertEquals(1, config.byId(1)?.id)
}
@Test
fun `rejects duplicate folder ids`() {
val yaml = """
library:
folders:
- id: 1
name: A
type: music
path: /music/a
- id: 1
name: B
type: music
path: /music/b
""".trimIndent()
assertFailsWith<IllegalArgumentException> { LibraryConfig.parse(yaml) }
}
@Test
fun `rejects duplicate folder paths`() {
val yaml = """
library:
folders:
- id: 1
name: A
type: music
path: /music/a
- id: 2
name: B
type: music
path: /music/a
""".trimIndent()
assertFailsWith<IllegalArgumentException> { LibraryConfig.parse(yaml) }
}
@Test
fun `rejects unknown folder types`() {
val yaml = """
library:
folders:
- id: 1
name: A
type: video
path: /music/a
""".trimIndent()
assertFailsWith<IllegalArgumentException> { LibraryConfig.parse(yaml) }
}
@Test
fun `validatePaths fails for a missing root`() {
val root = createTempDirectory("musicmanager-test")
try {
val yaml = """
library:
folders:
- id: 1
name: A
type: music
path: "${root.resolve("missing")}"
""".trimIndent()
assertFailsWith<IllegalArgumentException> {
LibraryConfig.parse(yaml).validatePaths()
}
} finally {
root.toFile().deleteRecursively()
}
}
@Test
fun `validatePaths accepts existing readable roots`() {
val root = createTempDirectory("musicmanager-test")
try {
val music = Files.createDirectories(root.resolve("music"))
val yaml = """
library:
folders:
- id: 1
name: A
type: music
path: "$music"
""".trimIndent()
LibraryConfig.parse(yaml).validatePaths()
} finally {
root.toFile().deleteRecursively()
}
}
}
@@ -8,26 +8,10 @@ import kotlin.test.assertTrue
class UsersConfigTest { class UsersConfigTest {
private val library = LibraryConfig.parse( @Test
fun `parses users and ignores old folder assignments`() {
val users = UsersConfig.parse(
""" """
library:
folders:
- id: 1
name: "Music A"
type: music
path: /music/a
- id: 2
name: "Music B"
type: music
path: /music/b
- id: 10
name: "Long-form"
type: longform
path: /longform
""".trimIndent(),
)
private val usersYaml = """
users: users:
- name: alice - name: alice
password: "secret-a" password: "secret-a"
@@ -35,21 +19,15 @@ class UsersConfigTest {
folders: [1] folders: [1]
- name: bob - name: bob
password: "secret-b" password: "secret-b"
folders: [1, 2] """.trimIndent(),
""".trimIndent() )
@Test
fun `parses users and shares long-form folders`() {
val users = UsersConfig.parse(usersYaml)
users.validateAgainst(library)
val alice = users.byName("alice")!! val alice = users.byName("alice")!!
val bob = users.byName("bob")!! val bob = users.byName("bob")!!
assertTrue(alice.admin) assertTrue(alice.admin)
assertFalse(bob.admin) assertFalse(bob.admin)
assertEquals(setOf(1, 10), alice.visibleFolderIds(library)) assertEquals(2, users.users.size)
assertEquals(setOf(1, 2, 10), bob.visibleFolderIds(library))
} }
@Test @Test
@@ -57,7 +35,6 @@ class UsersConfigTest {
val yaml = """ val yaml = """
users: users:
- name: alice - name: alice
folders: [1]
""".trimIndent() """.trimIndent()
assertFailsWith<IllegalArgumentException> { UsersConfig.parse(yaml) } assertFailsWith<IllegalArgumentException> { UsersConfig.parse(yaml) }
@@ -69,56 +46,27 @@ class UsersConfigTest {
users: users:
- name: alice - name: alice
password: a password: a
folders: [1]
- name: alice - name: alice
password: b password: b
folders: [2]
""".trimIndent() """.trimIndent()
assertFailsWith<IllegalArgumentException> { UsersConfig.parse(yaml) } assertFailsWith<IllegalArgumentException> { UsersConfig.parse(yaml) }
} }
@Test @Test
fun `rejects unknown folder references`() { fun `rejects an empty user list`() {
val users = UsersConfig.parse( assertFailsWith<IllegalArgumentException> { UsersConfig.parse("users: []") }
"""
users:
- name: alice
password: a
folders: [99]
""".trimIndent(),
)
assertFailsWith<IllegalArgumentException> { users.validateAgainst(library) }
}
@Test
fun `rejects a user with no visible folders`() {
val emptyLibrary = LibraryConfig.parse(
"""
library:
folders:
- id: 1
name: "Music"
type: music
path: /music
""".trimIndent(),
)
val users = UsersConfig.parse(
"""
users:
- name: alice
password: a
folders: []
""".trimIndent(),
)
assertFailsWith<IllegalArgumentException> { users.validateAgainst(emptyLibrary) }
} }
@Test @Test
fun `does not expose passwords in toString`() { fun `does not expose passwords in toString`() {
val alice = UsersConfig.parse(usersYaml).byName("alice")!! val alice = UsersConfig.parse(
"""
users:
- name: alice
password: "secret-a"
""".trimIndent(),
).byName("alice")!!
assertFalse(alice.toString().contains("secret-a")) assertFalse(alice.toString().contains("secret-a"))
assertTrue(alice.toString().contains("password=***")) assertTrue(alice.toString().contains("password=***"))
@@ -3,20 +3,18 @@ package win.taptap.musicmanager.db
import java.nio.file.Files import java.nio.file.Files
import java.nio.file.Path import java.nio.file.Path
import java.sql.DriverManager import java.sql.DriverManager
import org.flywaydb.core.Flyway
import org.flywaydb.core.api.MigrationVersion
import java.time.Instant import java.time.Instant
import kotlin.test.Test import kotlin.test.Test
import kotlin.test.assertEquals import kotlin.test.assertEquals
import kotlin.test.assertFalse
import kotlin.test.assertTrue import kotlin.test.assertTrue
import org.flywaydb.core.Flyway
import org.flywaydb.core.api.MigrationVersion
import org.jetbrains.exposed.sql.insert import org.jetbrains.exposed.sql.insert
import org.jetbrains.exposed.sql.selectAll import org.jetbrains.exposed.sql.selectAll
import org.jetbrains.exposed.sql.transactions.transaction import org.jetbrains.exposed.sql.transactions.transaction
import org.junit.jupiter.api.io.TempDir import org.junit.jupiter.api.io.TempDir
import win.taptap.musicmanager.config.AppConfig import win.taptap.musicmanager.config.AppConfig
import win.taptap.musicmanager.config.FolderType
import win.taptap.musicmanager.config.LibraryConfig
import win.taptap.musicmanager.config.MusicFolder
import win.taptap.musicmanager.config.UserAccount import win.taptap.musicmanager.config.UserAccount
import win.taptap.musicmanager.config.UsersConfig import win.taptap.musicmanager.config.UsersConfig
@@ -32,23 +30,24 @@ class DatabaseFactoryTest {
assertEquals("wal", pragma(db, "journal_mode")) assertEquals("wal", pragma(db, "journal_mode"))
assertEquals("1", pragma(db, "foreign_keys")) assertEquals("1", pragma(db, "foreign_keys"))
val migrations = db.dataSource.connection.use { connection -> val migrations = queryInt(db, "SELECT COUNT(*) FROM flyway_schema_history")
connection.createStatement().use { statement -> assertTrue(migrations >= 8, "Expected at least the V1..V8 migrations")
statement.executeQuery("SELECT COUNT(*) FROM flyway_schema_history").use { result ->
result.next() // The new schema exists and the mounted-library tables do not.
result.getInt(1) assertTrue(tableExists(db, "songs"))
} assertTrue(tableExists(db, "mood_songs"))
} assertTrue(tableExists(db, "mood_recommendations"))
} assertTrue(tableExists(db, "download_jobs"))
assertTrue(migrations >= 2, "Expected at least the V1 and V2 migrations") assertFalse(tableExists(db, "music_folders"))
assertFalse(tableExists(db, "artists"))
assertFalse(tableExists(db, "longform_progress"))
} finally { } finally {
db.close() db.close()
} }
} }
@Test @Test
fun `upgrades legacy liked and downloaded recommendations`() { fun `V8 wipes content but keeps users and device tokens`() {
val config = appConfig() val config = appConfig()
Files.createDirectories(config.dataDir) Files.createDirectories(config.dataDir)
val url = "jdbc:sqlite:${config.databasePath.toAbsolutePath()}" val url = "jdbc:sqlite:${config.databasePath.toAbsolutePath()}"
@@ -56,7 +55,7 @@ class DatabaseFactoryTest {
Flyway.configure() Flyway.configure()
.dataSource(url, null, null) .dataSource(url, null, null)
.locations("classpath:db/migration") .locations("classpath:db/migration")
.target(MigrationVersion.fromVersion("4")) .target(MigrationVersion.fromVersion("7"))
.load() .load()
.migrate() .migrate()
@@ -66,62 +65,84 @@ class DatabaseFactoryTest {
"INSERT INTO users (name, is_admin, updated_at) VALUES ('alice', 1, 'now')", "INSERT INTO users (name, is_admin, updated_at) VALUES ('alice', 1, 'now')",
) )
statement.executeUpdate( statement.executeUpdate(
"INSERT INTO playlists (id, owner, name, created_at, changed_at) " + "INSERT INTO api_tokens " +
"VALUES ('pl-1', 'alice', 'Mood', 'now', 'now')", "(id, user_name, token_hash, device_name, platform, created_at, last_used_at, expires_at) " +
"VALUES ('tk-1', 'alice', 'hash', 'phone', 'android', 'now', 'now', 'later')",
) )
statement.executeUpdate( statement.executeUpdate(
"INSERT INTO moods (playlist_id) VALUES ('pl-1')", "INSERT INTO music_folders (id, name, type, root_path) VALUES (1, 'Music', 'music', '/m')",
) )
statement.executeUpdate( statement.executeUpdate(
"INSERT INTO mood_recommendations " + "INSERT INTO artists (id, name, sort_name, search_text) VALUES ('ar-1', 'Artist', 'artist', 'artist')",
"(playlist_id, video_id, title, artist, state, first_seen, last_seen) " + )
"VALUES ('pl-1', 'v-1', 'Title', 'Artist', 'liked', 'now', 'now')", statement.executeUpdate(
"INSERT INTO albums (id, name, created_at, search_text) VALUES ('al-1', 'Album', 'now', 'album')",
)
statement.executeUpdate(
"INSERT INTO songs " +
"(id, album_id, folder_id, title, display_artist, album_name, duration, bit_rate, " +
"size, suffix, content_type, relative_path, file_mtime, created_at, updated_at, search_text) " +
"VALUES ('sg-1', 'al-1', 1, 'Song', 'Artist', 'Album', 120, 128, 10, 'mp3', " +
"'audio/mpeg', 'a/b.mp3', 0, 'now', 'now', 'song')",
)
statement.executeUpdate(
"INSERT INTO moods " +
"(id, owner, name, genres, target_size, refresh_hours, yt_enabled, created_at, changed_at, seed_percent) " +
"VALUES ('mo-1', 'alice', 'Focus', '[]', 30, 24, 1, 'now', 'now', 50)",
)
statement.executeUpdate(
"INSERT INTO mood_seeds (mood_id, song_id, origin, added_at) VALUES ('mo-1', 'sg-1', 'manual', 'now')",
)
statement.executeUpdate(
"INSERT INTO playback_sessions (id, user_name, mood_id, created_at, last_seen_at) " +
"VALUES ('ps-1', 'alice', 'mo-1', 'now', 'now')",
)
statement.executeUpdate(
"INSERT INTO playback_session_items " +
"(session_id, position, song_id, state, queued_at, updated_at) " +
"VALUES ('ps-1', 0, 'sg-1', 'queued', 'now', 'now')",
) )
} }
} }
val db = DatabaseFactory.connect(config) val db = DatabaseFactory.connect(config)
try { try {
val state = db.dataSource.connection.use { connection -> assertEquals(
connection.createStatement().use { statement -> 1,
statement.executeQuery( queryInt(db, "SELECT COUNT(*) FROM users WHERE name = 'alice'"),
"SELECT state FROM mood_recommendations WHERE video_id = 'v-1'", "Users must survive the wipe",
).use { result -> )
result.next() assertEquals(
result.getString(1) 1,
} queryInt(db, "SELECT COUNT(*) FROM api_tokens WHERE id = 'tk-1'"),
} "Device tokens must survive the wipe",
} )
assertEquals("new", state) assertEquals(0, queryInt(db, "SELECT COUNT(*) FROM songs"))
assertEquals(0, queryInt(db, "SELECT COUNT(*) FROM moods"))
assertEquals(0, queryInt(db, "SELECT COUNT(*) FROM playback_sessions"))
assertTrue(tableExists(db, "mood_songs"))
assertFalse(tableExists(db, "music_folders"))
} finally { } finally {
db.close() db.close()
} }
} }
@Test @Test
fun `syncs folders and users idempotently`() { fun `syncs users idempotently`() {
val db = DatabaseFactory.connect(appConfig()) val db = DatabaseFactory.connect(appConfig())
try { try {
val library = library()
val users = UsersConfig( val users = UsersConfig(
listOf( listOf(
UserAccount("alice", "pw", admin = true, folders = setOf(1)), UserAccount("alice", "pw", admin = true),
UserAccount("bob", "pw", admin = false, folders = setOf(1)), UserAccount("bob", "pw", admin = false),
), ),
) )
ConfigRepository.sync(library, users) ConfigRepository.sync(users)
ConfigRepository.sync(library, users) ConfigRepository.sync(users)
transaction(db.database) { transaction(db.database) {
assertEquals(2L, MusicFoldersTable.selectAll().count())
assertEquals(2L, UsersTable.selectAll().count()) assertEquals(2L, UsersTable.selectAll().count())
assertEquals(
"longform",
MusicFoldersTable.selectAll()
.where { MusicFoldersTable.id eq 10 }
.single()[MusicFoldersTable.type],
)
} }
} finally { } finally {
db.close() db.close()
@@ -129,65 +150,69 @@ class DatabaseFactoryTest {
} }
@Test @Test
fun `removing a user cascades their per-user state`() { fun `removing a user cascades their moods and playback state`() {
val db = DatabaseFactory.connect(appConfig()) val db = DatabaseFactory.connect(appConfig())
try { try {
val library = library() ConfigRepository.sync(
val aliceAndBob = UsersConfig( UsersConfig(
listOf( listOf(
UserAccount("alice", "pw", admin = true, folders = setOf(1)), UserAccount("alice", "pw", admin = true),
UserAccount("bob", "pw", admin = false, folders = setOf(1)), UserAccount("bob", "pw", admin = false),
),
), ),
) )
ConfigRepository.sync(library, aliceAndBob) val now = Instant.now().toString()
insertSong(db, folderId = 1) val moodId = "mo-1"
transaction(db.database) { transaction(db.database) {
UserSongStateTable.insert { SongsTable.insert {
it[userName] = "alice" it[id] = "sg-1"
it[videoId] = "video-1"
it[title] = "Song"
it[artist] = "Artist"
it[relativePath] = "video-1/song.mp3"
it[createdAt] = now
it[updatedAt] = now
}
MoodsTable.insert {
it[id] = moodId
it[owner] = "alice"
it[name] = "Focus"
it[createdAt] = now
it[changedAt] = now
}
MoodSongsTable.insert {
it[this.moodId] = moodId
it[songId] = "sg-1" it[songId] = "sg-1"
it[playCount] = 3 it[addedAt] = now
it[starredAt] = Instant.now().toString() }
PlaybackSessionsTable.insert {
it[id] = "ps-1"
it[userName] = "alice"
it[this.moodId] = moodId
it[createdAt] = now
it[lastSeenAt] = now
}
SessionItemsTable.insert {
it[sessionId] = "ps-1"
it[position] = 0
it[songId] = "sg-1"
it[queuedAt] = now
it[updatedAt] = now
} }
assertEquals(1L, UserSongStateTable.selectAll().count())
} }
ConfigRepository.sync( ConfigRepository.sync(
library, UsersConfig(listOf(UserAccount("bob", "pw", admin = false))),
UsersConfig(listOf(UserAccount("bob", "pw", admin = false, folders = setOf(1)))),
) )
transaction(db.database) { transaction(db.database) {
assertEquals(0L, UserSongStateTable.selectAll().count())
assertEquals(1L, UsersTable.selectAll().count()) assertEquals(1L, UsersTable.selectAll().count())
} assertEquals(0L, MoodsTable.selectAll().count())
} finally { assertEquals(0L, MoodSongsTable.selectAll().count())
db.close() assertEquals(0L, PlaybackSessionsTable.selectAll().count())
} // Songs are shared and have no owner, so they survive.
}
@Test
fun `removing a folder cascades its songs`() {
val db = DatabaseFactory.connect(appConfig())
try {
val library = library()
val users = UsersConfig(listOf(UserAccount("alice", "pw", admin = true, folders = setOf(1))))
ConfigRepository.sync(library, users)
insertSong(db, folderId = 1)
transaction(db.database) {
assertEquals(1L, SongsTable.selectAll().count()) assertEquals(1L, SongsTable.selectAll().count())
} }
ConfigRepository.sync(
LibraryConfig(library.folders.filter { it.id == 10 }),
UsersConfig(listOf(UserAccount("alice", "pw", admin = true, folders = emptySet()))),
)
transaction(db.database) {
assertEquals(0L, SongsTable.selectAll().count())
assertEquals(1L, MusicFoldersTable.selectAll().count())
}
} finally { } finally {
db.close() db.close()
} }
@@ -196,57 +221,10 @@ class DatabaseFactoryTest {
private fun appConfig(): AppConfig = AppConfig( private fun appConfig(): AppConfig = AppConfig(
port = 8080, port = 8080,
dataDir = tempDir.resolve("data"), dataDir = tempDir.resolve("data"),
libraryConfigPath = tempDir.resolve("library.yaml"),
usersFile = tempDir.resolve("users.yaml"), usersFile = tempDir.resolve("users.yaml"),
ffmpegPath = "ffmpeg", ffmpegPath = "ffmpeg",
) )
private fun library(): LibraryConfig {
val music = Files.createDirectories(tempDir.resolve("music"))
val longform = Files.createDirectories(tempDir.resolve("longform"))
return LibraryConfig(
listOf(
MusicFolder(1, "Music", FolderType.MUSIC, music),
MusicFolder(10, "Long-form", FolderType.LONGFORM, longform),
),
)
}
private fun insertSong(db: DatabaseHandle, folderId: Int) {
val now = Instant.now().toString()
transaction(db.database) {
ArtistsTable.insert {
it[id] = "ar-1"
it[name] = "Artist"
it[sortName] = "artist"
it[searchText] = "artist"
}
AlbumsTable.insert {
it[id] = "al-1"
it[name] = "Album"
it[createdAt] = now
it[searchText] = "album"
}
SongsTable.insert {
it[id] = "sg-1"
it[albumId] = "al-1"
it[this.folderId] = folderId
it[title] = "Song"
it[displayArtist] = "Artist"
it[albumName] = "Album"
it[duration] = 120
it[bitRate] = 128
it[size] = 1024
it[suffix] = "mp3"
it[contentType] = "audio/mpeg"
it[relativePath] = "artist/album/song.mp3"
it[fileMtime] = 0
it[createdAt] = now
it[searchText] = "song"
}
}
}
private fun pragma(db: DatabaseHandle, name: String): String = private fun pragma(db: DatabaseHandle, name: String): String =
db.dataSource.connection.use { connection -> db.dataSource.connection.use { connection ->
connection.createStatement().use { statement -> connection.createStatement().use { statement ->
@@ -256,4 +234,20 @@ class DatabaseFactoryTest {
} }
} }
} }
private fun queryInt(db: DatabaseHandle, sql: String): Int =
db.dataSource.connection.use { connection ->
connection.createStatement().use { statement ->
statement.executeQuery(sql).use { result ->
assertTrue(result.next(), "$sql returned no row")
result.getInt(1)
}
}
}
private fun tableExists(db: DatabaseHandle, name: String): Boolean =
queryInt(
db,
"SELECT COUNT(*) FROM sqlite_master WHERE type = 'table' AND name = '$name'",
) > 0
} }
@@ -0,0 +1,283 @@
package win.taptap.musicmanager.db
import java.nio.file.Files
import java.nio.file.Path
import java.time.Instant
import kotlin.test.Test
import kotlin.test.assertEquals
import kotlin.test.assertFalse
import kotlin.test.assertNotNull
import kotlin.test.assertNull
import kotlin.test.assertTrue
import org.junit.jupiter.api.io.TempDir
import win.taptap.musicmanager.config.AppConfig
import win.taptap.musicmanager.config.UserAccount
import win.taptap.musicmanager.config.UsersConfig
import win.taptap.musicmanager.media.MediaGcService
import win.taptap.musicmanager.media.MediaStore
import win.taptap.musicmanager.scanner.CoverArtService
class MoodRepositoryTest {
@TempDir
lateinit var tempDir: Path
@Test
fun `recommendation slots are promoted completed added and refilled`() {
withDatabase { mediaStore, _ ->
val mood = createMood(recommendationCount = 3)
storeCandidates(mood.id, 5)
val promoted = MoodRepository.promoteCandidates(mood.id, 3)
assertEquals(3, promoted.size)
assertEquals(3, MoodRepository.activeCount(mood.id))
assertEquals(
2,
MoodRepository.allRecommendations(mood.id).count { it.state == "new" },
)
val first = promoted.first()
val song = insertSong(mediaStore, "vid-${first.videoId}", "Track ${first.videoId}")
assertEquals(
RecommendationCompletion.READY,
MoodRepository.completeDownload(mood.id, first.videoId, song.id),
)
assertEquals("ready", MoodRepository.recommendation(mood.id, first.videoId)?.state)
// Add turns the slot into a permanent mood song and frees it.
assertEquals(
RecommendationCompletion.ALREADY_ADDED,
MoodRepository.markAdded(mood.id, first.videoId),
)
assertTrue(MoodRepository.songIds(mood.id).contains(song.id))
assertEquals("added", MoodRepository.recommendation(mood.id, first.videoId)?.state)
assertEquals(2, MoodRepository.activeCount(mood.id))
val refill = MoodRepository.promoteCandidates(mood.id, 1)
assertEquals(1, refill.size)
assertEquals(3, MoodRepository.activeCount(mood.id))
// Reject is permanent and frees the slot again.
MoodRepository.markRejected(mood.id, refill.first().videoId)
assertEquals(2, MoodRepository.activeCount(mood.id))
assertEquals(
"rejected",
MoodRepository.recommendation(mood.id, refill.first().videoId)?.state,
)
assertTrue(MoodRepository.decidedVideoIds(mood.id).contains(refill.first().videoId))
assertEquals(
1,
MoodRepository.allRecommendations(mood.id).count { it.state == "new" },
)
}
}
@Test
fun `decisions survive a discovery refresh`() {
withDatabase { _, _ ->
val mood = createMood(recommendationCount = 3)
storeCandidates(mood.id, 3)
val first = MoodRepository.promoteCandidates(mood.id, 1).single()
MoodRepository.markRejected(mood.id, first.videoId)
// The same video must not come back as a hidden candidate.
storeCandidates(mood.id, 3)
assertEquals(
"rejected",
MoodRepository.recommendation(mood.id, first.videoId)?.state,
)
}
}
@Test
fun `download completion after a rejection does not attach the song`() {
withDatabase { mediaStore, _ ->
val mood = createMood(recommendationCount = 2)
storeCandidates(mood.id, 2)
val recommendation = MoodRepository.promoteCandidates(mood.id, 1).single()
val song = insertSong(mediaStore, "vid-1", "One")
MoodRepository.markRejected(mood.id, recommendation.videoId)
assertEquals(
RecommendationCompletion.REJECTED,
MoodRepository.completeDownload(mood.id, recommendation.videoId, song.id),
)
assertTrue(MoodRepository.songIds(mood.id).isEmpty())
}
}
@Test
fun `adding while downloading attaches exactly one mood song`() {
withDatabase { mediaStore, _ ->
val mood = createMood(recommendationCount = 2)
storeCandidates(mood.id, 2)
val recommendation = MoodRepository.promoteCandidates(mood.id, 1).single()
val song = insertSong(mediaStore, "vid-1", "One")
// User presses Add while the download is still running.
MoodRepository.markAdded(mood.id, recommendation.videoId)
assertTrue(MoodRepository.songIds(mood.id).isEmpty())
assertEquals(
RecommendationCompletion.ALREADY_ADDED,
MoodRepository.completeDownload(mood.id, recommendation.videoId, song.id),
)
assertEquals(listOf(song.id), MoodRepository.songIds(mood.id))
assertFalse(MoodRepository.addSong(mood.id, song.id, "recommendation"))
}
}
@Test
fun `mood counts reflect songs and active recommendation states`() {
withDatabase { mediaStore, _ ->
val mood = createMood(recommendationCount = 3)
val song = insertSong(mediaStore, "vid-1", "One")
MoodRepository.addSong(mood.id, song.id, "search")
storeCandidates(mood.id, 3)
val promoted = MoodRepository.promoteCandidates(mood.id, 2)
val ready = insertSong(mediaStore, "vid-ready", "Ready")
MoodRepository.completeDownload(mood.id, promoted.first().videoId, ready.id)
val record = assertNotNull(MoodRepository.get(mood.id, "alice"))
assertEquals(1, record.songCount)
assertEquals(1, record.recommendationReady)
assertEquals(1, record.recommendationDownloading)
assertEquals(0, record.recommendationFailed)
MoodRepository.failRecommendation(mood.id, promoted.last().videoId, "boom", 3)
val failed = assertNotNull(MoodRepository.get(mood.id, "alice"))
assertEquals(0, failed.recommendationDownloading)
assertEquals(1, failed.recommendationFailed)
}
}
@Test
fun `removing a song only detaches it`() {
withDatabase { mediaStore, _ ->
val mood = createMood(recommendationCount = 0)
val song = insertSong(mediaStore, "vid-1", "One")
MoodRepository.addSong(mood.id, song.id, "search")
assertTrue(MoodRepository.removeSong(mood.id, song.id))
assertTrue(MoodRepository.songIds(mood.id).isEmpty())
assertNotNull(SongRepository.byId(song.id))
}
}
@Test
fun `GC keeps shared or playing files and deletes unreferenced ones`() {
withDatabase { mediaStore, gc ->
val firstMood = createMood(recommendationCount = 0, name = "First")
val secondMood = createMood(recommendationCount = 0, name = "Second")
val song = insertSong(mediaStore, "vid-1", "One")
val file = assertNotNull(mediaStore.file(song))
assertTrue(Files.isRegularFile(file))
MoodRepository.addSong(firstMood.id, song.id, "search")
MoodRepository.addSong(secondMood.id, song.id, "search")
MoodRepository.removeSong(firstMood.id, song.id)
assertFalse(gc.collect(song.id), "The second mood still references the song")
assertTrue(Files.isRegularFile(file))
// An open playback session protects the track even after detach.
val now = Instant.now().toString()
val session = RadioRepository.createSession("alice", secondMood.id, now)
RadioRepository.append(session.id, listOf(song.id))
MoodRepository.removeSong(secondMood.id, song.id)
assertFalse(gc.collect(song.id), "An open session still references the song")
RadioRepository.close(session.id, "alice")
assertTrue(gc.collect(song.id))
assertNull(SongRepository.byId(song.id))
assertFalse(Files.exists(mediaStore.root.resolve("vid-1")))
}
}
@Test
fun `a ready recommendation protects its file`() {
withDatabase { mediaStore, gc ->
val mood = createMood(recommendationCount = 2)
storeCandidates(mood.id, 2)
val recommendation = MoodRepository.promoteCandidates(mood.id, 1).single()
val song = insertSong(mediaStore, "vid-1", "One")
MoodRepository.completeDownload(mood.id, recommendation.videoId, song.id)
assertFalse(gc.collect(song.id))
MoodRepository.markRejected(mood.id, recommendation.videoId)
assertTrue(gc.collect(song.id))
}
}
private fun withDatabase(block: (MediaStore, MediaGcService) -> Unit) {
val handle = DatabaseFactory.connect(appConfig())
try {
ConfigRepository.sync(UsersConfig(listOf(UserAccount("alice", "pw", admin = true))))
val store = MediaStore(Files.createDirectories(tempDir.resolve("media")))
val gc = MediaGcService(store, CoverArtService(tempDir.resolve("data")))
block(store, gc)
} finally {
handle.close()
}
}
private fun createMood(recommendationCount: Int, name: String = "Focus"): MoodRecord =
MoodRepository.create(
owner = "alice",
name = name,
description = null,
query = null,
genres = emptyList(),
collectionPercent = 50,
recommendationCount = recommendationCount,
refreshHours = 24,
)
private fun storeCandidates(moodId: String, count: Int) {
MoodRepository.storeCandidates(
moodId,
(1..count).map { index ->
RecommendationUpsert(
videoId = "v-$index",
title = "Track $index",
artist = "Artist $index",
album = null,
duration = 180,
thumbnail = null,
)
},
Instant.now().toString(),
)
}
private fun insertSong(
mediaStore: MediaStore,
videoId: String,
title: String,
): SongRecord {
val file = mediaStore.target(videoId, title)
Files.createDirectories(file.parent)
Files.write(file, "fake-mp3".toByteArray())
return SongRepository.upsert(
SongUpsert(
videoId = videoId,
title = title,
artist = "Artist",
album = null,
duration = 180,
bitRate = 128,
size = Files.size(file),
relativePath = mediaStore.root.relativize(file).joinToString("/") { it.toString() },
fileMtime = Files.getLastModifiedTime(file).toMillis(),
thumbnail = null,
),
)
}
private fun appConfig(): AppConfig = AppConfig(
port = 8080,
dataDir = tempDir.resolve("data"),
usersFile = tempDir.resolve("users.yaml"),
ffmpegPath = "ffmpeg",
)
}
@@ -31,19 +31,42 @@ class DiscoveryServiceTest {
assertEquals(YtCandidate("c", "Loud", "Gamma", null, 90, null), parsed[2]) assertEquals(YtCandidate("c", "Loud", "Gamma", null, 90, null), parsed[2])
} }
@Test
fun `parses a single resolved video`() {
val json = """
{"id":"dQw4w9WgXcQ","title":"Rick Astley - Never Gonna Give You Up (Official Video)",
"channel":"Rick Astley","duration":213,"thumbnail":"http://t/x.jpg"}
""".trimIndent()
assertEquals(
YtCandidate(
videoId = "dQw4w9WgXcQ",
title = "Never Gonna Give You Up",
artist = "Rick Astley",
album = null,
duration = 213,
thumbnail = "http://t/x.jpg",
),
parseYtVideo(json),
)
}
@Test @Test
fun `malformed search output yields no candidates`() { fun `malformed search output yields no candidates`() {
assertTrue(parseYtSearchOutput("").isEmpty()) assertTrue(parseYtSearchOutput("").isEmpty())
assertTrue(parseYtSearchOutput("not json").isEmpty()) assertTrue(parseYtSearchOutput("not json").isEmpty())
assertTrue(parseYtSearchOutput("""{"entries": "nope"}""").isEmpty()) assertTrue(parseYtSearchOutput("""{"entries": "nope"}""").isEmpty())
assertEquals(null, parseYtVideo("not json"))
} }
@Test @Test
fun `music length keeps songs and unknown durations but drops long mixes`() { fun `music length keeps tracks between 30 seconds and 10 minutes`() {
assertTrue(isMusicLength(null)) assertTrue(isMusicLength(null))
assertTrue(isMusicLength(45)) assertTrue(isMusicLength(30))
assertTrue(isMusicLength(200)) assertTrue(isMusicLength(200))
assertFalse(isMusicLength(30)) assertTrue(isMusicLength(600))
assertFalse(isMusicLength(29))
assertFalse(isMusicLength(601))
assertFalse(isMusicLength(21 * 60)) assertFalse(isMusicLength(21 * 60))
} }
@@ -67,7 +90,7 @@ class DiscoveryServiceTest {
} }
@Test @Test
fun `queries combine text seed artists and genres`() { fun `queries combine text collection artists and genres`() {
val mood = mood(query = "late night", genres = listOf("Ambient", "Jazz")) val mood = mood(query = "late night", genres = listOf("Ambient", "Jazz"))
val queries = buildQueries(mood, listOf("Alpha", "Beta", "Alpha")) val queries = buildQueries(mood, listOf("Alpha", "Beta", "Alpha"))
@@ -80,15 +103,13 @@ class DiscoveryServiceTest {
} }
@Test @Test
fun `seed artists and library matches score higher`() { fun `collection artists score higher`() {
val candidate = YtCandidate("v", "Focus Two", "Beta", duration = 200) val candidate = YtCandidate("v", "Focus Two", "Beta", duration = 200)
val plain = scoreCandidate(candidate, listOf("Alpha"), "focus", matched = false) val plain = scoreCandidate(candidate, listOf("Alpha"), "focus")
val seedArtist = scoreCandidate(candidate, listOf("Beta"), "focus", matched = false) val seedArtist = scoreCandidate(candidate, listOf("Beta"), "focus")
val matched = scoreCandidate(candidate, listOf("Alpha"), "focus", matched = true)
assertTrue(seedArtist > plain) assertTrue(seedArtist > plain)
assertTrue(matched > plain)
} }
private fun mood( private fun mood(
@@ -101,15 +122,16 @@ class DiscoveryServiceTest {
description = null, description = null,
query = query, query = query,
genres = genres, genres = genres,
targetSize = 30, collectionPercent = 50,
seedPercent = 50, recommendationCount = 10,
refreshHours = 24, refreshHours = 24,
ytEnabled = true,
lastDiscoveryAt = null, lastDiscoveryAt = null,
createdAt = "2026-01-01T00:00:00Z", createdAt = "2026-01-01T00:00:00Z",
changedAt = "2026-01-01T00:00:00Z", changedAt = "2026-01-01T00:00:00Z",
lastError = null, lastError = null,
seedCount = 0, songCount = 0,
newRecommendations = 0, recommendationReady = 0,
recommendationDownloading = 0,
recommendationFailed = 0,
) )
} }
@@ -6,35 +6,28 @@ import java.nio.file.Path
import java.time.Instant import java.time.Instant
import kotlin.test.Test import kotlin.test.Test
import kotlin.test.assertEquals import kotlin.test.assertEquals
import kotlin.test.assertNotEquals
import kotlin.test.assertNotNull import kotlin.test.assertNotNull
import kotlin.test.assertNull
import kotlin.test.assertTrue import kotlin.test.assertTrue
import org.jetbrains.exposed.sql.SqlExpressionBuilder.eq
import org.jetbrains.exposed.sql.insert
import org.jetbrains.exposed.sql.insertIgnore
import org.jetbrains.exposed.sql.update
import org.jetbrains.exposed.sql.transactions.transaction
import org.junit.jupiter.api.io.TempDir import org.junit.jupiter.api.io.TempDir
import win.taptap.musicmanager.TestAudio
import win.taptap.musicmanager.config.AppConfig import win.taptap.musicmanager.config.AppConfig
import win.taptap.musicmanager.config.FolderType
import win.taptap.musicmanager.config.LibraryConfig
import win.taptap.musicmanager.config.MusicFolder
import win.taptap.musicmanager.config.UserAccount import win.taptap.musicmanager.config.UserAccount
import win.taptap.musicmanager.config.UsersConfig import win.taptap.musicmanager.config.UsersConfig
import win.taptap.musicmanager.db.AlbumArtistsTable
import win.taptap.musicmanager.db.AlbumsTable
import win.taptap.musicmanager.db.ArtistsTable
import win.taptap.musicmanager.db.ConfigRepository import win.taptap.musicmanager.db.ConfigRepository
import win.taptap.musicmanager.db.DatabaseFactory import win.taptap.musicmanager.db.DatabaseFactory
import win.taptap.musicmanager.db.DatabaseHandle import win.taptap.musicmanager.db.DatabaseHandle
import win.taptap.musicmanager.db.DownloadJobsTable
import win.taptap.musicmanager.db.DownloadRepository import win.taptap.musicmanager.db.DownloadRepository
import win.taptap.musicmanager.db.MoodRepository import win.taptap.musicmanager.db.MoodRepository
import win.taptap.musicmanager.db.MoodRecord
import win.taptap.musicmanager.db.RecommendationUpsert import win.taptap.musicmanager.db.RecommendationUpsert
import win.taptap.musicmanager.db.SongArtistsTable import win.taptap.musicmanager.db.SongRepository
import win.taptap.musicmanager.db.SongGenresTable import win.taptap.musicmanager.db.SongUpsert
import win.taptap.musicmanager.db.SongsTable import win.taptap.musicmanager.media.MediaGcService
import win.taptap.musicmanager.util.IdGenerator import win.taptap.musicmanager.media.MediaStore
import win.taptap.musicmanager.scanner.CoverArtService
import win.taptap.musicmanager.scanner.Mp3TagExtractor
import win.taptap.musicmanager.scanner.SongIndexer
class DownloadServiceTest { class DownloadServiceTest {
@@ -42,377 +35,297 @@ class DownloadServiceTest {
lateinit var tempDir: Path lateinit var tempDir: Path
@Test @Test
fun `downloaded tracks are promoted to recommendation matches and seeds`() { fun `a manual download lands in the media store and attaches to the mood`() {
withDatabase { handle, library, users -> TestAudio.assumeFfmpeg()
val mood = MoodRepository.create( withService { service, mediaStore, _ ->
owner = "alice", val mood = mood(recommendationCount = 0)
name = "Drei ???", DownloadRepository.enqueue("vid-manual", mood.id, DownloadRepository.KIND_SONG)
description = null,
query = null,
genres = emptyList(),
targetSize = 30,
refreshHours = 24,
ytEnabled = true,
seedSongIds = emptyList(),
)
MoodRepository.storeRecommendations(
mood.id,
listOf(
RecommendationUpsert(
videoId = "v-1",
title = "Neue Folge",
artist = "Die drei ???",
album = null,
duration = 300,
thumbnail = null,
matchedSongId = null,
),
),
"2026-01-01T00:00:00Z",
)
DownloadRepository.enqueue("alice", mood.id, "v-1", "Neue Folge", "Die drei ???")
var scanned = false
val service = DownloadService(
users = users,
library = library,
downloader = AudioDownloader { videoId, _, _, dir ->
val file = dir.resolve("$videoId.mp3")
Files.write(file, "fake-mp3".toByteArray())
file
},
tempDir = tempDir.resolve("downloads"),
startScan = {
if (!scanned) {
scanned = true
insertSong(handle, "Die drei ???", "Neue Folge")
}
true
},
isScanning = { false },
)
assertTrue(service.processOne()) assertTrue(service.processOne())
val job = assertNotNull(DownloadRepository.forRecommendation("alice", mood.id, "v-1")) val job = assertNotNull(
DownloadRepository.find(mood.id, "vid-manual", DownloadRepository.KIND_SONG),
)
assertEquals("done", job.state) assertEquals("done", job.state)
val songId = assertNotNull(job.songId) val songId = assertNotNull(job.songId)
assertEquals(listOf(songId), MoodRepository.songIds(mood.id))
val recommendation = assertNotNull(MoodRepository.recommendation(mood.id, "v-1")) val song = assertNotNull(SongRepository.byId(songId))
assertEquals("accepted", recommendation.state) assertEquals("vid-manual", song.videoId)
assertEquals(songId, recommendation.matchedSongId) assertEquals("Song vid-manual", song.title)
assertTrue(MoodRepository.seedSongIds(mood.id).contains(songId)) assertEquals("Artist", song.artist)
assertEquals("vid-manual/Song vid-manual.mp3", song.relativePath)
val imports = library.folders.first().path.resolve("AudioManager Imports") assertTrue(Files.isRegularFile(assertNotNull(mediaStore.file(song))))
val files = Files.walk(imports).use { stream ->
stream.filter { Files.isRegularFile(it) }.toList()
}
assertEquals(1, files.size)
assertEquals("Neue Folge.mp3", files.first().fileName.toString())
assertEquals("Neue Folge", files.first().parent.fileName.toString())
assertEquals("Die drei ___", files.first().parent.parent.fileName.toString())
} }
} }
@Test @Test
fun `downloaded file is promoted instead of an older same-title song`() { fun `an already downloaded song attaches without another download`() {
withDatabase { handle, library, users -> withService(downloader = AudioDownloader { _, _ -> error("downloader must not run") }) { service, mediaStore, _ ->
val mood = MoodRepository.create( val mood = mood(recommendationCount = 0)
owner = "alice", val song = insertSong(mediaStore, "vid-existing")
name = "Drei ???", DownloadRepository.enqueue("vid-existing", mood.id, DownloadRepository.KIND_SONG)
description = null,
query = null, assertTrue(service.processOne())
genres = emptyList(),
targetSize = 30, assertEquals(listOf(song.id), MoodRepository.songIds(mood.id))
refreshHours = 24, }
ytEnabled = true, }
seedSongIds = emptyList(),
) @Test
MoodRepository.storeRecommendations( fun `the same video is downloaded once for two moods`() {
TestAudio.assumeFfmpeg()
var downloads = 0
withService(downloader = AudioDownloader { videoId, dir -> downloads++; mp3(videoId, dir) }) { service, _, _ ->
val first = mood(recommendationCount = 0, name = "First")
val second = mood(recommendationCount = 0, name = "Second")
DownloadRepository.enqueue("vid-shared", first.id, DownloadRepository.KIND_SONG)
DownloadRepository.enqueue("vid-shared", second.id, DownloadRepository.KIND_SONG)
assertTrue(service.processOne())
assertTrue(service.processOne())
assertEquals(1, downloads)
assertEquals(1, SongRepository.count())
assertEquals(MoodRepository.songIds(first.id), MoodRepository.songIds(second.id))
}
}
@Test
fun `a recommendation download becomes ready but is not a mood song`() {
TestAudio.assumeFfmpeg()
withService { service, _, _ ->
val mood = mood(recommendationCount = 1)
storeRecommendation(mood.id, "vid-rec")
val promoted = MoodRepository.promoteCandidates(mood.id, 1).single()
DownloadRepository.enqueue(
promoted.videoId,
mood.id, mood.id,
listOf( DownloadRepository.KIND_RECOMMENDATION,
RecommendationUpsert(
videoId = "v-dup",
title = "Neue Folge",
artist = "Die drei ???",
album = null,
duration = 300,
thumbnail = null,
matchedSongId = null,
),
),
"2026-01-01T00:00:00Z",
)
DownloadRepository.enqueue("alice", mood.id, "v-dup", "Neue Folge", "Die drei ???")
val oldId = insertSong(
handle,
"Die drei ???",
"Neue Folge",
"Old/Die drei ??? - Neue Folge.mp3",
)
var newId: String? = null
val service = DownloadService(
users = users,
library = library,
downloader = AudioDownloader { videoId, _, _, dir ->
val file = dir.resolve("$videoId.mp3")
Files.write(file, "fake-mp3".toByteArray())
file
},
tempDir = tempDir.resolve("downloads"),
startScan = {
if (newId == null) {
newId = insertSong(
handle,
"Die drei ???",
"Neue Folge",
"AudioManager Imports/Die drei ___/Neue Folge/Neue Folge.mp3",
)
}
true
},
isScanning = { false },
) )
assertTrue(service.processOne()) assertTrue(service.processOne())
val job = assertNotNull(DownloadRepository.forRecommendation("alice", mood.id, "v-dup")) val recommendation = assertNotNull(
assertEquals("done", job.state) MoodRepository.recommendation(mood.id, "vid-rec"),
assertEquals(newId, job.songId) )
assertNotEquals(oldId, job.songId) assertEquals("ready", recommendation.state)
assertNotNull(recommendation.songId)
assertTrue(MoodRepository.songIds(mood.id).isEmpty())
} }
} }
@Test @Test
fun `failed downloads retry and then fail`() { fun `adding while downloading attaches when the job finishes`() {
withDatabase { _, library, users -> TestAudio.assumeFfmpeg()
val mood = MoodRepository.create( withService { service, _, _ ->
owner = "alice", val mood = mood(recommendationCount = 1)
name = "Broken", storeRecommendation(mood.id, "vid-rec")
description = null, MoodRepository.promoteCandidates(mood.id, 1)
query = null, DownloadRepository.enqueue(
genres = emptyList(), "vid-rec",
targetSize = 30, mood.id,
refreshHours = 24, DownloadRepository.KIND_RECOMMENDATION,
ytEnabled = true,
seedSongIds = emptyList(),
) )
DownloadRepository.enqueue("alice", mood.id, "v-2", "Missing", "Nobody") MoodRepository.markAdded(mood.id, "vid-rec")
val service = DownloadService(
users = users, assertTrue(service.processOne())
library = library,
downloader = AudioDownloader { _, _, _, _ -> error("network down") }, val recommendation = assertNotNull(MoodRepository.recommendation(mood.id, "vid-rec"))
tempDir = tempDir.resolve("downloads"), assertEquals("added", recommendation.state)
startScan = { true }, assertEquals(listOf(assertNotNull(recommendation.songId)), MoodRepository.songIds(mood.id))
isScanning = { false }, }
}
@Test
fun `rejecting while downloading leaves no mood song and collects the file`() {
TestAudio.assumeFfmpeg()
withService { service, mediaStore, _ ->
val mood = mood(recommendationCount = 1)
storeRecommendation(mood.id, "vid-rec")
MoodRepository.promoteCandidates(mood.id, 1)
DownloadRepository.enqueue(
"vid-rec",
mood.id,
DownloadRepository.KIND_RECOMMENDATION,
)
MoodRepository.markRejected(mood.id, "vid-rec")
assertTrue(service.processOne())
assertTrue(MoodRepository.songIds(mood.id).isEmpty())
assertNull(SongRepository.byVideoId("vid-rec"))
assertTrue(Files.notExists(mediaStore.root.resolve("vid-rec")))
}
}
@Test
fun `failures retry three times and leave the recommendation failed`() {
withService(downloader = AudioDownloader { _, _ -> error("network down") }) { service, _, _ ->
val mood = mood(recommendationCount = 1)
storeRecommendation(mood.id, "vid-broken")
MoodRepository.promoteCandidates(mood.id, 1)
DownloadRepository.enqueue(
"vid-broken",
mood.id,
DownloadRepository.KIND_RECOMMENDATION,
) )
assertTrue(service.processOne()) assertTrue(service.processOne())
assertEquals("queued", DownloadRepository.forRecommendation("alice", mood.id, "v-2")?.state)
assertTrue(service.processOne()) assertTrue(service.processOne())
assertEquals("queued", DownloadRepository.forRecommendation("alice", mood.id, "v-2")?.state)
assertTrue(service.processOne()) assertTrue(service.processOne())
val job = assertNotNull(DownloadRepository.forRecommendation("alice", mood.id, "v-2")) val job = assertNotNull(
DownloadRepository.find(mood.id, "vid-broken", DownloadRepository.KIND_RECOMMENDATION),
)
assertEquals("failed", job.state) assertEquals("failed", job.state)
assertEquals(3, job.attempts) assertEquals(3, job.attempts)
assertTrue(job.error?.contains("network down") == true) assertTrue(job.error?.contains("network down") == true)
}
}
val recommendation = assertNotNull(MoodRepository.recommendation(mood.id, "vid-broken"))
assertEquals("failed", recommendation.state)
@Test assertTrue(recommendation.error?.contains("network down") == true)
fun `stale downloading jobs are recovered`() {
withDatabase { handle, library, users ->
val mood = MoodRepository.create(
owner = "alice",
name = "Recovery",
description = null,
query = null,
genres = emptyList(),
targetSize = 30,
refreshHours = 24,
ytEnabled = true,
seedSongIds = emptyList(),
)
DownloadRepository.enqueue("alice", mood.id, "v-stale", "Stale", "Nobody")
val job = assertNotNull(DownloadRepository.forRecommendation("alice", mood.id, "v-stale"))
transaction(handle.database) {
DownloadJobsTable.update({ DownloadJobsTable.id eq job.id }) {
it[state] = "downloading"
it[updatedAt] = Instant.now().minusSeconds(3600).toString()
}
}
val claimed = assertNotNull(DownloadRepository.claimNext())
assertEquals(job.id, claimed.id)
assertEquals("downloading", claimed.state)
}
}
@Test
fun `failed verification removes the promoted file instead of duplicating it`() {
withDatabase { _, library, users ->
val mood = MoodRepository.create(
owner = "alice",
name = "Broken match",
description = null,
query = null,
genres = emptyList(),
targetSize = 30,
refreshHours = 24,
ytEnabled = true,
seedSongIds = emptyList(),
)
DownloadRepository.enqueue("alice", mood.id, "v-nomatch", "No Match", "Nobody")
val service = DownloadService(
users = users,
library = library,
downloader = AudioDownloader { videoId, _, _, dir ->
val file = dir.resolve("$videoId.mp3")
Files.write(file, "fake-mp3".toByteArray())
file
},
tempDir = tempDir.resolve("downloads"),
startScan = { true },
isScanning = { false },
)
repeat(3) { assertTrue(service.processOne()) }
val job = assertNotNull(DownloadRepository.forRecommendation("alice", mood.id, "v-nomatch"))
assertEquals("failed", job.state)
val imports = library.folders.first().path.resolve("AudioManager Imports")
if (Files.isDirectory(imports)) {
val files = Files.walk(imports).use { stream ->
stream.filter { Files.isRegularFile(it) }.toList()
}
assertTrue(files.isEmpty(), "leftover promoted files: $files")
}
} }
} }
@Test @Test
fun `permission errors fail without burning retries`() { fun `permission errors fail without burning retries`() {
withDatabase { _, library, users -> withService(
val mood = MoodRepository.create( downloader = AudioDownloader { _, _ -> throw AccessDeniedException("/data/media") },
owner = "alice", ) { service, _, _ ->
name = "Denied", val mood = mood(recommendationCount = 0)
description = null, DownloadRepository.enqueue("vid-denied", mood.id, DownloadRepository.KIND_SONG)
query = null,
genres = emptyList(),
targetSize = 30,
refreshHours = 24,
ytEnabled = true,
seedSongIds = emptyList(),
)
DownloadRepository.enqueue("alice", mood.id, "v-3", "Blocked", "Nobody")
val service = DownloadService(
users = users,
library = library,
downloader = AudioDownloader { _, _, _, _ ->
throw AccessDeniedException("/mnt/media/music/AudioManager Imports")
},
tempDir = tempDir.resolve("downloads"),
startScan = { true },
isScanning = { false },
)
assertTrue(service.processOne()) assertTrue(service.processOne())
val job = assertNotNull(DownloadRepository.forRecommendation("alice", mood.id, "v-3")) val job = assertNotNull(
DownloadRepository.find(mood.id, "vid-denied", DownloadRepository.KIND_SONG),
)
assertEquals("failed", job.state) assertEquals("failed", job.state)
assertEquals(1, job.attempts) assertEquals(1, job.attempts)
assertTrue(job.error?.contains("AccessDeniedException") == true) assertTrue(job.error?.contains("AccessDeniedException") == true)
} }
} }
private fun withDatabase(block: (DatabaseHandle, LibraryConfig, UsersConfig) -> Unit) { @Test
val musicDir = Files.createDirectories(tempDir.resolve("music")) fun `manual adds outrank recommendation downloads`() {
val library = LibraryConfig( withDatabase { _, _ ->
listOf(MusicFolder(1, "Music", FolderType.MUSIC, musicDir)), val mood = mood(recommendationCount = 1)
DownloadRepository.enqueue(
"vid-rec",
mood.id,
DownloadRepository.KIND_RECOMMENDATION,
) )
val users = UsersConfig( DownloadRepository.enqueue("vid-manual", mood.id, DownloadRepository.KIND_SONG)
listOf(UserAccount("alice", "pw", admin = true, folders = setOf(1))),
val claimed = assertNotNull(DownloadRepository.claimNext())
assertEquals("vid-manual", claimed.videoId)
assertEquals(DownloadRepository.KIND_SONG, claimed.kind)
}
}
private fun mood(recommendationCount: Int, name: String = "Focus"): MoodRecord =
MoodRepository.create(
owner = "alice",
name = name,
description = null,
query = null,
genres = emptyList(),
collectionPercent = 50,
recommendationCount = recommendationCount,
refreshHours = 24,
) )
val handle = DatabaseFactory.connect(
AppConfig( private fun storeRecommendation(moodId: String, videoId: String) {
port = 8080, MoodRepository.storeCandidates(
dataDir = tempDir.resolve("data"), moodId,
libraryConfigPath = tempDir.resolve("library.yaml"), listOf(
usersFile = tempDir.resolve("users.yaml"), RecommendationUpsert(
ffmpegPath = "ffmpeg", videoId = videoId,
downloadDir = tempDir.resolve("downloads"), title = "Recommended",
artist = "Artist",
album = null,
duration = 180,
thumbnail = null,
), ),
),
Instant.now().toString(),
)
}
private fun withService(
downloader: AudioDownloader = AudioDownloader { videoId, dir -> mp3(videoId, dir) },
block: (DownloadService, MediaStore, MediaGcService) -> Unit,
) {
withDatabase { handle, store ->
val coverArt = CoverArtService(tempDir.resolve("data"))
val indexer = SongIndexer(store, coverArt)
val gc = MediaGcService(store, coverArt)
val service = DownloadService(
downloader = downloader,
mediaStore = store,
indexer = indexer,
gc = gc,
tagExtractor = Mp3TagExtractor(),
tempDir = tempDir.resolve("downloads"),
) )
try { try {
ConfigRepository.sync(library, users) block(service, store, gc)
block(handle, library, users) } catch (t: Throwable) {
throw t
}
}
}
private fun withDatabase(block: (DatabaseHandle, MediaStore) -> Unit) {
val handle = DatabaseFactory.connect(appConfig())
try {
ConfigRepository.sync(UsersConfig(listOf(UserAccount("alice", "pw", admin = true))))
val store = MediaStore(Files.createDirectories(tempDir.resolve("media")))
block(handle, store)
} finally { } finally {
handle.close() handle.close()
} }
} }
private fun insertSong( private fun mp3(videoId: String, dir: Path): Path {
handle: DatabaseHandle, val file = dir.resolve("$videoId.mp3")
artistName: String, TestAudio.createMp3(
title: String, file,
relativePath: String = "AudioManager Imports/$artistName - $title.mp3", metadata = mapOf(
): String { "title" to "Song $videoId",
val artistId = IdGenerator.artistId(artistName) "artist" to "Artist",
val albumId = IdGenerator.albumId(artistName, "Downloads") "album" to "Album",
val songId = IdGenerator.songId(1, relativePath) ),
val now = Instant.now().toString() )
return file
}
transaction(handle.database) { private fun insertSong(mediaStore: MediaStore, videoId: String): win.taptap.musicmanager.db.SongRecord {
ArtistsTable.insertIgnore { val file = mediaStore.target(videoId, "Existing")
it[id] = artistId Files.createDirectories(file.parent)
it[name] = artistName Files.write(file, "fake-mp3".toByteArray())
it[sortName] = artistName return SongRepository.upsert(
it[searchText] = IdGenerator.normalize(artistName) SongUpsert(
} videoId = videoId,
AlbumsTable.insertIgnore { title = "Existing",
it[id] = albumId artist = "Artist",
it[name] = "Downloads" album = null,
it[createdAt] = now duration = 180,
it[searchText] = IdGenerator.normalize("$title $artistName") bitRate = 128,
} size = Files.size(file),
AlbumArtistsTable.insertIgnore { relativePath = mediaStore.root.relativize(file).joinToString("/") { it.toString() },
it[this.albumId] = albumId fileMtime = Files.getLastModifiedTime(file).toMillis(),
it[this.artistId] = artistId thumbnail = null,
it[position] = 0 ),
} )
SongsTable.insert {
it[id] = songId
it[this.albumId] = albumId
it[folderId] = 1
it[this.title] = title
it[displayArtist] = artistName
it[albumName] = "Downloads"
it[duration] = 300
it[bitRate] = 128
it[size] = 10
it[suffix] = "mp3"
it[contentType] = "audio/mpeg"
it[this.relativePath] = relativePath
it[fileMtime] = 0
it[createdAt] = now
it[updatedAt] = now
it[searchText] = IdGenerator.normalize("$title $artistName Downloads $artistName")
}
SongArtistsTable.insert {
it[this.songId] = songId
it[this.artistId] = artistId
it[position] = 0
}
SongGenresTable.insert {
it[this.songId] = songId
it[genre] = "Hörspiel"
}
}
return songId
} }
private fun appConfig(): AppConfig = AppConfig(
port = 8080,
dataDir = tempDir.resolve("data"),
usersFile = tempDir.resolve("users.yaml"),
ffmpegPath = "ffmpeg",
downloadDir = tempDir.resolve("downloads"),
)
} }
@@ -12,18 +12,21 @@ class RadioServiceTest {
private val service = RadioService() private val service = RadioService()
@Test @Test
fun `recently played songs are penalized`() { fun `recently played and already heard songs are penalized`() {
val song = song("s-1") val song = song("s-1")
val fresh = service.scoreCandidate(
song, emptySet(), emptyList(), emptyList(), null, affinity = 0.0, jitter = 0.0,
)
val recent = service.scoreCandidate(
song, emptySet(), emptyList(), emptyList(), null, affinity = 0.0,
recent = setOf("s-1"), jitter = 0.0,
)
assertEquals(0.0, fresh) assertEquals(
assertEquals(-6.0, recent) 0.0,
service.scoreCandidate(song, affinity = 0.0, recent = false, jitter = 0.0),
)
assertEquals(
-6.0,
service.scoreCandidate(song, affinity = 0.0, recent = true, jitter = 0.0),
)
assertEquals(
-20.0,
service.scoreCandidate(song, affinity = 0.0, recent = false, jitter = 0.0, heard = true),
)
} }
@Test @Test
@@ -34,145 +37,103 @@ class RadioServiceTest {
} }
@Test @Test
fun `seeds and matching genres are boosted`() { fun `higher scores are picked first`() {
val seeded = service.scoreCandidate( val scored = listOf(
song("s-1", genres = listOf("Ambient")), song("low", artist = "A") to 1.0,
seedIds = setOf("s-1"), song("high", artist = "B") to 9.0,
seedGenres = listOf("Ambient"),
seedArtistIds = emptyList(),
query = null,
affinity = 0.0,
jitter = 0.0,
)
val plain = service.scoreCandidate(
song("s-2"),
seedIds = setOf("s-1"),
seedGenres = listOf("Ambient"),
seedArtistIds = emptyList(),
query = null,
affinity = 0.0,
jitter = 0.0,
) )
assertEquals(11.5, seeded) val picked = service.pickMixed(scored, emptyList(), limit = 2, collectionPercent = 100)
assertEquals(0.0, plain)
assertEquals("high", picked.first().id)
} }
@Test @Test
fun `picks keep artists and albums apart`() { fun `picks keep artists and albums apart when possible`() {
val scored = listOf( val scored = listOf(
song("s1", artist = "A", albumId = "al1") to 0.0, song("s1", artist = "A", album = "a1") to 0.0,
song("s2", artist = "B", albumId = "al2") to 0.0, song("s2", artist = "B", album = "a2") to 0.0,
song("s3", artist = "C", albumId = "al3") to 0.0, song("s3", artist = "C", album = "a3") to 0.0,
song("s4", artist = "D", albumId = "al4") to 0.0, song("s4", artist = "D", album = "a4") to 0.0,
song("s5", artist = "A", albumId = "al5") to 0.0, song("s5", artist = "A", album = "a5") to 0.0,
song("s6", artist = "B", albumId = "al6") to 0.0, song("s6", artist = "B", album = "a6") to 0.0,
) )
val picked = service.pickDiverse(scored, limit = 6) val picked = service.pickMixed(
scored,
emptyList(),
limit = 6,
collectionPercent = 100,
artistSpacing = 2,
albumSpacing = 3,
)
assertEquals(6, picked.size) assertEquals(6, picked.size)
picked.indices.forEach { index -> picked.indices.forEach { index ->
val artists = picked.subList(index, minOf(index + 3, picked.size)).map { it.displayArtist } val artists = picked.subList(index, minOf(index + 3, picked.size)).map { it.artist }
assertEquals(artists.size, artists.distinct().size, "artist spacing at $index") assertEquals(artists.size, artists.distinct().size, "artist spacing at $index")
val albums = picked.subList(index, minOf(index + 4, picked.size)).map { it.albumId }
assertEquals(albums.size, albums.distinct().size, "album spacing at $index")
} }
} }
@Test @Test
fun `picks relax spacing when the pool is too small`() { fun `mood mix draws from collection and recommendations in proportion`() {
val scored = listOf( val collection = (1..20).map { song("col-$it", artist = "Col $it") }
song("s1", artist = "A", albumId = "al1") to 0.0,
song("s2", artist = "A", albumId = "al1") to 0.0,
)
val picked = service.pickDiverse(scored, limit = 2)
assertEquals(listOf("s1", "s2"), picked.map { it.id })
}
@Test
fun `higher scores are picked first`() {
val scored = listOf(
song("low", artist = "A", albumId = "al1") to 1.0,
song("high", artist = "B", albumId = "al2") to 9.0,
)
assertTrue(service.pickDiverse(scored, limit = 2).first().id == "high")
}
@Test
fun `mood mix draws from seeds and recommendations in proportion`() {
val seeds = (1..20).map { song("seed-$it", artist = "Seed $it") }
val recommendations = (1..20).map { song("rec-$it", artist = "Rec $it") } val recommendations = (1..20).map { song("rec-$it", artist = "Rec $it") }
val seedScored = seeds.mapIndexed { index, song -> song to (10.0 - index * 0.1) } val collectionScored = collection.mapIndexed { index, song -> song to (10.0 - index * 0.1) }
val recommendationScored = recommendations.mapIndexed { index, song -> song to (9.0 - index * 0.1) } val recommendationScored = recommendations.mapIndexed { index, song -> song to (9.0 - index * 0.1) }
val allSeeds = RadioService(Random(1)).pickMixed( val allCollection = RadioService(Random(1)).pickMixed(
seedScored, recommendationScored, limit = 10, seedPercent = 100, collectionScored, recommendationScored, limit = 10, collectionPercent = 100,
) )
assertTrue(allSeeds.all { it.id.startsWith("seed-") }) assertTrue(allCollection.all { it.id.startsWith("col-") })
val allRecommendations = RadioService(Random(1)).pickMixed( val allRecommendations = RadioService(Random(1)).pickMixed(
seedScored, recommendationScored, limit = 10, seedPercent = 0, collectionScored, recommendationScored, limit = 10, collectionPercent = 0,
) )
assertTrue(allRecommendations.all { it.id.startsWith("rec-") }) assertTrue(allRecommendations.all { it.id.startsWith("rec-") })
val mixed = RadioService(Random(7)).pickMixed( val mixed = RadioService(Random(7)).pickMixed(
seedScored, recommendationScored, limit = 20, seedPercent = 50, collectionScored, recommendationScored, limit = 20, collectionPercent = 50,
) )
assertEquals(20, mixed.size) assertEquals(20, mixed.size)
assertEquals(20, mixed.map { it.id }.distinct().size) assertEquals(20, mixed.map { it.id }.distinct().size)
assertTrue(mixed.any { it.id.startsWith("seed-") }) assertTrue(mixed.any { it.id.startsWith("col-") })
assertTrue(mixed.any { it.id.startsWith("rec-") }) assertTrue(mixed.any { it.id.startsWith("rec-") })
} }
@Test @Test
fun `mood mix falls back when one pool is empty`() { fun `mood mix falls back when one pool is empty`() {
val seeds = listOf(song("only-seed")) val onlyCollection = listOf(song("only-collection"))
val picked = RadioService(Random(1)).pickMixed( val picked = RadioService(Random(1)).pickMixed(
seedScored = seeds.map { it to 1.0 }, collectionScored = onlyCollection.map { it to 1.0 },
recommendationScored = emptyList(), recommendationScored = emptyList(),
limit = 5, limit = 5,
seedPercent = 0, collectionPercent = 0,
) )
assertEquals(listOf("only-seed"), picked.map { it.id }) assertEquals(listOf("only-collection"), picked.map { it.id })
} }
private fun song( private fun song(
id: String, id: String,
artist: String = "Artist $id", artist: String = "Artist $id",
albumId: String = "album-$id", album: String? = "Album $id",
genres: List<String> = emptyList(),
) = SongRecord( ) = SongRecord(
id = id, id = id,
albumId = albumId, videoId = "video-$id",
albumName = "Album",
artistId = "ar-$artist",
artistName = artist,
displayArtist = artist,
title = "Title $id", title = "Title $id",
trackNumber = 1, artist = artist,
discNumber = 1, album = album,
year = 2026,
duration = 180, duration = 180,
bitRate = 320, bitRate = 320,
size = 1_000, size = 1_000,
suffix = "mp3", suffix = "mp3",
contentType = "audio/mpeg", contentType = "audio/mpeg",
coverArt = null, relativePath = "video-$id/$id.mp3",
folderId = 1, fileMtime = 0,
relativePath = "$id.mp3", thumbnail = null,
created = "2026-01-01T00:00:00Z", createdAt = "2026-01-01T00:00:00Z",
updatedAt = "2026-01-01T00:00:00Z", updatedAt = "2026-01-01T00:00:00Z",
starredAt = null,
rating = null,
playCount = 0,
lastPlayedAt = null,
artists = emptyList(),
genres = genres,
) )
} }
@@ -2,33 +2,18 @@ package win.taptap.musicmanager.scanner
import java.nio.file.Files import java.nio.file.Files
import java.nio.file.Path import java.nio.file.Path
import java.nio.file.attribute.FileTime
import kotlin.test.Test import kotlin.test.Test
import kotlin.test.assertEquals import kotlin.test.assertEquals
import kotlin.test.assertNotNull import kotlin.test.assertNotNull
import kotlin.test.assertNull import kotlin.test.assertNull
import kotlin.test.assertTrue import kotlin.test.assertTrue
import org.jetbrains.exposed.sql.insert
import org.jetbrains.exposed.sql.selectAll
import org.jetbrains.exposed.sql.transactions.transaction
import org.junit.jupiter.api.io.TempDir import org.junit.jupiter.api.io.TempDir
import win.taptap.musicmanager.TestAudio import win.taptap.musicmanager.TestAudio
import win.taptap.musicmanager.config.AppConfig import win.taptap.musicmanager.config.AppConfig
import win.taptap.musicmanager.config.FolderType
import win.taptap.musicmanager.config.LibraryConfig
import win.taptap.musicmanager.config.MusicFolder
import win.taptap.musicmanager.config.UserAccount
import win.taptap.musicmanager.config.UsersConfig
import win.taptap.musicmanager.db.AlbumsTable
import win.taptap.musicmanager.db.ArtistsTable
import win.taptap.musicmanager.db.ConfigRepository
import win.taptap.musicmanager.db.DatabaseFactory import win.taptap.musicmanager.db.DatabaseFactory
import win.taptap.musicmanager.db.DatabaseHandle import win.taptap.musicmanager.db.DatabaseHandle
import win.taptap.musicmanager.db.LibraryTombstonesTable import win.taptap.musicmanager.db.SongRepository
import win.taptap.musicmanager.db.MediaRepository import win.taptap.musicmanager.media.MediaStore
import win.taptap.musicmanager.db.SongGenresTable
import win.taptap.musicmanager.db.SongsTable
import win.taptap.musicmanager.db.UserSongStateTable
import win.taptap.musicmanager.util.IdGenerator import win.taptap.musicmanager.util.IdGenerator
class MediaScannerTest { class MediaScannerTest {
@@ -36,242 +21,101 @@ class MediaScannerTest {
@TempDir @TempDir
lateinit var tempDir: Path lateinit var tempDir: Path
private lateinit var music: Path private lateinit var media: Path
private lateinit var longform: Path
@Test @Test
fun `scans, caches covers and keeps user state across metadata updates`() { fun `indexes the media root by video id and caches covers`() {
TestAudio.assumeFfmpeg() TestAudio.assumeFfmpeg()
withScanner { scanner, coverArt ->
val db = DatabaseFactory.connect(appConfig()) val one = media.resolve("vid-one/One.mp3")
try { val two = media.resolve("vid-two/Two.mp3")
val library = library() val cover = media.resolve("cover.png")
ConfigRepository.sync(library, users()) TestAudio.writePng(cover)
val albumDir = Files.createDirectories(music.resolve("Album Artist/Album"))
TestAudio.writePng(albumDir.resolve("cover.jpg"))
val one = albumDir.resolve("01 - One.mp3")
val two = albumDir.resolve("02 - Two.mp3")
TestAudio.createMp3(one, metadata = albumMetadata("One", "1/2"))
TestAudio.createMp3(two, metadata = albumMetadata("Two", "2/2"))
val book = longform.resolve("Author/Book/01 - Chapter One.mp3")
TestAudio.createMp3( TestAudio.createMp3(
book, one,
metadata = mapOf( metadata = mapOf(
"title" to "Chapter One", "title" to "One",
"artist" to "Author", "artist" to "Alpha",
"album_artist" to "Author", "album" to "Album",
"album" to "Book", "genre" to "Rock",
), ),
cover = cover,
) )
Files.writeString(music.resolve("notes.txt"), "not audio") TestAudio.createMp3(two, metadata = mapOf("title" to "Two", "artist" to "Beta"))
Files.writeString(media.resolve("notes.txt"), "not audio")
val scanner = MediaScanner(library, CoverArtService(tempDir.resolve("data"))) Files.writeString(media.resolve("loose.mp3"), "not in a video folder")
val first = scanner.scanBlocking() val first = scanner.scanBlocking()
assertNull(first.error) assertNull(first.error)
assertEquals(3L, first.count) assertEquals(2L, first.count)
assertEquals(3L, MediaRepository.countSongs()) assertEquals(2L, SongRepository.count())
val albumId = IdGenerator.albumId("Album Artist", "Album") val song = assertNotNull(SongRepository.byVideoId("vid-one"))
val songId = IdGenerator.songId(1, "Album Artist/Album/01 - One.mp3") assertEquals(IdGenerator.songId("vid-one"), song.id)
assertEquals("One", song.title)
assertEquals("Alpha", song.artist)
assertEquals("Album", song.album)
assertEquals("vid-one/One.mp3", song.relativePath)
assertEquals(128, song.bitRate)
assertEquals("mp3", song.suffix)
assertEquals("audio/mpeg", song.contentType)
assertEquals(Files.size(one), song.size)
transaction(db.database) { assertTrue(Files.isRegularFile(coverArt.coverFile(song.id)))
assertEquals(3L, SongsTable.selectAll().count())
assertEquals(2L, AlbumsTable.selectAll().count())
assertEquals(2L, ArtistsTable.selectAll().count())
assertEquals(2L, SongGenresTable.selectAll().count())
val album = AlbumsTable.selectAll() // Unchanged files are skipped on a second scan.
.where { AlbumsTable.id eq albumId }
.single()
assertEquals(albumId, album[AlbumsTable.coverArt])
val song = SongsTable.selectAll()
.where { SongsTable.id eq songId }
.single()
assertEquals(128, song[SongsTable.bitRate])
assertEquals("mp3", song[SongsTable.suffix])
assertEquals("audio/mpeg", song[SongsTable.contentType])
assertEquals(Files.size(one), song[SongsTable.size])
UserSongStateTable.insert {
it[userName] = "alice"
it[this.songId] = songId
it[playCount] = 7
}
}
assertTrue(Files.isRegularFile(tempDir.resolve("data/covers/$albumId.jpg")))
// Touching the file forces re-extraction; the play count must survive.
Files.setLastModifiedTime(one, FileTime.fromMillis(System.currentTimeMillis() + 2_000))
val second = scanner.scanBlocking() val second = scanner.scanBlocking()
assertEquals(3L, second.count) assertEquals(2L, second.count)
assertEquals(3L, MediaRepository.countSongs()) assertEquals(2L, SongRepository.count())
transaction(db.database) { // Deleting a file removes its row.
val state = UserSongStateTable.selectAll()
.where { UserSongStateTable.songId eq songId }
.single()
assertEquals(7, state[UserSongStateTable.playCount])
}
// Removing the whole album and the long-form book cleans up orphans.
Files.delete(two) Files.delete(two)
Files.delete(one)
Files.delete(book)
val third = scanner.scanBlocking() val third = scanner.scanBlocking()
assertNull(third.error) assertNull(third.error)
assertEquals(0L, third.count) // notes.txt is ignored, the three MP3s are gone assertEquals(1L, third.count)
assertEquals(0L, MediaRepository.countSongs()) assertNull(SongRepository.byVideoId("vid-two"))
transaction(db.database) { // Re-adding it indexes it again.
assertEquals(0L, SongsTable.selectAll().count()) TestAudio.createMp3(two, metadata = mapOf("title" to "Two", "artist" to "Beta"))
assertEquals(0L, AlbumsTable.selectAll().count()) val fourth = scanner.scanBlocking()
assertEquals(0L, ArtistsTable.selectAll().count()) assertEquals(2L, fourth.count)
} assertNotNull(SongRepository.byVideoId("vid-two"))
} finally {
db.close()
} }
} }
@Test @Test
fun `an unreadable folder does not delete its songs`() { fun `a missing media root does not delete its songs`() {
TestAudio.assumeFfmpeg() TestAudio.assumeFfmpeg()
val db = DatabaseFactory.connect(appConfig()) withScanner { scanner, _ ->
try { val file = media.resolve("vid-one/One.mp3")
val library = library()
ConfigRepository.sync(library, users())
val file = music.resolve("Artist/Album/song.mp3")
TestAudio.createMp3(file) TestAudio.createMp3(file)
val scanner = MediaScanner(library, CoverArtService(tempDir.resolve("data")))
assertEquals(1L, scanner.scanBlocking().count) assertEquals(1L, scanner.scanBlocking().count)
assertEquals(1L, MediaRepository.countSongs())
// Simulate an unmounted or temporarily unreadable folder. media.toFile().deleteRecursively()
music.toFile().deleteRecursively()
val broken = scanner.scanBlocking() val broken = scanner.scanBlocking()
assertNotNull(broken.error) assertNotNull(broken.error)
assertEquals(1L, MediaRepository.countSongs()) assertEquals(1L, SongRepository.count())
} finally {
db.close()
} }
} }
private fun withScanner(block: (MediaScanner, CoverArtService) -> Unit) {
@Test
fun `re-adding a deleted file clears its tombstone`() {
TestAudio.assumeFfmpeg()
val db = DatabaseFactory.connect(appConfig()) val db = DatabaseFactory.connect(appConfig())
try { try {
val library = library() val store = MediaStore(Files.createDirectories(tempDir.resolve("media")))
ConfigRepository.sync(library, users()) media = store.root
val file = music.resolve("Artist/Album/song.mp3") val coverArt = CoverArtService(tempDir.resolve("data"))
TestAudio.createMp3(file) val scanner = MediaScanner(store, SongIndexer(store, coverArt))
val scanner = MediaScanner(library, CoverArtService(tempDir.resolve("data"))) block(scanner, coverArt)
scanner.scanBlocking()
val songId = IdGenerator.songId(1, "Artist/Album/song.mp3")
Files.delete(file)
scanner.scanBlocking()
transaction(db.database) {
assertEquals(
1L,
LibraryTombstonesTable.selectAll()
.where { LibraryTombstonesTable.entityId eq songId }
.count(),
)
}
TestAudio.createMp3(file)
scanner.scanBlocking()
transaction(db.database) {
assertEquals(
0L,
LibraryTombstonesTable.selectAll()
.where { LibraryTombstonesTable.entityId eq songId }
.count(),
)
}
} finally { } finally {
db.close() db.close()
} }
} }
@Test
fun `unchanged files are skipped on a second scan`() {
TestAudio.assumeFfmpeg()
val db = DatabaseFactory.connect(appConfig())
try {
val library = library()
ConfigRepository.sync(library, users())
val file = music.resolve("Artist/Album/song.mp3")
TestAudio.createMp3(file)
TestAudio.writePng(file.parent.resolve("cover.jpg"))
val scanner = MediaScanner(library, CoverArtService(tempDir.resolve("data")))
scanner.scanBlocking()
val albumId = IdGenerator.albumId("Artist", "Album")
transaction(db.database) {
val album = AlbumsTable.selectAll()
.where { AlbumsTable.id eq albumId }
.single()
assertEquals(albumId, album[AlbumsTable.coverArt])
}
assertTrue(Files.isRegularFile(tempDir.resolve("data/covers/$albumId.jpg")))
val before = transaction(db.database) {
SongsTable.selectAll().single()[SongsTable.fileMtime]
}
val mtimeBefore = Files.getLastModifiedTime(file).toMillis()
assertEquals(mtimeBefore, before)
val second = scanner.scanBlocking()
assertEquals(1L, second.count)
assertEquals(1L, MediaRepository.countSongs())
} finally {
db.close()
}
}
private fun albumMetadata(title: String, track: String): Map<String, String> = mapOf(
"title" to title,
"artist" to "Album Artist",
"album_artist" to "Album Artist",
"album" to "Album",
"track" to track,
"genre" to "Rock",
)
private fun appConfig(): AppConfig = AppConfig( private fun appConfig(): AppConfig = AppConfig(
port = 8080, port = 8080,
dataDir = tempDir.resolve("data"), dataDir = tempDir.resolve("data"),
libraryConfigPath = tempDir.resolve("library.yaml"),
usersFile = tempDir.resolve("users.yaml"), usersFile = tempDir.resolve("users.yaml"),
ffmpegPath = "ffmpeg", ffmpegPath = "ffmpeg",
) )
private fun library(): LibraryConfig {
music = Files.createDirectories(tempDir.resolve("music"))
longform = Files.createDirectories(tempDir.resolve("longform"))
return LibraryConfig(
listOf(
MusicFolder(1, "Music", FolderType.MUSIC, music),
MusicFolder(10, "Long-form", FolderType.LONGFORM, longform),
),
)
}
private fun users(): UsersConfig = UsersConfig(
listOf(UserAccount("alice", "pw", admin = true, folders = setOf(1))),
)
} }
@@ -9,7 +9,6 @@ import kotlin.test.assertNull
import kotlin.test.assertTrue import kotlin.test.assertTrue
import org.junit.jupiter.api.io.TempDir import org.junit.jupiter.api.io.TempDir
import win.taptap.musicmanager.TestAudio import win.taptap.musicmanager.TestAudio
import win.taptap.musicmanager.util.IdGenerator
class Mp3TagExtractorTest { class Mp3TagExtractorTest {
@@ -17,95 +16,66 @@ class Mp3TagExtractorTest {
lateinit var tempDir: Path lateinit var tempDir: Path
@Test @Test
fun `reads ID3 tags and separates album artist from track artists`() { fun `reads ID3 tags from a downloaded MP3`() {
TestAudio.assumeFfmpeg() TestAudio.assumeFfmpeg()
val root = Files.createDirectories(tempDir.resolve("music")) val file = tempDir.resolve("video-id/02 - Song.mp3")
val file = root.resolve("Album Artist/Album/02 - Song.mp3")
TestAudio.createMp3( TestAudio.createMp3(
file, file,
metadata = mapOf( metadata = mapOf(
"title" to "Test Song", "title" to "Test Song",
"artist" to "Track Artist", "artist" to "Track Artist",
"album_artist" to "Album Artist",
"album" to "Test Album", "album" to "Test Album",
"track" to "2/10", "track" to "2/10",
"disc" to "1/2",
"date" to "2021-05-01", "date" to "2021-05-01",
"genre" to "Rock", "genre" to "Rock",
), ),
) )
val track = assertNotNull(Mp3TagExtractor().extract(1, root, file)) val song = assertNotNull(Mp3TagExtractor().extract(file))
assertEquals("Test Song", track.title) assertEquals("Test Song", song.title)
assertEquals("Album Artist/Album/02 - Song.mp3", track.relativePath) assertEquals("Track Artist", song.artist)
assertEquals("Test Album", track.albumTitle) assertEquals("Test Album", song.album)
assertEquals("Track Artist", track.displayArtist) assertTrue(song.durationSeconds in 1..4)
assertEquals(listOf("Track Artist"), track.artists.map { it.name }) assertTrue(song.bitRate > 0)
assertEquals(listOf("Album Artist"), track.albumArtists.map { it.name }) assertEquals(Files.size(file), song.size)
assertEquals("Album Artist", track.albumArtists.first().name) assertEquals("mp3", song.suffix)
assertEquals(2, track.trackNumber) assertEquals("audio/mpeg", song.contentType)
assertEquals(1, track.discNumber)
assertEquals(2021, track.year)
assertEquals(listOf("Rock"), track.genres)
assertTrue(track.durationSeconds in 1..4)
assertTrue(track.bitRate > 0)
assertEquals(Files.size(file), track.size)
assertEquals("mp3", track.suffix)
assertEquals("audio/mpeg", track.contentType)
assertEquals(
IdGenerator.songId(1, "Album Artist/Album/02 - Song.mp3"),
track.songId,
)
assertEquals(
IdGenerator.albumId("Album Artist", "Test Album"),
track.albumId,
)
} }
@Test @Test
fun `falls back to the folder structure when tags are missing`() { fun `falls back to the file name and a generic artist`() {
TestAudio.assumeFfmpeg() TestAudio.assumeFfmpeg()
val root = Files.createDirectories(tempDir.resolve("music")) val file = tempDir.resolve("video-id/My Track.mp3")
val file = root.resolve("Artist/Album/song.mp3")
TestAudio.createMp3(file) TestAudio.createMp3(file)
val track = assertNotNull(Mp3TagExtractor().extract(1, root, file)) val song = assertNotNull(Mp3TagExtractor().extract(file))
assertEquals("song", track.title) assertEquals("My Track", song.title)
assertEquals("Album", track.albumTitle) assertEquals("Unknown artist", song.artist)
assertEquals("Artist", track.displayArtist) assertNull(song.album)
assertEquals(listOf("Artist"), track.albumArtists.map { it.name })
} }
@Test @Test
fun `reads embedded artwork and sidecar lyrics`() { fun `reads embedded artwork`() {
TestAudio.assumeFfmpeg() TestAudio.assumeFfmpeg()
val root = Files.createDirectories(tempDir.resolve("music")) val cover = tempDir.resolve("cover.png")
val cover = root.resolve("cover.png")
TestAudio.writePng(cover) TestAudio.writePng(cover)
val file = root.resolve("Artist/Album/song.mp3") val file = tempDir.resolve("video-id/covered.mp3")
TestAudio.createMp3( TestAudio.createMp3(file, metadata = mapOf("title" to "Covered"), cover = cover)
file,
metadata = mapOf("title" to "Covered"),
cover = cover,
)
Files.writeString(file.resolveSibling("song.lrc"), "[00:00.00]Hello")
val track = assertNotNull(Mp3TagExtractor().extract(1, root, file)) val song = assertNotNull(Mp3TagExtractor().extract(file))
assertNotNull(track.coverBytes) assertNotNull(song.coverBytes)
assertTrue(track.coverBytes!!.isNotEmpty()) assertTrue(song.coverBytes!!.isNotEmpty())
assertEquals("[00:00.00]Hello", track.lyrics)
} }
@Test @Test
fun `ignores non-mp3 files`() { fun `ignores non-mp3 files`() {
val root = Files.createDirectories(tempDir.resolve("music")) val file = tempDir.resolve("video-id/song.flac")
val file = root.resolve("song.flac") Files.createDirectories(file.parent)
Files.writeString(file, "not audio") Files.writeString(file, "not audio")
assertNull(Mp3TagExtractor().extract(1, root, file)) assertNull(Mp3TagExtractor().extract(file))
} }
} }
@@ -15,42 +15,29 @@ class IdGeneratorTest {
} }
@Test @Test
fun `IDs are deterministic, prefixed and 24 hex characters`() { fun `song IDs are deterministic, prefixed and 24 hex characters`() {
val first = IdGenerator.artistId("The Beatles") val first = IdGenerator.songId("dQw4w9WgXcQ")
val second = IdGenerator.artistId("the beatles") val second = IdGenerator.songId(" dQw4w9WgXcQ ")
assertEquals(first, second) assertEquals(first, second)
assertTrue(first.startsWith("ar-")) assertTrue(first.startsWith("sg-"))
assertEquals(27, first.length) assertEquals(27, first.length)
val album = IdGenerator.albumId("The Beatles", "Abbey Road")
assertTrue(album.startsWith("al-"))
assertEquals(27, album.length)
val song = IdGenerator.songId(1, "The Beatles/Abbey Road/01 - Come Together.mp3")
assertTrue(song.startsWith("sg-"))
assertEquals(27, song.length)
} }
@Test @Test
fun `path separators do not change the song ID`() { fun `different videos produce different IDs`() {
val unix = IdGenerator.songId(1, "Artist/Album/song.mp3") assertNotEquals(
val windows = IdGenerator.songId(1, "Artist\\Album\\song.mp3") IdGenerator.songId("dQw4w9WgXcQ"),
val prefixed = IdGenerator.songId(1, "./Artist/Album/song.mp3") IdGenerator.songId("dQw4w9WgXcR"),
)
assertEquals(unix, windows)
assertEquals(unix, prefixed)
} }
@Test @Test
fun `folder and album changes produce different IDs`() { fun `mood IDs are deterministic for the same inputs`() {
assertNotEquals( val id = IdGenerator.moodId("Alice", "Focus", "2026-01-01T00:00:00Z")
IdGenerator.songId(1, "Artist/Album/song.mp3"),
IdGenerator.songId(2, "Artist/Album/song.mp3"), assertTrue(id.startsWith("mo-"))
) assertEquals(id, IdGenerator.moodId("alice", " Focus ", "2026-01-01T00:00:00Z"))
assertNotEquals( assertNotEquals(id, IdGenerator.moodId("Alice", "Focus", "2026-01-02T00:00:00Z"))
IdGenerator.albumId("Artist", "Album"),
IdGenerator.albumId("Artist", "Other Album"),
)
} }
} }