feat: base discovery on the whole mood collection
CI / chart (push) Successful in 3s
CI / test (push) Successful in 1m56s
CI / chart-push (push) Successful in 5s
CI / image (push) Successful in 3m34s

- seed from every artist (weighted by song count) and every album, not one track
- walk Deezer's graph: artist top tracks, related artists, album deep cuts
- score by artist frequency, collection album, catalog affinity and hint text
- penalise repeated artists so a 30-slot mood spans artists, not one
- refresh the pool when a song is added instead of waiting for refreshHours
- skip discovery for a mood with no hint, genres or songs
- extend tests and the smoke test with the artist graph
This commit is contained in:
TapTap committed 2026-11-17 10:37:47 +00:00
1 parent a78d359d0c
commit 62bcefad20
13 files changed
+911 -72

No files matched your search

+69 -16
View File
@@ -254,19 +254,32 @@ V5/V6/V7 before V8 can wipe it.
States: `new` (hidden candidate pool), `downloading`, `ready`, `failed`,
`added`, `rejected`.
- Discovery stores up to a bounded pool of `new` candidates (e.g. 60) from the
collection's artists/titles + `query` + `genres` + mood name. Exclude video
ids already in `mood_songs`, `added` or `rejected`.
- Discovery reads the **whole collection** (not one seed track): every artist
with its song count, every album, `query`, `genres` and the mood name.
- Discovery has two candidate sources, merged and ranked together:
1. **Text queries**: `query`, then up to 3 collection artists ordered by how
many songs the mood has, then up to 2 genres, capped at 4 queries.
2. **Catalog similarity graph** (`MusicCatalog.discover`): up to 5 collection
artists -> their top tracks; up to 2 of those artists' related artists ->
their top tracks; up to 2 collection albums -> album deep cuts. Deezer
implements this; YouTube Music has no such API and returns nothing.
- Scoring: exact collection artist `3.0 + 0.5` per extra song (capped `+1.5`),
partial artist `1.5`, album already in the collection `+1.0`, catalog
affinity (related artist `1.5`, album deep cut `1.0`, artist top `0.5`),
mood words in the title `+1.0`, over 8 minutes `-1.0`.
- Stores up to a bounded pool of `new` candidates (60). Excludes external ids
already in `mood_songs`, `added` or `rejected`.
- `topUp(mood, force)`:
1. Count active = `downloading + ready + failed`.
2. If `active >= recommendationCount`, stop.
3. Promote the best `new` candidates to `downloading` and enqueue
`download_jobs` until the count is met.
4. If more slots remain and (`force || discoveryDue(mood)`), run discovery,
then promote again.
- `force = true` on user add/reject/retry and on the explicit refresh action.
Coalesce per-mood discovery with an in-flight guard. Background passes use
`force = false` and respect `refreshHours`.
2. Promote the best `new` candidates to `downloading` and enqueue
`download_jobs` until `recommendationCount` is met.
3. Run discovery when slots are missing, or when a song was added after the
last pass (the pool is based on the collection), and
(`force || discoveryDue(mood)`).
4. Promote again after the pass.
- `force = true` on user add/reject/retry, on a finished manual download and on
the explicit refresh action. Background passes use `force = false` and run
hourly.
- `recommendationCount = 0` stops discovery and downloads for the mood.
- On download success:
- `downloading` -> `ready` with `song_id`.
@@ -281,6 +294,43 @@ States: `new` (hidden candidate pool), `downloading`, `ready`, `failed`,
- Retry: `failed` -> `downloading`, reset the job, top up.
- `new` candidates are not shown in the client.
### How this compares to the big services
The commercial engines are trained on behaviour this server does not have: no
account-level listening history, no cross-user co-occurrence, no audio models.
Still, their published architecture explains which parts of the problem can be
replicated from a mood collection alone.
- **Spotify** combines content-based features (audio features such as
danceability, energy and valence, plus metadata and text/visual embeddings)
with collaborative filtering that treats playlist co-occurrence as the
similarity signal ("two tracks are similar if people curate them together").
Retrieval produces candidates, a ranking stage orders them, and each surface
has its own reward (skips, saves, session length).
- **YouTube Music** runs retrieval -> ranking -> filtering. Candidates come from
a deep model over watch history, and a 2024 transformer encodes the *sequence*
of user actions (intent, how much of a track was played, artist, language)
before blending with the ranker. Audio-text embeddings (MuLan) describe the
content itself.
- **Deezer Flow / Flow Moods** (RecSys 2022) mixes collaborative filtering,
audio analysis and professional curator mood annotations to build
mood-specific radios; the 2026 Flow Tuner adds genre toggles, explicit
dislikes, long-term habits and context such as time of day.
What this server reproduces without telemetry:
| Signal | Here | There |
|---|---|---|
| Taste profile | whole mood collection, artists weighted by song count + albums | per-user history and playlists |
| Similarity | catalog graph (related artists, artist top tracks, album deep cuts) + text queries | learned embeddings + co-occurrence |
| Feedback | add / not interested (permanent) | skips, saves, play counts, survey models |
| Context | mood hint, genres, refresh window | session, time of day, device, activity |
| Ranking | weighted score + per-artist diversity penalty | deep rankers with hundreds of features |
Not reproducible here: cross-user collaborative filtering, audio analysis and
sequence models. The add/reject lifecycle and the collection itself are the
only behavioural signals, which is why discovery leans on the catalog graph.
### 4.4 Download pipeline
- `download_jobs` are global; `kind=song` (manual add) has higher priority than
@@ -626,9 +676,12 @@ Phases 1-8 are implemented in both repositories.
the scanner and the downloader.
- `MoodRepository` implements the song collection, hidden `new` candidates,
active slots and the add/reject/retry lifecycle.
- `DiscoveryService` + `TopUpService` implement collection-based discovery and
slot refills. `TopUpService` coalesces work behind a mutex and can be triggered
fire-and-forget from API handlers.
- `DiscoveryService` + `TopUpService` implement whole-collection discovery and
slot refills: text queries plus `MusicCatalog.discover` (Deezer related
artists, artist top tracks, album deep cuts; YouTube Music text only).
`TopUpService` coalesces work behind a mutex and can be triggered
fire-and-forget from API handlers and finished downloads. A song added after
the last pass marks discovery due again, so the pool follows the collection.
- `DownloadService` is a single global worker with `song` jobs outranking
`recommendation` jobs. It indexes immediately after moving the file.
- `MediaGcService` implements refcounted deletion (moods, active recommendations,
@@ -658,8 +711,8 @@ Phases 1-8 are implemented in both repositories.
- Deezer is the default for new moods; every mood stores its own source and
can switch it, which clears the active recommendation pool and refills it.
- Chart 0.4.0 adds `DEFAULT_SOURCE`, `DEEZER_*` and an optional ARL secret.
- Tests: 83 server tests, 64-step smoke test with fake `yt-dlp`, `rip` and a
fake Deezer API.
- Tests: 94 server tests, 66-step smoke test with fake `yt-dlp`, `rip` and a
fake Deezer API (including the artist graph).
### Client