feat: base discovery on the whole mood collection
- seed from every artist (weighted by song count) and every album, not one track - walk Deezer's graph: artist top tracks, related artists, album deep cuts - score by artist frequency, collection album, catalog affinity and hint text - penalise repeated artists so a 30-slot mood spans artists, not one - refresh the pool when a song is added instead of waiting for refreshHours - skip discovery for a mood with no hint, genres or songs - extend tests and the smoke test with the artist graph
This commit is contained in:
1 parent
a78d359d0c
commit
62bcefad20
13 files changed
+911
-72
No files matched your search
+69
-16
@@ -254,19 +254,32 @@ V5/V6/V7 before V8 can wipe it.
|
||||
States: `new` (hidden candidate pool), `downloading`, `ready`, `failed`,
|
||||
`added`, `rejected`.
|
||||
|
||||
- Discovery stores up to a bounded pool of `new` candidates (e.g. 60) from the
|
||||
collection's artists/titles + `query` + `genres` + mood name. Exclude video
|
||||
ids already in `mood_songs`, `added` or `rejected`.
|
||||
- Discovery reads the **whole collection** (not one seed track): every artist
|
||||
with its song count, every album, `query`, `genres` and the mood name.
|
||||
- Discovery has two candidate sources, merged and ranked together:
|
||||
1. **Text queries**: `query`, then up to 3 collection artists ordered by how
|
||||
many songs the mood has, then up to 2 genres, capped at 4 queries.
|
||||
2. **Catalog similarity graph** (`MusicCatalog.discover`): up to 5 collection
|
||||
artists -> their top tracks; up to 2 of those artists' related artists ->
|
||||
their top tracks; up to 2 collection albums -> album deep cuts. Deezer
|
||||
implements this; YouTube Music has no such API and returns nothing.
|
||||
- Scoring: exact collection artist `3.0 + 0.5` per extra song (capped `+1.5`),
|
||||
partial artist `1.5`, album already in the collection `+1.0`, catalog
|
||||
affinity (related artist `1.5`, album deep cut `1.0`, artist top `0.5`),
|
||||
mood words in the title `+1.0`, over 8 minutes `-1.0`.
|
||||
- Stores up to a bounded pool of `new` candidates (60). Excludes external ids
|
||||
already in `mood_songs`, `added` or `rejected`.
|
||||
- `topUp(mood, force)`:
|
||||
1. Count active = `downloading + ready + failed`.
|
||||
2. If `active >= recommendationCount`, stop.
|
||||
3. Promote the best `new` candidates to `downloading` and enqueue
|
||||
`download_jobs` until the count is met.
|
||||
4. If more slots remain and (`force || discoveryDue(mood)`), run discovery,
|
||||
then promote again.
|
||||
- `force = true` on user add/reject/retry and on the explicit refresh action.
|
||||
Coalesce per-mood discovery with an in-flight guard. Background passes use
|
||||
`force = false` and respect `refreshHours`.
|
||||
2. Promote the best `new` candidates to `downloading` and enqueue
|
||||
`download_jobs` until `recommendationCount` is met.
|
||||
3. Run discovery when slots are missing, or when a song was added after the
|
||||
last pass (the pool is based on the collection), and
|
||||
(`force || discoveryDue(mood)`).
|
||||
4. Promote again after the pass.
|
||||
- `force = true` on user add/reject/retry, on a finished manual download and on
|
||||
the explicit refresh action. Background passes use `force = false` and run
|
||||
hourly.
|
||||
- `recommendationCount = 0` stops discovery and downloads for the mood.
|
||||
- On download success:
|
||||
- `downloading` -> `ready` with `song_id`.
|
||||
@@ -281,6 +294,43 @@ States: `new` (hidden candidate pool), `downloading`, `ready`, `failed`,
|
||||
- Retry: `failed` -> `downloading`, reset the job, top up.
|
||||
- `new` candidates are not shown in the client.
|
||||
|
||||
### How this compares to the big services
|
||||
|
||||
The commercial engines are trained on behaviour this server does not have: no
|
||||
account-level listening history, no cross-user co-occurrence, no audio models.
|
||||
Still, their published architecture explains which parts of the problem can be
|
||||
replicated from a mood collection alone.
|
||||
|
||||
- **Spotify** combines content-based features (audio features such as
|
||||
danceability, energy and valence, plus metadata and text/visual embeddings)
|
||||
with collaborative filtering that treats playlist co-occurrence as the
|
||||
similarity signal ("two tracks are similar if people curate them together").
|
||||
Retrieval produces candidates, a ranking stage orders them, and each surface
|
||||
has its own reward (skips, saves, session length).
|
||||
- **YouTube Music** runs retrieval -> ranking -> filtering. Candidates come from
|
||||
a deep model over watch history, and a 2024 transformer encodes the *sequence*
|
||||
of user actions (intent, how much of a track was played, artist, language)
|
||||
before blending with the ranker. Audio-text embeddings (MuLan) describe the
|
||||
content itself.
|
||||
- **Deezer Flow / Flow Moods** (RecSys 2022) mixes collaborative filtering,
|
||||
audio analysis and professional curator mood annotations to build
|
||||
mood-specific radios; the 2026 Flow Tuner adds genre toggles, explicit
|
||||
dislikes, long-term habits and context such as time of day.
|
||||
|
||||
What this server reproduces without telemetry:
|
||||
|
||||
| Signal | Here | There |
|
||||
|---|---|---|
|
||||
| Taste profile | whole mood collection, artists weighted by song count + albums | per-user history and playlists |
|
||||
| Similarity | catalog graph (related artists, artist top tracks, album deep cuts) + text queries | learned embeddings + co-occurrence |
|
||||
| Feedback | add / not interested (permanent) | skips, saves, play counts, survey models |
|
||||
| Context | mood hint, genres, refresh window | session, time of day, device, activity |
|
||||
| Ranking | weighted score + per-artist diversity penalty | deep rankers with hundreds of features |
|
||||
|
||||
Not reproducible here: cross-user collaborative filtering, audio analysis and
|
||||
sequence models. The add/reject lifecycle and the collection itself are the
|
||||
only behavioural signals, which is why discovery leans on the catalog graph.
|
||||
|
||||
### 4.4 Download pipeline
|
||||
|
||||
- `download_jobs` are global; `kind=song` (manual add) has higher priority than
|
||||
@@ -626,9 +676,12 @@ Phases 1-8 are implemented in both repositories.
|
||||
the scanner and the downloader.
|
||||
- `MoodRepository` implements the song collection, hidden `new` candidates,
|
||||
active slots and the add/reject/retry lifecycle.
|
||||
- `DiscoveryService` + `TopUpService` implement collection-based discovery and
|
||||
slot refills. `TopUpService` coalesces work behind a mutex and can be triggered
|
||||
fire-and-forget from API handlers.
|
||||
- `DiscoveryService` + `TopUpService` implement whole-collection discovery and
|
||||
slot refills: text queries plus `MusicCatalog.discover` (Deezer related
|
||||
artists, artist top tracks, album deep cuts; YouTube Music text only).
|
||||
`TopUpService` coalesces work behind a mutex and can be triggered
|
||||
fire-and-forget from API handlers and finished downloads. A song added after
|
||||
the last pass marks discovery due again, so the pool follows the collection.
|
||||
- `DownloadService` is a single global worker with `song` jobs outranking
|
||||
`recommendation` jobs. It indexes immediately after moving the file.
|
||||
- `MediaGcService` implements refcounted deletion (moods, active recommendations,
|
||||
@@ -658,8 +711,8 @@ Phases 1-8 are implemented in both repositories.
|
||||
- Deezer is the default for new moods; every mood stores its own source and
|
||||
can switch it, which clears the active recommendation pool and refills it.
|
||||
- Chart 0.4.0 adds `DEFAULT_SOURCE`, `DEEZER_*` and an optional ARL secret.
|
||||
- Tests: 83 server tests, 64-step smoke test with fake `yt-dlp`, `rip` and a
|
||||
fake Deezer API.
|
||||
- Tests: 94 server tests, 66-step smoke test with fake `yt-dlp`, `rip` and a
|
||||
fake Deezer API (including the artist graph).
|
||||
|
||||
### Client
|
||||
|
||||
|
||||
Reference in new issue
Block a user