feat: base discovery on the whole mood collection
CI / chart (push) Successful in 3s
CI / test (push) Successful in 1m56s
CI / chart-push (push) Successful in 5s
CI / image (push) Successful in 3m34s

- seed from every artist (weighted by song count) and every album, not one track
- walk Deezer's graph: artist top tracks, related artists, album deep cuts
- score by artist frequency, collection album, catalog affinity and hint text
- penalise repeated artists so a 30-slot mood spans artists, not one
- refresh the pool when a song is added instead of waiting for refreshHours
- skip discovery for a mood with no hint, genres or songs
- extend tests and the smoke test with the artist graph
This commit is contained in:
TapTap committed 2026-11-17 10:37:47 +00:00
1 parent a78d359d0c
commit 62bcefad20
13 files changed
+899 -60

No files matched your search

+5
View File
@@ -36,6 +36,11 @@ keeps playing without a mounted music library.
- **Recommendations are active slots.** A mood keeps `recommendationCount` - **Recommendations are active slots.** A mood keeps `recommendationCount`
(0–30, default 10) recommendations in `downloading`, `ready` or `failed` (0–30, default 10) recommendations in `downloading`, `ready` or `failed`
state. Hidden `new` candidates refill the slots automatically. state. Hidden `new` candidates refill the slots automatically.
- **Discovery uses the whole mood.** Every artist (weighted by how many songs
the mood has), every album, the hint and the genres feed one pass. On Deezer
the server also walks Deezer's artist graph: top tracks of the collection
artists, their related artists, and deep cuts from the collection's albums.
Adding a song makes the pool refresh instead of waiting for `refreshHours`.
- **Add / Not interested are permanent.** Adding promotes a recommendation into - **Add / Not interested are permanent.** Adding promotes a recommendation into
the collection. Rejecting is remembered per mood. Both free a slot and the the collection. Rejecting is remembered per mood. Both free a slot and the
server refills it. server refills it.
+69 -16
View File
@@ -254,19 +254,32 @@ V5/V6/V7 before V8 can wipe it.
States: `new` (hidden candidate pool), `downloading`, `ready`, `failed`, States: `new` (hidden candidate pool), `downloading`, `ready`, `failed`,
`added`, `rejected`. `added`, `rejected`.
- Discovery stores up to a bounded pool of `new` candidates (e.g. 60) from the - Discovery reads the **whole collection** (not one seed track): every artist
collection's artists/titles + `query` + `genres` + mood name. Exclude video with its song count, every album, `query`, `genres` and the mood name.
ids already in `mood_songs`, `added` or `rejected`. - Discovery has two candidate sources, merged and ranked together:
1. **Text queries**: `query`, then up to 3 collection artists ordered by how
many songs the mood has, then up to 2 genres, capped at 4 queries.
2. **Catalog similarity graph** (`MusicCatalog.discover`): up to 5 collection
artists -> their top tracks; up to 2 of those artists' related artists ->
their top tracks; up to 2 collection albums -> album deep cuts. Deezer
implements this; YouTube Music has no such API and returns nothing.
- Scoring: exact collection artist `3.0 + 0.5` per extra song (capped `+1.5`),
partial artist `1.5`, album already in the collection `+1.0`, catalog
affinity (related artist `1.5`, album deep cut `1.0`, artist top `0.5`),
mood words in the title `+1.0`, over 8 minutes `-1.0`.
- Stores up to a bounded pool of `new` candidates (60). Excludes external ids
already in `mood_songs`, `added` or `rejected`.
- `topUp(mood, force)`: - `topUp(mood, force)`:
1. Count active = `downloading + ready + failed`. 1. Count active = `downloading + ready + failed`.
2. If `active >= recommendationCount`, stop. 2. Promote the best `new` candidates to `downloading` and enqueue
3. Promote the best `new` candidates to `downloading` and enqueue `download_jobs` until `recommendationCount` is met.
`download_jobs` until the count is met. 3. Run discovery when slots are missing, or when a song was added after the
4. If more slots remain and (`force || discoveryDue(mood)`), run discovery, last pass (the pool is based on the collection), and
then promote again. (`force || discoveryDue(mood)`).
- `force = true` on user add/reject/retry and on the explicit refresh action. 4. Promote again after the pass.
Coalesce per-mood discovery with an in-flight guard. Background passes use - `force = true` on user add/reject/retry, on a finished manual download and on
`force = false` and respect `refreshHours`. the explicit refresh action. Background passes use `force = false` and run
hourly.
- `recommendationCount = 0` stops discovery and downloads for the mood. - `recommendationCount = 0` stops discovery and downloads for the mood.
- On download success: - On download success:
- `downloading` -> `ready` with `song_id`. - `downloading` -> `ready` with `song_id`.
@@ -281,6 +294,43 @@ States: `new` (hidden candidate pool), `downloading`, `ready`, `failed`,
- Retry: `failed` -> `downloading`, reset the job, top up. - Retry: `failed` -> `downloading`, reset the job, top up.
- `new` candidates are not shown in the client. - `new` candidates are not shown in the client.
### How this compares to the big services
The commercial engines are trained on behaviour this server does not have: no
account-level listening history, no cross-user co-occurrence, no audio models.
Still, their published architecture explains which parts of the problem can be
replicated from a mood collection alone.
- **Spotify** combines content-based features (audio features such as
danceability, energy and valence, plus metadata and text/visual embeddings)
with collaborative filtering that treats playlist co-occurrence as the
similarity signal ("two tracks are similar if people curate them together").
Retrieval produces candidates, a ranking stage orders them, and each surface
has its own reward (skips, saves, session length).
- **YouTube Music** runs retrieval -> ranking -> filtering. Candidates come from
a deep model over watch history, and a 2024 transformer encodes the *sequence*
of user actions (intent, how much of a track was played, artist, language)
before blending with the ranker. Audio-text embeddings (MuLan) describe the
content itself.
- **Deezer Flow / Flow Moods** (RecSys 2022) mixes collaborative filtering,
audio analysis and professional curator mood annotations to build
mood-specific radios; the 2026 Flow Tuner adds genre toggles, explicit
dislikes, long-term habits and context such as time of day.
What this server reproduces without telemetry:
| Signal | Here | There |
|---|---|---|
| Taste profile | whole mood collection, artists weighted by song count + albums | per-user history and playlists |
| Similarity | catalog graph (related artists, artist top tracks, album deep cuts) + text queries | learned embeddings + co-occurrence |
| Feedback | add / not interested (permanent) | skips, saves, play counts, survey models |
| Context | mood hint, genres, refresh window | session, time of day, device, activity |
| Ranking | weighted score + per-artist diversity penalty | deep rankers with hundreds of features |
Not reproducible here: cross-user collaborative filtering, audio analysis and
sequence models. The add/reject lifecycle and the collection itself are the
only behavioural signals, which is why discovery leans on the catalog graph.
### 4.4 Download pipeline ### 4.4 Download pipeline
- `download_jobs` are global; `kind=song` (manual add) has higher priority than - `download_jobs` are global; `kind=song` (manual add) has higher priority than
@@ -626,9 +676,12 @@ Phases 1-8 are implemented in both repositories.
the scanner and the downloader. the scanner and the downloader.
- `MoodRepository` implements the song collection, hidden `new` candidates, - `MoodRepository` implements the song collection, hidden `new` candidates,
active slots and the add/reject/retry lifecycle. active slots and the add/reject/retry lifecycle.
- `DiscoveryService` + `TopUpService` implement collection-based discovery and - `DiscoveryService` + `TopUpService` implement whole-collection discovery and
slot refills. `TopUpService` coalesces work behind a mutex and can be triggered slot refills: text queries plus `MusicCatalog.discover` (Deezer related
fire-and-forget from API handlers. artists, artist top tracks, album deep cuts; YouTube Music text only).
`TopUpService` coalesces work behind a mutex and can be triggered
fire-and-forget from API handlers and finished downloads. A song added after
the last pass marks discovery due again, so the pool follows the collection.
- `DownloadService` is a single global worker with `song` jobs outranking - `DownloadService` is a single global worker with `song` jobs outranking
`recommendation` jobs. It indexes immediately after moving the file. `recommendation` jobs. It indexes immediately after moving the file.
- `MediaGcService` implements refcounted deletion (moods, active recommendations, - `MediaGcService` implements refcounted deletion (moods, active recommendations,
@@ -658,8 +711,8 @@ Phases 1-8 are implemented in both repositories.
- Deezer is the default for new moods; every mood stores its own source and - Deezer is the default for new moods; every mood stores its own source and
can switch it, which clears the active recommendation pool and refills it. can switch it, which clears the active recommendation pool and refills it.
- Chart 0.4.0 adds `DEFAULT_SOURCE`, `DEEZER_*` and an optional ARL secret. - Chart 0.4.0 adds `DEFAULT_SOURCE`, `DEEZER_*` and an optional ARL secret.
- Tests: 83 server tests, 64-step smoke test with fake `yt-dlp`, `rip` and a - Tests: 94 server tests, 66-step smoke test with fake `yt-dlp`, `rip` and a
fake Deezer API. fake Deezer API (including the artist graph).
### Client ### Client
+70 -2
View File
@@ -114,6 +114,37 @@ TRACKS = {
"artist": {"name": "DJ"}, "album": {"title": "Long"}}, "artist": {"name": "DJ"}, "album": {"title": "Long"}},
} }
# Only resolvable by URL, so adding it cannot race a recommendation download.
EXTRA_TRACKS = {
"55555555": {"id": 55555555, "title": "One More Time", "duration": 320,
"readable": True, "artist": {"name": "Daft Punk"},
"album": {"title": "Discovery"}},
}
# Artist graph: the mood holds Daft Punk, so discovery walks to a related act
# and to a deep cut of an album already in the collection.
ARTISTS = {
"27": {"id": 27, "name": "Daft Punk"},
"99": {"id": 99, "name": "Daft Punk Tribute"},
}
TOP_TRACKS = {
"27": [TRACKS["77777777"]],
"99": [{"id": 88888888, "title": "Robot Rock", "duration": 305,
"readable": True, "artist": {"name": "Daft Punk Tribute"},
"album": {"title": "Tribute"}}],
}
RELATED = {"27": [ARTISTS["99"]]}
ALBUMS = {"5": {"id": 5, "title": "Random Access Memories",
"artist": {"name": "Daft Punk"}}}
ALBUM_TRACKS = {
"5": [{"id": 99999999, "title": "Contact", "duration": 382,
"readable": True, "artist": {"name": "Daft Punk"},
"album": {"title": "Random Access Memories"}}],
}
STATS = {"artist_search": 0, "artist_top": 0, "artist_related": 0,
"album_search": 0, "album_tracks": 0}
class Handler(BaseHTTPRequestHandler): class Handler(BaseHTTPRequestHandler):
def log_message(self, *args): def log_message(self, *args):
pass pass
@@ -128,10 +159,28 @@ class Handler(BaseHTTPRequestHandler):
def do_GET(self): def do_GET(self):
parsed = urllib.parse.urlparse(self.path) parsed = urllib.parse.urlparse(self.path)
parts = [p for p in parsed.path.split("/") if p]
if parsed.path == "/search/track": if parsed.path == "/search/track":
self.reply({"data": list(TRACKS.values())}) self.reply({"data": list(TRACKS.values())})
elif parsed.path == "/search/artist":
STATS["artist_search"] += 1
self.reply({"data": list(ARTISTS.values())})
elif parsed.path == "/search/album":
STATS["album_search"] += 1
self.reply({"data": list(ALBUMS.values())})
elif parsed.path == "/stats":
self.reply(STATS)
elif len(parts) == 3 and parts[0] == "artist" and parts[2] == "top":
STATS["artist_top"] += 1
self.reply({"data": TOP_TRACKS.get(parts[1], [])})
elif len(parts) == 3 and parts[0] == "artist" and parts[2] == "related":
STATS["artist_related"] += 1
self.reply({"data": RELATED.get(parts[1], [])})
elif len(parts) == 3 and parts[0] == "album" and parts[2] == "tracks":
STATS["album_tracks"] += 1
self.reply({"data": ALBUM_TRACKS.get(parts[1], [])})
elif parsed.path.startswith("/track/"): elif parsed.path.startswith("/track/"):
track = TRACKS.get(parsed.path.rsplit("/", 1)[-1]) track = TRACKS.get(parts[-1]) or EXTRA_TRACKS.get(parts[-1])
if track is None: if track is None:
self.reply({"error": {"type": "DataException", "message": "no track"}}, 404) self.reply({"error": {"type": "DataException", "message": "no track"}}, 404)
else: else:
@@ -160,6 +209,7 @@ case "$id" in
67238732) title="Instant Crush"; artist="Daft Punk"; album="Random Access Memories" ;; 67238732) title="Instant Crush"; artist="Daft Punk"; album="Random Access Memories" ;;
66609426) title="Get Lucky"; artist="Daft Punk"; album="Get Lucky" ;; 66609426) title="Get Lucky"; artist="Daft Punk"; album="Get Lucky" ;;
77777777) title="Around the World"; artist="Daft Punk"; album="Homework" ;; 77777777) title="Around the World"; artist="Daft Punk"; album="Homework" ;;
55555555) title="One More Time"; artist="Daft Punk"; album="Discovery" ;;
*) title="Deezer Track $id"; artist="Deezer Artist"; album="Deezer Album" ;; *) title="Deezer Track $id"; artist="Deezer Artist"; album="Deezer Album" ;;
esac esac
ffmpeg -hide_banner -loglevel error -y \ ffmpeg -hide_banner -loglevel error -y \
@@ -333,8 +383,26 @@ expect "add a Deezer track" "202" "POST" "/moods/$DZ_MOOD/songs" \
wait_for 90 "Deezer download attaches to the mood" 'd["mood"]["songCount"] >= 1' "/moods/$DZ_MOOD" wait_for 90 "Deezer download attaches to the mood" 'd["mood"]["songCount"] >= 1' "/moods/$DZ_MOOD"
wait_for 90 "Deezer recommendations download" 'd["mood"]["recommendationReady"] >= 1' "/moods/$DZ_MOOD" wait_for 90 "Deezer recommendations download" 'd["mood"]["recommendationReady"] >= 1' "/moods/$DZ_MOOD"
expect "add a Deezer URL" "202" "POST" "/moods/$DZ_MOOD/songs" \ expect "add a Deezer URL" "202" "POST" "/moods/$DZ_MOOD/songs" \
'{"url":"https://www.deezer.com/track/66609426"}' '{"url":"https://www.deezer.com/track/55555555"}'
wait_for 90 "Deezer URL attaches to the mood" 'd["mood"]["songCount"] >= 2' "/moods/$DZ_MOOD" wait_for 90 "Deezer URL attaches to the mood" 'd["mood"]["songCount"] >= 2' "/moods/$DZ_MOOD"
# The collection changed, so the next top-up re-runs discovery against the
# artist graph instead of waiting for the refresh window.
DZ_STATS=""
for _ in $(seq 1 90); do
DZ_STATS="$(curl -sf "http://127.0.0.1:${DZ_API_PORT}/stats" || true)"
if [[ -n "$DZ_STATS" ]] && printf '%s' "$DZ_STATS" | python3 -c '
import json, sys
d = json.load(sys.stdin)
sys.exit(0 if d["artist_related"] > 0 and d["album_tracks"] > 0 else 1)
'; then
break
fi
sleep 1
done
expect_eq "Deezer discovery walks the artist graph" "true" \
"$(printf '%s' "$DZ_STATS" | python3 -c 'import json,sys;d=json.load(sys.stdin);print(str(d["artist_related"] > 0 and d["artist_top"] > 0).lower())')"
expect_eq "Deezer discovery reads album deep cuts" "true" \
"$(printf '%s' "$DZ_STATS" | python3 -c 'import json,sys;d=json.load(sys.stdin);print(str(d["album_tracks"] > 0).lower())')"
if [[ -f "$MEDIA/deezer-67238732/Instant Crush.mp3" ]]; then if [[ -f "$MEDIA/deezer-67238732/Instant Crush.mp3" ]]; then
pass "Deezer media path" pass "Deezer media path"
else else
@@ -105,6 +105,7 @@ fun main() {
indexer = indexer, indexer = indexer,
gc = gc, gc = gc,
tempDir = config.downloadDir, tempDir = config.downloadDir,
onSongAttached = topUpService::trigger,
) )
if (config.defaultSource == MusicSource.DEEZER && config.deezerArl == null) { if (config.defaultSource == MusicSource.DEEZER && config.deezerArl == null) {
log.warn("DEFAULT_SOURCE is deezer but DEEZER_ARL is empty: downloads will fail") log.warn("DEFAULT_SOURCE is deezer but DEEZER_ARL is empty: downloads will fail")
@@ -117,6 +117,14 @@ object MoodRepository {
?.let { moodRecord(it) } ?.let { moodRecord(it) }
} }
/** Owner-agnostic lookup for background work that only has a mood id. */
fun byId(id: String): MoodRecord? = transaction {
MoodsTable.selectAll()
.where { MoodsTable.id eq id }
.singleOrNull()
?.let { moodRecord(it) }
}
fun create( fun create(
owner: String, owner: String,
name: String, name: String,
@@ -590,7 +598,10 @@ object MoodRepository {
.map { moodRecord(it) } .map { moodRecord(it) }
.filter { mood -> .filter { mood ->
mood.recommendationCount > 0 && mood.recommendationCount > 0 &&
isStale(mood.lastDiscoveryAt, mood.refreshHours, now) (
isStale(mood.lastDiscoveryAt, mood.refreshHours, now) ||
songsAddedSince(mood.id, mood.lastDiscoveryAt)
)
} }
.take(limit.coerceAtLeast(1)) .take(limit.coerceAtLeast(1))
} }
@@ -613,7 +624,23 @@ object MoodRepository {
} }
fun isDiscoveryDue(mood: MoodRecord, now: Instant = Instant.now()): Boolean = fun isDiscoveryDue(mood: MoodRecord, now: Instant = Instant.now()): Boolean =
isStale(mood.lastDiscoveryAt, mood.refreshHours, now) isStale(mood.lastDiscoveryAt, mood.refreshHours, now) ||
songsAddedSince(mood.id, mood.lastDiscoveryAt)
/**
* The collection is the discovery seed, so a song added after the last pass
* makes the hidden candidate pool stale, even when every slot is filled.
* Both timestamps come from [Instant.toString], so the lexical comparison is
* also chronological.
*/
fun songsAddedSince(moodId: String, since: String?): Boolean {
if (since == null) return false
return transaction {
MoodSongsTable.selectAll()
.where { MoodSongsTable.moodId eq moodId }
.any { row -> row[MoodSongsTable.addedAt] > since }
}
}
// ------------------------------------------------------------ internal // ------------------------------------------------------------ internal
@@ -6,6 +6,7 @@ import java.net.http.HttpClient
import java.net.http.HttpRequest import java.net.http.HttpRequest
import java.net.http.HttpResponse import java.net.http.HttpResponse
import java.time.Duration import java.time.Duration
import java.util.concurrent.ConcurrentHashMap
import kotlinx.serialization.SerialName import kotlinx.serialization.SerialName
import kotlinx.serialization.Serializable import kotlinx.serialization.Serializable
import kotlinx.serialization.json.Json import kotlinx.serialization.json.Json
@@ -16,8 +17,8 @@ import kotlinx.serialization.json.jsonPrimitive
import win.taptap.musicmanager.source.MusicSource import win.taptap.musicmanager.source.MusicSource
/** /**
* Deezer's public catalog API. No user token is needed for search or track * Deezer's public catalog API. No user token is needed for search, track
* metadata; the ARL is only used by the downloader. * metadata or the similarity graph; the ARL is only used by the downloader.
* *
* Deezer returns canonical tracks with a square album cover, which is also what * Deezer returns canonical tracks with a square album cover, which is also what
* the indexer embeds into the downloaded MP3. * the indexer embeds into the downloaded MP3.
@@ -39,6 +40,10 @@ class DeezerSearch(
.followRedirects(HttpClient.Redirect.NORMAL) .followRedirects(HttpClient.Redirect.NORMAL)
.build() .build()
/** Artist and album lookups are stable for a release, so cache them. */
private val artistIds = ConcurrentHashMap<String, Lookup<Long>>()
private val albumRefs = ConcurrentHashMap<String, Lookup<DeezerAlbumRefDto>>()
override fun search(query: String, limit: Int): List<TrackCandidate> { override fun search(query: String, limit: Int): List<TrackCandidate> {
val count = limit.coerceIn(1, MAX_RESULTS) val count = limit.coerceIn(1, MAX_RESULTS)
val url = "$apiBase/search/track?q=${encode(query)}&limit=$count&output=json" val url = "$apiBase/search/track?q=${encode(query)}&limit=$count&output=json"
@@ -54,6 +59,147 @@ class DeezerSearch(
return parseDeezerTrack(get("$apiBase/track/$trackId")) return parseDeezerTrack(get("$apiBase/track/$trackId"))
} }
/**
* Walks Deezer's own graph from the entire collection:
*
* 1. the most represented artists → their top tracks;
* 2. those artists' related artists → their top tracks;
* 3. the collection's albums → their deep cuts.
*
* A mood with 40 songs by 6 artists therefore still yields fresh material
* even after the obvious artist hits are already in the mood. Every request
* is best-effort; a blocked endpoint does not fail the pass.
*/
override fun discover(seeds: DiscoverySeeds, limit: Int): List<TrackCandidate> {
if (seeds.isEmpty()) return emptyList()
val found = LinkedHashMap<String, TrackCandidate>()
val seedArtists = seeds.artists
.take(MAX_SEED_ARTISTS)
.mapNotNull { seed -> artistId(seed.name)?.let { seed to it } }
seedArtists.forEach { (_, id) ->
artistTopTracks(id, SEED_TOP_LIMIT).forEach {
add(found, it, SEED_TOP_AFFINITY)
}
}
val related = LinkedHashSet<Long>()
seedArtists.take(MAX_RELATED_SEEDS).forEach { (_, id) ->
related += relatedArtists(id, MAX_RELATED_PER_SEED)
}
val seedIds = seedArtists.map { it.second }.toSet()
related.asSequence()
.filterNot { it in seedIds }
.take(MAX_RELATED_ARTISTS)
.forEach { id ->
artistTopTracks(id, RELATED_TOP_LIMIT).forEach {
add(found, it, RELATED_AFFINITY)
}
}
seeds.albums
.take(MAX_SEED_ALBUMS)
.mapNotNull(::albumRef)
.forEach { album ->
albumTracks(album, ALBUM_TRACK_LIMIT).forEach {
add(found, it, ALBUM_AFFINITY)
}
}
return found.values.take(limit.coerceAtLeast(1))
}
/** Keeps the highest weight when the same track comes from two edges. */
private fun add(
into: MutableMap<String, TrackCandidate>,
candidate: TrackCandidate,
affinity: Double,
) {
if (candidate.externalId.isEmpty()) return
val existing = into[candidate.externalId]
if (existing == null || existing.affinity < affinity) {
into[candidate.externalId] = candidate.copy(affinity = affinity)
}
}
private fun artistId(name: String): Long? = lookup(
cache = artistIds,
key = matchKey(name),
request = "$apiBase/search/artist?q=${encode(name)}&limit=$ARTIST_LOOKUP_LIMIT&output=json",
) { body ->
val parsed = json.decodeFromString<DeezerArtistListResponse>(body)
parsed.data.firstOrNull { artistMatches(it.name, name) && it.id > 0 }?.id
?: parsed.data.firstOrNull { it.id > 0 }?.id
}
private fun albumRef(seed: SeedAlbum): DeezerAlbumRefDto? {
val titleKey = matchKey(seed.title)
if (titleKey.isEmpty()) return null
val artistKey = matchKey(seed.artist)
return lookup(
cache = albumRefs,
key = "$titleKey|$artistKey",
request = "$apiBase/search/album?q=${encode("${seed.title} ${seed.artist}")}" +
"&limit=$ALBUM_LOOKUP_LIMIT&output=json",
) { body ->
val parsed = json.decodeFromString<DeezerAlbumListResponse>(body)
parsed.data.firstOrNull { album ->
album.id > 0 &&
matchKey(album.title) == titleKey &&
(album.artist == null || artistMatches(album.artist.name, seed.artist))
}
}
}
/**
* Resolves a name to an id once. Misses are cached too, so a mood full of
* obscure artists does not re-search them on every refresh.
*/
private inline fun <T : Any> lookup(
cache: MutableMap<String, Lookup<T>>,
key: String,
request: String,
crossinline parse: (String) -> T?,
): T? {
if (key.isEmpty()) return null
cache[key]?.let { return it.value }
val value = runCatching { parse(get(request)) }.getOrNull()
cache[key] = Lookup(value)
return value
}
private fun relatedArtists(id: Long, limit: Int): List<Long> = runCatching {
val body = get("$apiBase/artist/$id/related?limit=$limit&output=json")
json.decodeFromString<DeezerArtistListResponse>(body)
.data
.map { it.id }
.filter { it > 0 }
}.getOrDefault(emptyList())
private fun artistTopTracks(id: Long, limit: Int): List<TrackCandidate> = runCatching {
val body = get("$apiBase/artist/$id/top?limit=$limit&output=json")
json.decodeFromString<DeezerTracksResponse>(body).data.mapNotNull { it.toCandidate() }
}.getOrDefault(emptyList())
/**
* `/album/{id}/tracks` omits the album object, so inject the album the
* collection already gave us to keep the cover and the album name.
*/
private fun albumTracks(
album: DeezerAlbumRefDto,
limit: Int,
): List<TrackCandidate> = runCatching {
val body = get("$apiBase/album/${album.id}/tracks?limit=$limit&output=json")
json.decodeFromString<DeezerTracksResponse>(body).data
.mapNotNull { it.toCandidate() }
.map { candidate ->
candidate.copy(
album = candidate.album ?: album.title.takeIf { it.isNotBlank() },
thumbnail = candidate.thumbnail ?: album.cover(),
)
}
}.getOrDefault(emptyList())
private fun get(url: String): String { private fun get(url: String): String {
throttle() throttle()
val request = HttpRequest.newBuilder(URI.create(url)) val request = HttpRequest.newBuilder(URI.create(url))
@@ -111,8 +257,31 @@ class DeezerSearch(
} }
} }
/** `value == null` is a cached miss. */
private data class Lookup<T>(val value: T?)
private companion object { private companion object {
const val MAX_RESULTS = 25 const val MAX_RESULTS = 25
/** Bounds one discovery pass to roughly twenty throttled requests. */
const val MAX_SEED_ARTISTS = 5
const val MAX_RELATED_SEEDS = 2
const val MAX_RELATED_PER_SEED = 8
const val MAX_RELATED_ARTISTS = 6
const val MAX_SEED_ALBUMS = 2
const val SEED_TOP_LIMIT = 25
const val RELATED_TOP_LIMIT = 10
const val ALBUM_TRACK_LIMIT = 40
const val ARTIST_LOOKUP_LIMIT = 5
const val ALBUM_LOOKUP_LIMIT = 5
/** A hit by an artist already in the mood; the scorer adds the rest. */
const val SEED_TOP_AFFINITY = 0.5
/** A hit by an artist Deezer considers similar to the collection. */
const val RELATED_AFFINITY = 1.5
/** Another track from an album already in the mood. */
const val ALBUM_AFFINITY = 1.0
const val MIN_REQUEST_INTERVAL_NANOS = 120_000_000L const val MIN_REQUEST_INTERVAL_NANOS = 120_000_000L
val THROTTLE_LOCK = Any() val THROTTLE_LOCK = Any()
@@ -152,6 +321,29 @@ internal fun parseDeezerTrack(json: String): TrackCandidate? {
@Serializable @Serializable
private data class DeezerSearchResponse(val data: List<DeezerTrackDto> = emptyList()) private data class DeezerSearchResponse(val data: List<DeezerTrackDto> = emptyList())
@Serializable
private data class DeezerTracksResponse(val data: List<DeezerTrackDto> = emptyList())
@Serializable
private data class DeezerArtistListResponse(val data: List<DeezerArtistRefDto> = emptyList())
@Serializable
private data class DeezerArtistRefDto(val id: Long = 0, val name: String = "")
@Serializable
private data class DeezerAlbumListResponse(val data: List<DeezerAlbumRefDto> = emptyList())
@Serializable
private data class DeezerAlbumRefDto(
val id: Long = 0,
val title: String = "",
val artist: DeezerArtistDto? = null,
@SerialName("cover_big") val coverBig: String? = null,
@SerialName("cover_xl") val coverXl: String? = null,
) {
fun cover(): String? = coverBig ?: coverXl
}
@Serializable @Serializable
private data class DeezerTrackDto( private data class DeezerTrackDto(
val id: Long = 0, val id: Long = 0,
@@ -10,6 +10,7 @@ import org.slf4j.LoggerFactory
import win.taptap.musicmanager.db.MoodRecord import win.taptap.musicmanager.db.MoodRecord
import win.taptap.musicmanager.db.MoodRepository import win.taptap.musicmanager.db.MoodRepository
import win.taptap.musicmanager.db.RecommendationUpsert import win.taptap.musicmanager.db.RecommendationUpsert
import win.taptap.musicmanager.db.SongRecord
import win.taptap.musicmanager.util.IdGenerator import win.taptap.musicmanager.util.IdGenerator
data class DiscoveryOutcome( data class DiscoveryOutcome(
@@ -39,24 +40,54 @@ class DiscoveryService(
val catalog = catalogs.forSource(mood.source) val catalog = catalogs.forSource(mood.source)
val collection = MoodRepository.songRecords(mood.id) val collection = MoodRepository.songRecords(mood.id)
val seedArtists = collection.map { it.artist }.filter { it.isNotBlank() } val artists = artistsByFrequency(collection)
val queries = buildQueries(mood, seedArtists) val albums = albumsOf(collection)
if (queries.isEmpty()) { val seeds = DiscoverySeeds(
MoodRepository.markDiscovered(mood.id) artists = artists,
return DiscoveryOutcome(0) albums = albums,
hint = mood.query,
genres = mood.genres,
name = mood.name,
)
val queries = buildQueries(mood, artists.map { it.name })
if (queries.isEmpty() && seeds.isEmpty()) {
// A new mood with no hint, no genres and no songs has nothing to
// base music on. Leave the slots empty instead of downloading
// whatever the mood's name happens to match.
log.debug("Skipping discovery for mood {}: no music signal yet", mood.name)
return DiscoveryOutcome(0, skipped = true)
} }
// Queries are independent; running them together keeps a refresh within // Text queries and the catalog's similarity graph run together: the
// the client's request timeout even when a catalog is slow. // graph knows about related artists and album deep cuts, which a title
// search cannot express.
val results = coroutineScope { val results = coroutineScope {
queries.map { query -> val searches = queries.map { query ->
async(Dispatchers.IO) { async(Dispatchers.IO) { runCatching { catalog.search(query, QUERY_LIMIT) } }
runCatching { catalog.search(query, QUERY_LIMIT) }
} }
}.awaitAll() val expansion = async(Dispatchers.IO) {
if (seeds.isEmpty()) {
emptyList()
} else {
runCatching { catalog.discover(seeds, DISCOVERY_LIMIT) }
.onFailure {
log.debug(
"{} expansion failed for mood {}: {}",
catalog.source,
mood.name,
it.message,
)
} }
val failures = results.filter { it.isFailure } .getOrDefault(emptyList())
if (failures.size == queries.size) { }
}
searches.awaitAll() to expansion.await()
}
val searches = results.first
val discovered = results.second
val failures = searches.filter { it.isFailure }
if (searches.isNotEmpty() && failures.size == searches.size && discovered.isEmpty()) {
val message = failures.firstNotNullOfOrNull { it.exceptionOrNull()?.message } val message = failures.firstNotNullOfOrNull { it.exceptionOrNull()?.message }
?: "Search failed" ?: "Search failed"
log.warn("{} discovery failed for mood {}: {}", catalog.source, mood.name, message) log.warn("{} discovery failed for mood {}: {}", catalog.source, mood.name, message)
@@ -65,9 +96,11 @@ class DiscoveryService(
} }
val candidates = LinkedHashMap<String, TrackCandidate>() val candidates = LinkedHashMap<String, TrackCandidate>()
results.mapNotNull { it.getOrNull() }.flatten().forEach { candidate -> (searches.mapNotNull { it.getOrNull() }.flatten() + discovered).forEach { candidate ->
if (isMusicLength(candidate.duration)) { if (!isMusicLength(candidate.duration)) return@forEach
candidates.putIfAbsent(candidate.externalId, candidate) val existing = candidates[candidate.externalId]
if (existing == null || existing.affinity < candidate.affinity) {
candidates[candidate.externalId] = candidate
} }
} }
@@ -85,18 +118,23 @@ class DiscoveryService(
.toSet() .toSet()
val moodText = listOfNotNull(mood.query, mood.description, mood.name).joinToString(" ") val moodText = listOfNotNull(mood.query, mood.description, mood.name).joinToString(" ")
val artistWeights = artists
.associate { matchKey(it.name) to it.songs.toDouble() }
.filterKeys { it.isNotEmpty() }
val albumKeys = albums
.map { matchKey(it.title) }
.filter { it.isNotEmpty() }
.toSet()
val upserts = candidates.values val ranked = rankCandidates(
candidates.values
.filterNot { it.externalId in excluded } .filterNot { it.externalId in excluded }
.map { candidate -> .map { candidate ->
candidate to scoreCandidate(candidate, seedArtists, moodText) candidate to scoreCandidate(candidate, artistWeights, albumKeys, moodText)
} },
.sortedWith( CANDIDATE_POOL,
compareByDescending<Pair<TrackCandidate, Double>> { it.second }
.thenBy { it.first.title },
) )
.take(CANDIDATE_POOL) val upserts = ranked.map { candidate ->
.map { (candidate, _) ->
RecommendationUpsert( RecommendationUpsert(
source = mood.source, source = mood.source,
externalId = candidate.externalId, externalId = candidate.externalId,
@@ -121,12 +159,22 @@ class DiscoveryService(
CANDIDATE_POOL, CANDIDATE_POOL,
) )
MoodRepository.markDiscovered(mood.id) MoodRepository.markDiscovered(mood.id)
log.info("{} discovery for mood {} stored {} candidates", catalog.source, mood.name, stored) log.info(
"{} discovery for mood {} stored {} candidates from {} searches and {} expanded hits",
catalog.source,
mood.name,
stored,
queries.size,
discovered.size,
)
return DiscoveryOutcome(stored = stored) return DiscoveryOutcome(stored = stored)
} }
private companion object { private companion object {
const val QUERY_LIMIT = 10 const val QUERY_LIMIT = 10
/** The graph expands far more than a single search page does. */
const val DISCOVERY_LIMIT = 200
const val CANDIDATE_POOL = 60 const val CANDIDATE_POOL = 60
} }
} }
@@ -144,28 +192,96 @@ internal fun isMusicLength(seconds: Int?): Boolean =
internal fun buildQueries(mood: MoodRecord, seedArtists: List<String>): List<String> { internal fun buildQueries(mood: MoodRecord, seedArtists: List<String>): List<String> {
val queries = LinkedHashSet<String>() val queries = LinkedHashSet<String>()
mood.query?.trim()?.takeIf { it.isNotEmpty() }?.let(queries::add) mood.query?.trim()?.takeIf { it.isNotEmpty() }?.let(queries::add)
seedArtists.map { it.trim() }.filter { it.isNotEmpty() }.distinct().take(2).forEach { // The collection is the strongest signal, so its artists fill the slots
// before genres do. They arrive ordered by how many songs the mood holds.
seedArtists.map { it.trim() }.filter { it.isNotEmpty() }.distinct().take(3).forEach {
queries += it queries += it
} }
mood.genres.map { it.trim() }.filter { it.isNotEmpty() }.take(2).forEach { mood.genres.map { it.trim() }.filter { it.isNotEmpty() }.take(2).forEach {
queries += "$it mix" queries += "$it mix"
} }
if (queries.isEmpty()) queries += mood.name
return queries.distinctBy { it.lowercase() }.take(4) return queries.distinctBy { it.lowercase() }.take(4)
} }
/** Collection artists, most represented first. */
internal fun artistsByFrequency(collection: List<SongRecord>): List<SeedArtist> =
collection
.map { it.artist.trim() }
.filter { it.isNotEmpty() }
.groupingBy { it }
.eachCount()
.entries
.sortedWith(
compareByDescending<Map.Entry<String, Int>> { it.value }.thenBy { it.key.lowercase() },
)
.map { SeedArtist(it.key, it.value) }
/** Distinct albums of the collection, deduplicated loosely. */
internal fun albumsOf(collection: List<SongRecord>): List<SeedAlbum> =
collection.mapNotNull { song ->
val album = song.album?.trim()?.takeIf { it.isNotEmpty() } ?: return@mapNotNull null
SeedAlbum(album, song.artist)
}.distinctBy { "${matchKey(it.title)}|${matchKey(it.artist)}" }
/**
* Picks the best [limit] candidates while penalising an artist that keeps
* winning: without it, a mood with three songs by one artist would fill every
* slot with that artist. Each additional pick by the same artist costs
* [ARTIST_REPEAT_PENALTY], so a related artist eventually takes a slot.
*/
internal fun rankCandidates(
scored: List<Pair<TrackCandidate, Double>>,
limit: Int,
): List<TrackCandidate> {
val remaining = scored.toMutableList()
val chosen = ArrayList<TrackCandidate>(limit.coerceAtMost(remaining.size))
val perArtist = HashMap<String, Int>()
while (chosen.size < limit && remaining.isNotEmpty()) {
var bestIndex = 0
var bestScore = Double.NEGATIVE_INFINITY
for (index in remaining.indices) {
val candidate = remaining[index].first
val picked = perArtist[matchKey(candidate.artist)] ?: 0
val adjusted = remaining[index].second - ARTIST_REPEAT_PENALTY * picked
if (adjusted > bestScore) {
bestScore = adjusted
bestIndex = index
}
}
val candidate = remaining.removeAt(bestIndex).first
chosen += candidate
val key = matchKey(candidate.artist)
perArtist[key] = (perArtist[key] ?: 0) + 1
}
return chosen
}
/**
* Ranks a candidate against the whole collection:
*
* - an artist already in the mood is the strongest signal, and an artist with
* more songs in the mood ranks higher;
* - another track from an album already in the mood is a strong signal;
* - a catalog-supplied [TrackCandidate.affinity] (related artist, top track)
* is added as-is;
* - the mood's own words in the title are a tie-breaker.
*/
internal fun scoreCandidate( internal fun scoreCandidate(
candidate: TrackCandidate, candidate: TrackCandidate,
seedArtists: List<String>, artistWeights: Map<String, Double>,
albumKeys: Set<String>,
moodText: String, moodText: String,
): Double { ): Double {
var score = 0.0 var score = candidate.affinity
val artist = matchKey(candidate.artist) val artist = matchKey(candidate.artist)
val seedKeys = seedArtists.map { matchKey(it) }.filter { it.isNotEmpty() } val exact = artistWeights[artist]
when { when {
artist.isNotEmpty() && artist in seedKeys -> score += 3.0 exact != null -> score += 3.0 + (0.5 * (exact - 1)).coerceAtMost(1.5)
artist.isNotEmpty() && seedKeys.any { artist.contains(it) || it.contains(artist) } -> score += 1.5 artist.isNotEmpty() && artistWeights.keys.any { artistMatches(artist, it) } -> score += 1.5
} }
val album = matchKey(candidate.album.orEmpty())
if (album.isNotEmpty() && album in albumKeys) score += 1.0
val text = matchKey(moodText) val text = matchKey(moodText)
if (text.isNotEmpty() && matchKey(candidate.title).contains(text)) score += 1.0 if (text.isNotEmpty() && matchKey(candidate.title).contains(text)) score += 1.0
if ((candidate.duration ?: 0) > 8 * 60) score -= 1.0 if ((candidate.duration ?: 0) > 8 * 60) score -= 1.0
@@ -198,3 +314,4 @@ internal fun artistMatches(a: String, b: String): Boolean {
internal const val MIN_TRACK_SECONDS = 30 internal const val MIN_TRACK_SECONDS = 30
internal const val MAX_TRACK_SECONDS = 600 internal const val MAX_TRACK_SECONDS = 600
internal const val ARTIST_REPEAT_PENALTY = 1.0
@@ -2,7 +2,13 @@ package win.taptap.musicmanager.discovery
import win.taptap.musicmanager.source.MusicSource import win.taptap.musicmanager.source.MusicSource
/** One source-specific search hit, already split into an artist and a title. */ /**
* One source-specific search hit, already split into an artist and a title.
*
* [affinity] is a catalog-supplied similarity weight: the catalog knows where a
* hit came from (a related artist, an album deep cut), which is information a
* title search cannot express.
*/
data class TrackCandidate( data class TrackCandidate(
val source: String, val source: String,
val externalId: String, val externalId: String,
@@ -11,8 +17,29 @@ data class TrackCandidate(
val album: String? = null, val album: String? = null,
val duration: Int? = null, val duration: Int? = null,
val thumbnail: String? = null, val thumbnail: String? = null,
val affinity: Double = 0.0,
) )
/** One artist of the mood collection and how many of its songs the mood holds. */
data class SeedArtist(val name: String, val songs: Int)
/** One album of the mood collection. */
data class SeedAlbum(val title: String, val artist: String)
/**
* The whole mood collection, handed to a catalog that can expand it through its
* own similarity graph instead of title searches alone.
*/
data class DiscoverySeeds(
val artists: List<SeedArtist> = emptyList(),
val albums: List<SeedAlbum> = emptyList(),
val hint: String? = null,
val genres: List<String> = emptyList(),
val name: String = "",
) {
fun isEmpty(): Boolean = artists.isEmpty() && albums.isEmpty()
}
/** A music catalog that can be searched and can resolve a pasted URL. */ /** A music catalog that can be searched and can resolve a pasted URL. */
interface MusicCatalog { interface MusicCatalog {
val source: String val source: String
@@ -21,6 +48,16 @@ interface MusicCatalog {
/** Resolves a pasted track URL, or returns null when it is not a track. */ /** Resolves a pasted track URL, or returns null when it is not a track. */
fun resolve(url: String): TrackCandidate? fun resolve(url: String): TrackCandidate?
/**
* Expands the whole collection through the catalog's own similarity graph:
* related artists, artist top tracks and album deep cuts. Catalogs without
* such an API keep the default empty list and rely on text queries.
*
* Implementations must be best-effort: one failing sub-request returns what
* the other requests found instead of failing the whole discovery pass.
*/
fun discover(seeds: DiscoverySeeds, limit: Int): List<TrackCandidate> = emptyList()
} }
/** /**
@@ -38,6 +38,19 @@ class TopUpService(
} }
} }
/**
* Same, for work that only knows the id: a finished download attaches a song
* and changes the collection the next discovery pass is based on.
*/
fun trigger(moodId: String) {
val scope = backgroundScope ?: return
scope.launch(Dispatchers.IO) {
val mood = MoodRepository.byId(moodId) ?: return@launch
runCatching { topUp(mood, force = true) }
.onFailure { log.warn("Recommendation top-up for mood {} failed", mood.name, it) }
}
}
suspend fun topUp(mood: MoodRecord, force: Boolean = false): TopUpResult = mutex.withLock { suspend fun topUp(mood: MoodRecord, force: Boolean = false): TopUpResult = mutex.withLock {
var current = MoodRepository.get(mood.id, mood.owner) ?: return@withLock TopUpResult(0) var current = MoodRepository.get(mood.id, mood.owner) ?: return@withLock TopUpResult(0)
if (current.recommendationCount <= 0) return@withLock TopUpResult(0) if (current.recommendationCount <= 0) return@withLock TopUpResult(0)
@@ -46,7 +59,10 @@ class TopUpService(
val stillMissing = current.recommendationCount - MoodRepository.activeCount(current.id) val stillMissing = current.recommendationCount - MoodRepository.activeCount(current.id)
var discoveryError: String? = null var discoveryError: String? = null
if (stillMissing > 0 && (force || MoodRepository.isDiscoveryDue(current))) { val collectionChanged = MoodRepository.songsAddedSince(current.id, current.lastDiscoveryAt)
if ((stillMissing > 0 || collectionChanged) &&
(force || MoodRepository.isDiscoveryDue(current))
) {
val outcome = discovery.refresh(current, force) val outcome = discovery.refresh(current, force)
discoveryError = outcome.error discoveryError = outcome.error
current = MoodRepository.get(mood.id, mood.owner) ?: current current = MoodRepository.get(mood.id, mood.owner) ?: current
@@ -37,6 +37,12 @@ class DownloadService(
private val tagExtractor: Mp3TagExtractor = Mp3TagExtractor(), private val tagExtractor: Mp3TagExtractor = Mp3TagExtractor(),
private val tempDir: Path, private val tempDir: Path,
private val maxAttempts: Int = 3, private val maxAttempts: Int = 3,
/**
* Called after a manual download becomes a permanent mood song. The mood
* collection is the discovery seed, so the recommendation pool has to be
* refreshed instead of waiting for the hourly worker.
*/
private val onSongAttached: (String) -> Unit = {},
) { ) {
private val log = LoggerFactory.getLogger(DownloadService::class.java) private val log = LoggerFactory.getLogger(DownloadService::class.java)
@@ -119,6 +125,8 @@ class DownloadService(
DownloadRepository.KIND_SONG -> { DownloadRepository.KIND_SONG -> {
MoodRepository.addSong(job.moodId, song.id, "search") MoodRepository.addSong(job.moodId, song.id, "search")
DownloadRepository.markDone(job.id, song.id) DownloadRepository.markDone(job.id, song.id)
runCatching { onSongAttached(job.moodId) }
.onFailure { log.warn("Mood {} top-up failed", job.moodId, it) }
} }
else -> { else -> {
when ( when (
@@ -179,6 +179,28 @@ class MoodRepositoryTest {
} }
} }
@Test
fun `adding a song makes discovery due again`() {
withDatabase { mediaStore, _ ->
val mood = createMood(recommendationCount = 2)
assertTrue(MoodRepository.isDiscoveryDue(mood), "A never-seen mood is due")
MoodRepository.markDiscovered(mood.id)
val fresh = assertNotNull(MoodRepository.get(mood.id, "alice"))
assertFalse(MoodRepository.isDiscoveryDue(fresh), "Nothing changed since the pass")
// Timestamps are compared strictly; keep them distinguishable.
Thread.sleep(5)
val song = insertSong(mediaStore, MusicSource.YOUTUBE, "vid-1", "One")
MoodRepository.addSong(mood.id, song.id, "search")
val changed = assertNotNull(MoodRepository.get(mood.id, "alice"))
assertTrue(
MoodRepository.isDiscoveryDue(changed),
"The collection is the discovery seed, so the pool is stale",
)
}
}
@Test @Test
fun `download completion after a rejection does not attach the song`() { fun `download completion after a rejection does not attach the song`() {
withDatabase { mediaStore, _ -> withDatabase { mediaStore, _ ->
@@ -0,0 +1,137 @@
package win.taptap.musicmanager.discovery
import com.sun.net.httpserver.HttpServer
import java.net.InetSocketAddress
import java.util.concurrent.CopyOnWriteArrayList
import kotlin.test.Test
import kotlin.test.assertEquals
import kotlin.test.assertTrue
/**
* Exercises the Deezer similarity walk against a local stand-in API, so the
* request shapes and the affinity weights stay pinned without network access.
*/
class DeezerDiscoverTest {
@Test
fun `expands the collection through related artists and album deep cuts`() {
val requests = CopyOnWriteArrayList<String>()
val server = serve { path ->
requests += path
when (path) {
"/search/artist" ->
ok("""{"data":[{"id":27,"name":"Daft Punk"}]}""")
"/search/album" ->
ok(
"""{"data":[{"id":5,"title":"Random Access Memories",
"artist":{"name":"Daft Punk"}}]}""",
)
"/artist/27/top" ->
ok(
"""{"data":[
{"id":1,"title":"Instant Crush","duration":337,"readable":true,
"artist":{"name":"Daft Punk"},
"album":{"title":"Random Access Memories"}},
{"id":2,"title":"Too Long","duration":900,"readable":true,
"artist":{"name":"Daft Punk"},
"album":{"title":"Random Access Memories"}}]}""",
)
"/artist/27/related" ->
ok("""{"data":[{"id":99,"name":"Daft Punk Tribute"}]}""")
"/artist/99/top" ->
ok(
"""{"data":[
{"id":3,"title":"Robot Rock","duration":300,"readable":true,
"artist":{"name":"Daft Punk Tribute"}}]}""",
)
"/album/5/tracks" ->
ok(
"""{"data":[
{"id":4,"title":"Contact","duration":382,"readable":true,
"artist":{"name":"Daft Punk"},
"album":{"title":"Random Access Memories"}}]}""",
)
else -> 404 to """{"error":{"type":"DataException","message":"not found"}}"""
}
}
try {
val found = catalog(server).discover(seeds(), limit = 50).associateBy { it.externalId }
// The 15-minute track is not music-length and never reaches the pool.
assertEquals(setOf("1", "3", "4"), found.keys)
assertEquals(0.5, found.getValue("1").affinity)
assertEquals(1.5, found.getValue("3").affinity)
assertEquals(1.0, found.getValue("4").affinity)
// `/album/{id}/tracks` has no album object; the search result fills it in.
assertEquals("Random Access Memories", found.getValue("4").album)
assertTrue("/artist/27/related" in requests)
assertTrue("/artist/99/top" in requests)
assertTrue("/album/5/tracks" in requests)
} finally {
server.stop(0)
}
}
@Test
fun `a failing endpoint does not fail the whole expansion`() {
val server = serve { path ->
when (path) {
"/search/artist" -> ok("""{"data":[{"id":27,"name":"Daft Punk"}]}""")
"/artist/27/top" ->
ok(
"""{"data":[
{"id":1,"title":"Instant Crush","duration":337,"readable":true,
"artist":{"name":"Daft Punk"}}]}""",
)
else -> 500 to """{"error":{"type":"DataException","message":"boom"}}"""
}
}
try {
val found = catalog(server).discover(seeds(), limit = 50)
assertEquals(listOf("1"), found.map { it.externalId })
} finally {
server.stop(0)
}
}
@Test
fun `an empty collection makes no requests`() {
val requests = CopyOnWriteArrayList<String>()
val server = serve { path ->
requests += path
404 to """{"error":{"type":"DataException","message":"not found"}}"""
}
try {
assertTrue(catalog(server).discover(DiscoverySeeds(), limit = 50).isEmpty())
assertTrue(requests.isEmpty())
} finally {
server.stop(0)
}
}
private fun seeds() = DiscoverySeeds(
artists = listOf(SeedArtist("Daft Punk", 4)),
albums = listOf(SeedAlbum("Random Access Memories", "Daft Punk")),
)
private fun catalog(server: HttpServer) = DeezerSearch(
apiBase = "http://127.0.0.1:${server.address.port}",
timeoutSeconds = 5,
)
private fun ok(body: String) = 200 to body
private fun serve(route: (String) -> Pair<Int, String>): HttpServer {
val server = HttpServer.create(InetSocketAddress("127.0.0.1", 0), 0)
server.createContext("/") { exchange ->
val (status, body) = route(exchange.requestURI.path)
val bytes = body.toByteArray()
exchange.responseHeaders.add("Content-Type", "application/json")
exchange.sendResponseHeaders(status, bytes.size.toLong())
exchange.responseBody.use { it.write(bytes) }
}
server.start()
return server
}
}
@@ -6,6 +6,7 @@ import kotlin.test.assertEquals
import kotlin.test.assertFalse import kotlin.test.assertFalse
import kotlin.test.assertTrue import kotlin.test.assertTrue
import win.taptap.musicmanager.db.MoodRecord import win.taptap.musicmanager.db.MoodRecord
import win.taptap.musicmanager.db.SongRecord
import win.taptap.musicmanager.source.MusicSource import win.taptap.musicmanager.source.MusicSource
class DiscoveryServiceTest { class DiscoveryServiceTest {
@@ -209,8 +210,8 @@ class DiscoveryServiceTest {
} }
@Test @Test
fun `queries fall back to the mood name`() { fun `a mood without a hint genres or songs has no query`() {
assertEquals(listOf("Focus"), buildQueries(mood(), emptyList())) assertTrue(buildQueries(mood(), emptyList()).isEmpty())
} }
@Test @Test
@@ -223,12 +224,167 @@ class DiscoveryServiceTest {
duration = 200, duration = 200,
) )
val plain = scoreCandidate(candidate, listOf("Alpha"), "focus") val plain = scoreCandidate(candidate, mapOf("alpha" to 1.0), emptySet(), "focus")
val seedArtist = scoreCandidate(candidate, listOf("Beta"), "focus") val seedArtist = scoreCandidate(candidate, mapOf("beta" to 1.0), emptySet(), "focus")
assertTrue(seedArtist > plain) assertTrue(seedArtist > plain)
} }
@Test
fun `an artist with more songs in the mood ranks higher`() {
val candidate = TrackCandidate(
source = MusicSource.DEEZER,
externalId = "1",
title = "Focus Two",
artist = "Beta",
duration = 200,
)
val once = scoreCandidate(candidate, mapOf("beta" to 1.0), emptySet(), "")
val often = scoreCandidate(candidate, mapOf("beta" to 12.0), emptySet(), "")
assertTrue(often > once)
}
@Test
fun `album deep cuts outrank tracks from unrelated albums`() {
val candidate = TrackCandidate(
source = MusicSource.DEEZER,
externalId = "1",
title = "Contact",
artist = "Daft Punk",
album = "Random Access Memories",
duration = 382,
)
val unrelated = scoreCandidate(candidate, emptyMap(), setOf("homework"), "")
val sameAlbum = scoreCandidate(
candidate,
emptyMap(),
setOf(matchKey("Random Access Memories")),
"",
)
assertEquals(1.0, sameAlbum - unrelated)
}
@Test
fun `ranking spreads picks across artists instead of repeating one`() {
val lead = (1..12).map {
TrackCandidate(
source = MusicSource.DEEZER,
externalId = "lead-$it",
title = "Lead $it",
artist = "Alpha",
duration = 200,
)
}
val related = (1..6).map {
TrackCandidate(
source = MusicSource.DEEZER,
externalId = "related-$it",
title = "Related $it",
artist = "Beta",
duration = 200,
affinity = 1.5,
)
}
val scored = (lead + related).map {
it to scoreCandidate(it, mapOf("alpha" to 1.0), emptySet(), "")
}
val ranked = rankCandidates(scored, limit = 8)
assertEquals(8, ranked.size)
assertTrue(ranked.any { it.artist == "Beta" }, "A related artist must take a slot")
assertTrue(
ranked.count { it.artist == "Alpha" } <= 5,
"The lead artist must not fill every slot",
)
}
@Test
fun `catalog affinity raises a related artist hit`() {
val candidate = TrackCandidate(
source = MusicSource.DEEZER,
externalId = "1",
title = "Robot Rock",
artist = "Daft Punk Tribute",
duration = 300,
)
val plain = scoreCandidate(candidate, emptyMap(), emptySet(), "")
val related = scoreCandidate(
candidate.copy(affinity = 1.5),
emptyMap(),
emptySet(),
"",
)
assertEquals(1.5, related - plain)
}
@Test
fun `queries prefer collection artists over genres`() {
val mood = mood(query = "late night", genres = listOf("Ambient", "Jazz"))
val queries = buildQueries(mood, listOf("Alpha", "Beta", "Gamma", "Delta"))
assertEquals(listOf("late night", "Alpha", "Beta", "Gamma"), queries)
}
@Test
fun `collection artists are ordered by how many songs the mood has`() {
val collection = listOf(
song("Alpha"),
song("Beta"),
song("Alpha"),
song("Beta"),
song("Alpha"),
)
assertEquals(
listOf(SeedArtist("Alpha", 3), SeedArtist("Beta", 2)),
artistsByFrequency(collection),
)
}
@Test
fun `albums are collected once per title and artist`() {
val collection = listOf(
song("Daft Punk", album = "Random Access Memories"),
song("Daft Punk", album = "Random Access Memories"),
song("Daft Punk", album = "Homework"),
song("Other", album = null),
)
assertEquals(
listOf(
SeedAlbum("Random Access Memories", "Daft Punk"),
SeedAlbum("Homework", "Daft Punk"),
),
albumsOf(collection),
)
}
private fun song(artist: String, album: String? = "Album") = SongRecord(
id = "sg-$artist-$album",
source = MusicSource.DEEZER,
externalId = "1",
title = "Title",
artist = artist,
album = album,
duration = 180,
bitRate = 128,
size = 1,
suffix = "mp3",
contentType = "audio/mpeg",
relativePath = "deezer-1/Title.mp3",
fileMtime = 0,
thumbnail = null,
createdAt = "2026-01-01T00:00:00Z",
updatedAt = "2026-01-01T00:00:00Z",
)
private fun mood( private fun mood(
query: String? = null, query: String? = null,
genres: List<String> = emptyList(), genres: List<String> = emptyList(),