feat: discover YT recommendations and tune the radio
CI / chart (push) Successful in 26s
CI / chart-push (push) Successful in 5s
CI / test (push) Successful in 2m6s
CI / image (push) Failing after 2m54s

yt-dlp now feeds mood discovery. Queries come from the free-text hint, seed
artists and genres; they run in parallel, are filtered to music-length videos,
matched against library songs by normalized title and artist, scored, and
stored as review candidates. Refresh, accept and reject endpoints are added:
accepting a library match adds a seed immediately, while accepted tracks that
are not owned wait for the future downloader. Decisions survive refreshes and
stale undecided candidates are pruned.

A background pass refreshes stale moods every hour, and the image ships a
pinned yt-dlp binary (YTDLP_PATH, also added to the Nix shell and chart).

Radio now avoids tracks played in the last 7 days when the pool is large
enough, spaces artists and albums across a batch, and weighs skips three times
as heavily as completions. The chart moves to 0.2.2.
This commit is contained in:
TapTap committed 2026-11-15 12:44:30 +00:00
1 parent 67eed8d262
commit c5aa000d1a
18 files changed
+1061 -37

No files matched your search

+8 -1
View File
@@ -12,9 +12,15 @@ RUN gradle --no-daemon shadowJar
# glibc (not Alpine/musl): sqlite-jdbc ships no native library for Linux-Musl. # glibc (not Alpine/musl): sqlite-jdbc ships no native library for Linux-Musl.
FROM eclipse-temurin:21-jre-jammy FROM eclipse-temurin:21-jre-jammy
ARG YTDLP_VERSION=2026.10.26
RUN apt-get update \ RUN apt-get update \
&& apt-get install -y --no-install-recommends ffmpeg \ && apt-get install -y --no-install-recommends ca-certificates curl ffmpeg \
&& rm -rf /var/lib/apt/lists/* \ && rm -rf /var/lib/apt/lists/* \
&& curl -fsSL "https://github.com/yt-dlp/yt-dlp/releases/download/${YTDLP_VERSION}/yt-dlp" \
-o /usr/local/bin/yt-dlp \
&& chmod 0755 /usr/local/bin/yt-dlp \
&& yt-dlp --version \
&& groupadd -g 10001 app \ && groupadd -g 10001 app \
&& useradd -u 10001 -g app -M -s /usr/sbin/nologin app && useradd -u 10001 -g app -M -s /usr/sbin/nologin app
@@ -25,6 +31,7 @@ ENV DATA_DIR=/data \
LIBRARY_CONFIG=/etc/musicmanager/library.yaml \ LIBRARY_CONFIG=/etc/musicmanager/library.yaml \
USERS_FILE=/etc/musicmanager/users.yaml \ USERS_FILE=/etc/musicmanager/users.yaml \
FFMPEG_PATH=ffmpeg \ FFMPEG_PATH=ffmpeg \
YTDLP_PATH=yt-dlp \
PORT=8080 PORT=8080
USER 10001:10001 USER 10001:10001
+14 -5
View File
@@ -109,6 +109,7 @@ Passwords are read into memory at startup and never written to SQLite. In produc
| `LIBRARY_CONFIG` | `./config/library.yaml` | Library folders | | `LIBRARY_CONFIG` | `./config/library.yaml` | Library folders |
| `USERS_FILE` | `./config/users.yaml` | Users and passwords | | `USERS_FILE` | `./config/users.yaml` | Users and passwords |
| `FFMPEG_PATH` | `ffmpeg` | Transcoder binary | | `FFMPEG_PATH` | `ffmpeg` | Transcoder binary |
| `YTDLP_PATH` | `yt-dlp` | `yt-dlp` binary used for YT Music discovery |
## The mood model ## The mood model
@@ -117,8 +118,8 @@ A mood is a private collection owned by one user:
- **Seeds** are permanent anchors. Every seed is always eligible for the radio. - **Seeds** are permanent anchors. Every seed is always eligible for the radio.
- **Genres / free-text hint** widen the candidate pool. - **Genres / free-text hint** widen the candidate pool.
- **Target size** is the desired batch and session size, not a hard playlist length. - **Target size** is the desired batch and session size, not a hard playlist length.
- **Radio** is generated by the server for each playback session. It mixes seeds, related genres and artists, text matches, and the rest of the owner's music. - **Radio** is generated by the server for each playback session. It mixes seeds, related genres and artists, text matches, and the rest of the owner's music. Tracks started, finished or skipped in the last 7 days are skipped while the library is large enough, and artists/albums are spaced across the batch.
- **Signals** come from playback: completed tracks gain weight, skipped tracks lose weight. There are no likes. - **Signals** come from playback: completed tracks gain weight, skips weigh three times as much. There are no likes.
- **Add to mood** is explicit. The client offers *add to current mood* and *choose another mood*, and the server turns the track into a manual seed. - **Add to mood** is explicit. The client offers *add to current mood* and *choose another mood*, and the server turns the track into a manual seed.
- **Long-form is excluded** at seed validation and radio generation. - **Long-form is excluded** at seed validation and radio generation.
@@ -175,7 +176,10 @@ Song objects contain deterministic IDs (`sg-…`), album/artist names, duration,
- `POST /moods/{id}/seeds` — `{ songId }` - `POST /moods/{id}/seeds` — `{ songId }`
- `DELETE /moods/{id}/seeds/{songId}` - `DELETE /moods/{id}/seeds/{songId}`
- `GET /moods/{id}/preview?limit=20` — unpersisted radio preview. - `GET /moods/{id}/preview?limit=20` — unpersisted radio preview.
- `GET /moods/{id}/recommendations` — YT Music candidates (see roadmap). - `GET /moods/{id}/recommendations` — YT Music candidates with state and matched library song.
- `POST /moods/{id}/recommendations/refresh` — run discovery now (bypasses `refreshHours`).
- `POST /moods/{id}/recommendations/{videoId}/accept` — accept; the library match becomes a seed immediately.
- `POST /moods/{id}/recommendations/{videoId}/reject` — reject; the candidate is not offered again.
### Radio sessions ### Radio sessions
@@ -201,6 +205,9 @@ src/main/kotlin/win/taptap/musicmanager/
| |-- AppConfig.kt # Port, data dir, ffmpeg, env overrides | |-- AppConfig.kt # Port, data dir, ffmpeg, env overrides
| |-- LibraryConfig.kt # Folder IDs, types and paths | |-- LibraryConfig.kt # Folder IDs, types and paths
| `-- UsersConfig.kt # Users, admins and folder assignments | `-- UsersConfig.kt # Users, admins and folder assignments
|-- discovery/
| |-- YtSearch.kt # yt-dlp search and result parsing
| `-- DiscoveryService.kt # Queries, matching, scoring, pruning
|-- api/ |-- api/
| |-- ApiRoutes.kt # /api/v1 wiring, auth and system endpoints | |-- ApiRoutes.kt # /api/v1 wiring, auth and system endpoints
| |-- ApiModels.kt # Native DTOs | |-- ApiModels.kt # Native DTOs
@@ -223,6 +230,8 @@ src/main/kotlin/win/taptap/musicmanager/
`-- util/ # Deterministic IDs, sort names `-- util/ # Deterministic IDs, sort names
``` ```
YT discovery runs in the background for stale moods (`refreshHours`) and on demand from the mood screen. Queries come from the mood text, seed artists and genres; results are deduplicated, filtered to music-length videos, matched to library songs by normalized title and artist, and scored. Undecided candidates not seen in a refresh are pruned; accepted and rejected decisions are permanent. Accepted tracks that are not in the library wait for the future downloader, which will drop them into an import folder and rescan.
The old OpenSubsonic layer (`/rest`, `subsonic/`, playlist materialization) was removed in 0.2.0. The server and the KMP client are the two halves of the same product; there is no compatibility layer to preserve and no server-rendered UI. The old OpenSubsonic layer (`/rest`, `subsonic/`, playlist materialization) was removed in 0.2.0. The server and the KMP client are the two halves of the same product; there is no compatibility layer to preserve and no server-rendered UI.
## Database ## Database
@@ -324,7 +333,7 @@ The smoke test generates tagged MP3s, starts the server, and verifies login, syn
- [x] Native bearer-auth API, sync deltas, tombstones. - [x] Native bearer-auth API, sync deltas, tombstones.
- [x] Mood CRUD, seeds, server-driven radio, playback events, add-to-mood. - [x] Mood CRUD, seeds, server-driven radio, playback events, add-to-mood.
- [x] Kotlin Multiplatform client foundation (Android, desktop, web). - [x] Kotlin Multiplatform client foundation (Android, desktop, web).
- [ ] Offline downloads and offline playback across clients. - [x] Offline downloads and offline playback across clients.
- [ ] YT Music discovery through `yt-dlp`, recommendation list, accept/reject. - [x] YT Music discovery through `yt-dlp`, recommendation list, accept/reject.
- [ ] Optional downloader that writes accepted tracks into an import folder and rescans. - [ ] Optional downloader that writes accepted tracks into an import folder and rescans.
- [ ] Home-screen widgets, Android Auto, and richer radio tuning. - [ ] Home-screen widgets, Android Auto, and richer radio tuning.
+1 -1
View File
@@ -2,5 +2,5 @@ apiVersion: v2
name: audiomanager name: audiomanager
description: Mood-first self-hosted music server description: Mood-first self-hosted music server
type: application type: application
version: 0.2.1 version: 0.2.2
appVersion: "0.1.0" appVersion: "0.1.0"
@@ -42,6 +42,8 @@ spec:
value: /etc/audiomanager/users.yaml value: /etc/audiomanager/users.yaml
- name: FFMPEG_PATH - name: FFMPEG_PATH
value: ffmpeg value: ffmpeg
- name: YTDLP_PATH
value: yt-dlp
{{- with .Values.env }} {{- with .Values.env }}
{{- toYaml . | nindent 12 }} {{- toYaml . | nindent 12 }}
{{- end }} {{- end }}
+2 -1
View File
@@ -5,6 +5,7 @@ pkgs.mkShell {
jdk21 jdk21
gradle gradle
ffmpeg ffmpeg
yt-dlp
sqlite sqlite
git git
curl curl
@@ -17,7 +18,7 @@ pkgs.mkShell {
JAVA_HOME = pkgs.jdk21.home; JAVA_HOME = pkgs.jdk21.home;
shellHook = '' shellHook = ''
echo "MusicManager dev shell (JDK 21, Gradle, ffmpeg, sqlite, helm, kustomize)" echo "MusicManager dev shell (JDK 21, Gradle, ffmpeg, yt-dlp, sqlite, helm, kustomize)"
java -version java -version
echo "" echo ""
echo "Test: gradle test" echo "Test: gradle test"
@@ -19,11 +19,16 @@ import io.ktor.server.response.respond
import io.ktor.server.response.respondText import io.ktor.server.response.respondText
import io.ktor.server.routing.get import io.ktor.server.routing.get
import io.ktor.server.routing.routing import io.ktor.server.routing.routing
import java.time.Instant
import java.util.concurrent.TimeUnit
import kotlinx.coroutines.CoroutineScope import kotlinx.coroutines.CoroutineScope
import kotlinx.coroutines.Dispatchers import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.SupervisorJob import kotlinx.coroutines.SupervisorJob
import kotlinx.coroutines.cancelAndJoin import kotlinx.coroutines.cancelAndJoin
import kotlinx.coroutines.delay
import kotlinx.coroutines.isActive
import kotlinx.coroutines.job import kotlinx.coroutines.job
import kotlinx.coroutines.launch
import kotlinx.coroutines.runBlocking import kotlinx.coroutines.runBlocking
import kotlinx.serialization.json.Json import kotlinx.serialization.json.Json
import org.slf4j.LoggerFactory import org.slf4j.LoggerFactory
@@ -37,6 +42,9 @@ import win.taptap.musicmanager.api.apiRoutes
import win.taptap.musicmanager.db.ConfigRepository import win.taptap.musicmanager.db.ConfigRepository
import win.taptap.musicmanager.db.DatabaseFactory import win.taptap.musicmanager.db.DatabaseFactory
import win.taptap.musicmanager.db.MediaRepository import win.taptap.musicmanager.db.MediaRepository
import win.taptap.musicmanager.db.MoodRepository
import win.taptap.musicmanager.discovery.DiscoveryService
import win.taptap.musicmanager.discovery.YtDlpSearch
import win.taptap.musicmanager.radio.RadioService import win.taptap.musicmanager.radio.RadioService
import win.taptap.musicmanager.scanner.CoverArtService import win.taptap.musicmanager.scanner.CoverArtService
import win.taptap.musicmanager.scanner.MediaScanner import win.taptap.musicmanager.scanner.MediaScanner
@@ -58,6 +66,7 @@ fun main() {
val coverArtService = CoverArtService(config.dataDir) val coverArtService = CoverArtService(config.dataDir)
val transcoder = FfmpegTranscoder(config.ffmpegPath) val transcoder = FfmpegTranscoder(config.ffmpegPath)
val radioService = RadioService() val radioService = RadioService()
val discoveryService = DiscoveryService(YtDlpSearch(config.ytDlpPath))
val scanner = MediaScanner( val scanner = MediaScanner(
library = library, library = library,
coverArtService = coverArtService, coverArtService = coverArtService,
@@ -67,6 +76,17 @@ fun main() {
scanner.start(applicationScope) scanner.start(applicationScope)
} }
// Moods refresh their YT recommendations in the background; the client
// can also force a pass from the mood screen.
applicationScope.launch(Dispatchers.IO) {
delay(TimeUnit.MINUTES.toMillis(2))
while (isActive) {
runCatching { discoverStaleMoods(users, library, discoveryService) }
.onFailure { log.warn("Background discovery pass failed", it) }
delay(TimeUnit.HOURS.toMillis(1))
}
}
val server = embeddedServer(Netty, port = config.port, host = "0.0.0.0") { val server = embeddedServer(Netty, port = config.port, host = "0.0.0.0") {
install(ContentNegotiation) { install(ContentNegotiation) {
json( json(
@@ -136,6 +156,7 @@ fun main() {
coverArtService = coverArtService, coverArtService = coverArtService,
transcoder = transcoder, transcoder = transcoder,
radioService = radioService, radioService = radioService,
discoveryService = discoveryService,
startScan = { scanner.start(applicationScope) }, startScan = { scanner.start(applicationScope) },
scanStatus = { scanner.status() }, scanStatus = { scanner.status() },
) )
@@ -163,3 +184,15 @@ fun main() {
throw t throw t
} }
} }
private suspend fun discoverStaleMoods(
users: UsersConfig,
library: LibraryConfig,
discovery: DiscoveryService,
) {
MoodRepository.staleDiscovery(Instant.now(), 10).forEach { mood ->
val owner = users.byName(mood.owner) ?: return@forEach
runCatching { discovery.refresh(mood, owner, library) }
.onFailure { log.warn("Discovery for mood {} failed", mood.name, it) }
}
}
@@ -133,6 +133,13 @@ data class MoodDetailDto(
val recommendations: List<RecommendationDto> = emptyList(), val recommendations: List<RecommendationDto> = emptyList(),
) )
@Serializable
data class RecommendationDecisionDto(
val detail: MoodDetailDto,
val recommendation: RecommendationDto,
val seedAdded: Boolean = false,
)
@Serializable @Serializable
data class CreateMoodRequest( data class CreateMoodRequest(
val name: String, val name: String,
@@ -16,6 +16,7 @@ import win.taptap.musicmanager.api.UserDto
import win.taptap.musicmanager.config.LibraryConfig import win.taptap.musicmanager.config.LibraryConfig
import win.taptap.musicmanager.config.UsersConfig import win.taptap.musicmanager.config.UsersConfig
import win.taptap.musicmanager.db.ApiTokenRepository import win.taptap.musicmanager.db.ApiTokenRepository
import win.taptap.musicmanager.discovery.DiscoveryService
import win.taptap.musicmanager.radio.RadioService import win.taptap.musicmanager.radio.RadioService
import win.taptap.musicmanager.scanner.CoverArtService import win.taptap.musicmanager.scanner.CoverArtService
import win.taptap.musicmanager.scanner.ScanStatus import win.taptap.musicmanager.scanner.ScanStatus
@@ -33,6 +34,7 @@ fun Route.apiRoutes(
coverArtService: CoverArtService, coverArtService: CoverArtService,
transcoder: FfmpegTranscoder, transcoder: FfmpegTranscoder,
radioService: RadioService, radioService: RadioService,
discoveryService: DiscoveryService,
startScan: () -> Boolean, startScan: () -> Boolean,
scanStatus: () -> ScanStatus, scanStatus: () -> ScanStatus,
) { ) {
@@ -125,7 +127,7 @@ fun Route.apiRoutes(
} }
libraryRoutes(users, library, coverArtService, transcoder) libraryRoutes(users, library, coverArtService, transcoder)
moodRoutes(users, library, radioService) moodRoutes(users, library, radioService, discoveryService)
sessionRoutes(users, library, radioService) sessionRoutes(users, library, radioService)
} }
} }
@@ -7,12 +7,15 @@ import io.ktor.server.routing.delete
import io.ktor.server.routing.get import io.ktor.server.routing.get
import io.ktor.server.routing.patch import io.ktor.server.routing.patch
import io.ktor.server.routing.post import io.ktor.server.routing.post
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.withContext
import kotlinx.serialization.Serializable import kotlinx.serialization.Serializable
import win.taptap.musicmanager.api.AddSeedRequest import win.taptap.musicmanager.api.AddSeedRequest
import win.taptap.musicmanager.api.AddToMoodResponse import win.taptap.musicmanager.api.AddToMoodResponse
import win.taptap.musicmanager.api.CreateMoodRequest import win.taptap.musicmanager.api.CreateMoodRequest
import win.taptap.musicmanager.api.MoodDetailDto import win.taptap.musicmanager.api.MoodDetailDto
import win.taptap.musicmanager.api.MoodDto import win.taptap.musicmanager.api.MoodDto
import win.taptap.musicmanager.api.RecommendationDecisionDto
import win.taptap.musicmanager.api.RecommendationDto import win.taptap.musicmanager.api.RecommendationDto
import win.taptap.musicmanager.api.SearchResponse import win.taptap.musicmanager.api.SearchResponse
import win.taptap.musicmanager.api.UpdateMoodRequest import win.taptap.musicmanager.api.UpdateMoodRequest
@@ -24,6 +27,7 @@ import win.taptap.musicmanager.config.UsersConfig
import win.taptap.musicmanager.db.LibraryRepository import win.taptap.musicmanager.db.LibraryRepository
import win.taptap.musicmanager.db.MoodRecord import win.taptap.musicmanager.db.MoodRecord
import win.taptap.musicmanager.db.MoodRepository import win.taptap.musicmanager.db.MoodRepository
import win.taptap.musicmanager.discovery.DiscoveryService
import win.taptap.musicmanager.radio.RadioService import win.taptap.musicmanager.radio.RadioService
@Serializable @Serializable
@@ -33,6 +37,7 @@ fun Route.moodRoutes(
users: UsersConfig, users: UsersConfig,
library: LibraryConfig, library: LibraryConfig,
radioService: RadioService, radioService: RadioService,
discoveryService: DiscoveryService,
) { ) {
get("/moods") { get("/moods") {
val user = call.requireApiUser(users) val user = call.requireApiUser(users)
@@ -147,6 +152,50 @@ fun Route.moodRoutes(
val mood = ownedMood(call.parameters["id"], user) val mood = ownedMood(call.parameters["id"], user)
call.respond(recommendations(mood, user, library)) call.respond(recommendations(mood, user, library))
} }
/** Runs a discovery pass now, bypassing the mood's refresh interval. */
post("/moods/{id}/recommendations/refresh") {
val user = call.requireApiUser(users)
val mood = ownedMood(call.parameters["id"], user)
if (!mood.ytEnabled) conflict("YT discovery is disabled for this mood")
val outcome = withContext(Dispatchers.IO) {
discoveryService.refresh(mood, user, library, force = true)
}
outcome.error?.let { message ->
throw ApiException(HttpStatusCode.BadGateway, "discovery_failed", message)
}
val refreshed = MoodRepository.get(mood.id, user.name) ?: mood
call.respond(detail(refreshed, user, library))
}
post("/moods/{id}/recommendations/{videoId}/accept") {
val user = call.requireApiUser(users)
val mood = ownedMood(call.parameters["id"], user)
val videoId = call.parameters["videoId"] ?: badRequest("Video id is required")
val recommendation = MoodRepository.recommendation(mood.id, videoId)
?: notFound("Recommendation")
val matched = recommendation.matchedSongId?.let { songId ->
LibraryRepository.song(songId, user.musicFolderIds(library), user.name)
}
val seedAdded = matched != null &&
MoodRepository.addSeed(mood.id, user.name, matched.id, "recommendation")
val alreadyAccepted = recommendation.state == "accepted"
MoodRepository.setRecommendationState(mood.id, videoId, "accepted")
if (!alreadyAccepted) MoodRepository.incrementRecommendationAdds(mood.id, videoId)
call.respond(decision(mood.id, videoId, user, library, seedAdded))
}
post("/moods/{id}/recommendations/{videoId}/reject") {
val user = call.requireApiUser(users)
val mood = ownedMood(call.parameters["id"], user)
val videoId = call.parameters["videoId"] ?: badRequest("Video id is required")
MoodRepository.recommendation(mood.id, videoId) ?: notFound("Recommendation")
MoodRepository.setRecommendationState(mood.id, videoId, "rejected")
call.respond(decision(mood.id, videoId, user, library, seedAdded = false))
}
} }
private fun ownedMood(id: String?, user: UserAccount): MoodRecord { private fun ownedMood(id: String?, user: UserAccount): MoodRecord {
@@ -205,5 +254,23 @@ private fun recommendations(
) )
} }
private fun decision(
moodId: String,
videoId: String,
user: UserAccount,
library: LibraryConfig,
seedAdded: Boolean,
): RecommendationDecisionDto {
val mood = MoodRepository.get(moodId, user.name) ?: notFound("Mood")
val detail = detail(mood, user, library)
val recommendation = detail.recommendations.firstOrNull { it.videoId == videoId }
?: notFound("Recommendation")
return RecommendationDecisionDto(
detail = detail,
recommendation = recommendation,
seedAdded = seedAdded,
)
}
@Serializable @Serializable
data class RecommendationsResponse(val recommendations: List<RecommendationDto>) data class RecommendationsResponse(val recommendations: List<RecommendationDto>)
@@ -14,6 +14,7 @@ data class AppConfig(
val libraryConfigPath: Path, val libraryConfigPath: Path,
val usersFile: Path, val usersFile: Path,
val ffmpegPath: String, val ffmpegPath: String,
val ytDlpPath: String = "yt-dlp",
) { ) {
val databasePath: Path get() = dataDir.resolve("musicmanager.db") val databasePath: Path get() = dataDir.resolve("musicmanager.db")
@@ -33,6 +34,7 @@ data class AppConfig(
libraryConfigPath = Path.of(value("LIBRARY_CONFIG", "./config/library.yaml")), libraryConfigPath = Path.of(value("LIBRARY_CONFIG", "./config/library.yaml")),
usersFile = Path.of(value("USERS_FILE", "./config/users.yaml")), usersFile = Path.of(value("USERS_FILE", "./config/users.yaml")),
ffmpegPath = value("FFMPEG_PATH", "ffmpeg"), ffmpegPath = value("FFMPEG_PATH", "ffmpeg"),
ytDlpPath = value("YTDLP_PATH", "yt-dlp"),
) )
} }
} }
@@ -1,12 +1,14 @@
package win.taptap.musicmanager.db package win.taptap.musicmanager.db
import java.time.Instant import java.time.Instant
import java.time.temporal.ChronoUnit
import kotlinx.serialization.builtins.ListSerializer import kotlinx.serialization.builtins.ListSerializer
import kotlinx.serialization.builtins.serializer import kotlinx.serialization.builtins.serializer
import kotlinx.serialization.json.Json import kotlinx.serialization.json.Json
import org.jetbrains.exposed.sql.ResultRow import org.jetbrains.exposed.sql.ResultRow
import org.jetbrains.exposed.sql.SortOrder import org.jetbrains.exposed.sql.SortOrder
import org.jetbrains.exposed.sql.SqlExpressionBuilder.eq import org.jetbrains.exposed.sql.SqlExpressionBuilder.eq
import org.jetbrains.exposed.sql.SqlExpressionBuilder.inList
import org.jetbrains.exposed.sql.and import org.jetbrains.exposed.sql.and
import org.jetbrains.exposed.sql.deleteWhere import org.jetbrains.exposed.sql.deleteWhere
import org.jetbrains.exposed.sql.insert import org.jetbrains.exposed.sql.insert
@@ -40,6 +42,16 @@ data class MoodSeedRecord(
val addedAt: String, val addedAt: String,
) )
data class RecommendationUpsert(
val videoId: String,
val title: String,
val artist: String,
val album: String?,
val duration: Int?,
val thumbnail: String?,
val matchedSongId: String?,
)
data class RecommendationRecord( data class RecommendationRecord(
val videoId: String, val videoId: String,
val title: String, val title: String,
@@ -212,21 +224,117 @@ object MoodRepository {
MoodRecommendationsTable.selectAll() MoodRecommendationsTable.selectAll()
.where { MoodRecommendationsTable.moodId eq id } .where { MoodRecommendationsTable.moodId eq id }
.orderBy(MoodRecommendationsTable.lastSeen to SortOrder.DESC) .orderBy(MoodRecommendationsTable.lastSeen to SortOrder.DESC)
.map { .map { it.toRecommendationRecord() }
RecommendationRecord( }
videoId = it[MoodRecommendationsTable.videoId],
title = it[MoodRecommendationsTable.title], fun recommendation(moodId: String, videoId: String): RecommendationRecord? = transaction {
artist = it[MoodRecommendationsTable.artist], MoodRecommendationsTable.selectAll()
album = it[MoodRecommendationsTable.album], .where {
duration = it[MoodRecommendationsTable.duration], (MoodRecommendationsTable.moodId eq moodId) and
thumbnail = it[MoodRecommendationsTable.thumbnail], (MoodRecommendationsTable.videoId eq videoId)
state = it[MoodRecommendationsTable.state],
matchedSongId = it[MoodRecommendationsTable.matchedSongId],
timesAdded = it[MoodRecommendationsTable.timesAdded],
firstSeen = it[MoodRecommendationsTable.firstSeen],
lastSeen = it[MoodRecommendationsTable.lastSeen],
)
} }
.singleOrNull()
?.toRecommendationRecord()
}
/**
* Inserts new candidates and refreshes the metadata of known ones. Decisions
* (accepted/rejected) and the add counter survive a refresh; stale undecided
* candidates are removed so the review list does not grow without bound.
*/
fun storeRecommendations(
moodId: String,
items: List<RecommendationUpsert>,
seenAt: String,
): Int = transaction {
val existing = MoodRecommendationsTable.selectAll()
.where { MoodRecommendationsTable.moodId eq moodId }
.associateBy { it[MoodRecommendationsTable.videoId] }
var stored = 0
items.forEach { item ->
val row = existing[item.videoId]
if (row == null) {
MoodRecommendationsTable.insert {
it[MoodRecommendationsTable.moodId] = moodId
it[videoId] = item.videoId
it[title] = item.title.take(500)
it[artist] = item.artist.take(300)
it[album] = item.album?.take(300)
it[duration] = item.duration
it[thumbnail] = item.thumbnail?.take(500)
it[state] = "new"
it[matchedSongId] = item.matchedSongId
it[timesAdded] = 0
it[firstSeen] = seenAt
it[lastSeen] = seenAt
}
} else {
MoodRecommendationsTable.update({
(MoodRecommendationsTable.moodId eq moodId) and
(MoodRecommendationsTable.videoId eq item.videoId)
}) {
it[title] = item.title.take(500)
it[artist] = item.artist.take(300)
it[album] = item.album?.take(300)
it[duration] = item.duration
it[thumbnail] = item.thumbnail?.take(500)
it[matchedSongId] = item.matchedSongId
?: row[MoodRecommendationsTable.matchedSongId]
it[lastSeen] = seenAt
}
}
stored += 1
}
val keep = items.map { it.videoId }.toSet()
val stale = existing.keys - keep
if (stale.isNotEmpty()) {
MoodRecommendationsTable.deleteWhere {
(MoodRecommendationsTable.moodId eq moodId) and
(MoodRecommendationsTable.videoId inList stale.toList()) and
(MoodRecommendationsTable.state eq "new")
}
}
stored
}
fun setRecommendationState(moodId: String, videoId: String, state: String): Boolean = transaction {
MoodRecommendationsTable.update({
(MoodRecommendationsTable.moodId eq moodId) and
(MoodRecommendationsTable.videoId eq videoId)
}) {
it[MoodRecommendationsTable.state] = state
} > 0
}
fun incrementRecommendationAdds(moodId: String, videoId: String) {
transaction {
val current = MoodRecommendationsTable.selectAll()
.where {
(MoodRecommendationsTable.moodId eq moodId) and
(MoodRecommendationsTable.videoId eq videoId)
}
.singleOrNull()
?.get(MoodRecommendationsTable.timesAdded)
?: return@transaction
MoodRecommendationsTable.update({
(MoodRecommendationsTable.moodId eq moodId) and
(MoodRecommendationsTable.videoId eq videoId)
}) {
it[timesAdded] = current + 1
}
}
}
/** Moods whose YT discovery is due, oldest first. */
fun staleDiscovery(now: Instant, limit: Int): List<MoodRecord> = transaction {
MoodsTable.selectAll()
.where { MoodsTable.ytEnabled eq true }
.orderBy(MoodsTable.lastDiscoveryAt to SortOrder.ASC)
.map { moodRecord(it) }
.filter { mood -> isStale(mood.lastDiscoveryAt, mood.refreshHours, now) }
.take(limit.coerceAtLeast(1))
} }
fun markError(id: String, message: String) { fun markError(id: String, message: String) {
@@ -258,6 +366,26 @@ object MoodRepository {
} }
} }
private fun isStale(lastDiscoveryAt: String?, refreshHours: Int, now: Instant): Boolean {
if (lastDiscoveryAt == null) return true
val at = runCatching { Instant.parse(lastDiscoveryAt) }.getOrNull() ?: return true
return !at.plus(refreshHours.toLong().coerceAtLeast(1), ChronoUnit.HOURS).isAfter(now)
}
private fun ResultRow.toRecommendationRecord() = RecommendationRecord(
videoId = this[MoodRecommendationsTable.videoId],
title = this[MoodRecommendationsTable.title],
artist = this[MoodRecommendationsTable.artist],
album = this[MoodRecommendationsTable.album],
duration = this[MoodRecommendationsTable.duration],
thumbnail = this[MoodRecommendationsTable.thumbnail],
state = this[MoodRecommendationsTable.state],
matchedSongId = this[MoodRecommendationsTable.matchedSongId],
timesAdded = this[MoodRecommendationsTable.timesAdded],
firstSeen = this[MoodRecommendationsTable.firstSeen],
lastSeen = this[MoodRecommendationsTable.lastSeen],
)
private fun moodRecord(row: ResultRow): MoodRecord { private fun moodRecord(row: ResultRow): MoodRecord {
val id = row[MoodsTable.id] val id = row[MoodsTable.id]
val seedCount = MoodSeedsTable.selectAll() val seedCount = MoodSeedsTable.selectAll()
@@ -3,6 +3,9 @@ package win.taptap.musicmanager.db
import java.time.Instant import java.time.Instant
import org.jetbrains.exposed.sql.SortOrder import org.jetbrains.exposed.sql.SortOrder
import org.jetbrains.exposed.sql.SqlExpressionBuilder.eq import org.jetbrains.exposed.sql.SqlExpressionBuilder.eq
import org.jetbrains.exposed.sql.SqlExpressionBuilder.greaterEq
import org.jetbrains.exposed.sql.SqlExpressionBuilder.inList
import org.jetbrains.exposed.sql.SqlExpressionBuilder.inSubQuery
import org.jetbrains.exposed.sql.and import org.jetbrains.exposed.sql.and
import org.jetbrains.exposed.sql.insert import org.jetbrains.exposed.sql.insert
import org.jetbrains.exposed.sql.selectAll import org.jetbrains.exposed.sql.selectAll
@@ -25,8 +28,8 @@ data class SessionItemRecord(
) )
data class AffinityRecord(val completions: Int, val skips: Int) { data class AffinityRecord(val completions: Int, val skips: Int) {
/** Skips weigh double so a mood stops serving rejected tracks quickly. */ /** Skips weigh triple so a mood stops serving rejected tracks quickly. */
val score: Double get() = completions - skips * 2.0 val score: Double get() = completions - skips * 3.0
} }
/** /**
@@ -185,6 +188,28 @@ object RadioRepository {
true true
} }
/**
* Songs the user started, finished or skipped since [since]. Used to avoid
* serving the same tracks every day across sessions and moods.
*/
fun recentSongIds(userName: String, since: Instant, limit: Int = 500): Set<String> = transaction {
val sessions = PlaybackSessionsTable
.select(PlaybackSessionsTable.id)
.where { PlaybackSessionsTable.userName eq userName }
SessionItemsTable
.select(SessionItemsTable.songId)
.where {
(SessionItemsTable.sessionId inSubQuery sessions) and
(SessionItemsTable.state inList listOf("started", "completed", "skipped")) and
(SessionItemsTable.updatedAt greaterEq since.toString())
}
.orderBy(SessionItemsTable.updatedAt to SortOrder.DESC)
.limit(limit)
.map { it[SessionItemsTable.songId] }
.toSet()
}
fun affinity(userName: String, songIds: Collection<String>): Map<String, AffinityRecord> { fun affinity(userName: String, songIds: Collection<String>): Map<String, AffinityRecord> {
if (songIds.isEmpty()) return emptyMap() if (songIds.isEmpty()) return emptyMap()
return transaction { return transaction {
@@ -0,0 +1,220 @@
package win.taptap.musicmanager.discovery
import java.time.Instant
import java.time.temporal.ChronoUnit
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.async
import kotlinx.coroutines.awaitAll
import kotlinx.coroutines.coroutineScope
import org.slf4j.LoggerFactory
import win.taptap.musicmanager.config.FolderType
import win.taptap.musicmanager.config.LibraryConfig
import win.taptap.musicmanager.config.UserAccount
import win.taptap.musicmanager.db.LibraryRepository
import win.taptap.musicmanager.db.MoodRecord
import win.taptap.musicmanager.db.MoodRepository
import win.taptap.musicmanager.db.RecommendationUpsert
import win.taptap.musicmanager.util.IdGenerator
data class DiscoveryOutcome(
val stored: Int,
val matched: Int,
val skipped: Boolean = false,
val error: String? = null,
)
/**
* Turns mood seeds, genres and free text into YouTube Music candidates. New
* candidates are stored for review; accepted ones either become seeds (when the
* library already has the track) or wait for the future downloader.
*/
class DiscoveryService(
private val search: YtSearch,
private val clock: () -> Instant = { Instant.now() },
) {
private val log = LoggerFactory.getLogger(DiscoveryService::class.java)
suspend fun refresh(
mood: MoodRecord,
user: UserAccount,
library: LibraryConfig,
force: Boolean = false,
): DiscoveryOutcome {
if (!mood.ytEnabled) return DiscoveryOutcome(0, 0, skipped = true)
val now = clock()
if (!force && isFresh(mood.lastDiscoveryAt, mood.refreshHours, now)) {
return DiscoveryOutcome(0, 0, skipped = true)
}
val folders = library.folders
.asSequence()
.filter { it.type == FolderType.MUSIC && it.id in user.folders }
.map { it.id }
.toSet()
if (folders.isEmpty()) {
MoodRepository.markError(mood.id, "No music folder is visible to this user")
return DiscoveryOutcome(0, 0, error = "No music folder is visible to this user")
}
val seeds = LibraryRepository.songsByIds(MoodRepository.seedSongIds(mood.id), folders, user.name)
val seedIds = seeds.map { it.id }.toSet()
val seedArtists = seeds.map { it.displayArtist }.filter { it.isNotBlank() }
val queries = buildQueries(mood, seedArtists)
if (queries.isEmpty()) {
MoodRepository.markDiscovered(mood.id)
return DiscoveryOutcome(0, 0)
}
// Queries are independent; running them together keeps a refresh within
// the client's request timeout even when yt-dlp is slow.
val results = coroutineScope {
queries.map { query ->
async(Dispatchers.IO) {
runCatching { search.search(query, QUERY_LIMIT) }
}
}.awaitAll()
}
val failures = results.filter { it.isFailure }
if (failures.size == queries.size) {
val message = failures.firstNotNullOfOrNull { it.exceptionOrNull()?.message }
?: "Search failed"
log.warn("YT discovery failed for mood {}: {}", mood.name, message)
MoodRepository.markError(mood.id, message)
return DiscoveryOutcome(0, 0, error = message)
}
val candidates = LinkedHashMap<String, YtCandidate>()
results.mapNotNull { it.getOrNull() }.flatten().forEach { candidate ->
if (isMusicLength(candidate.duration)) {
candidates.putIfAbsent(candidate.videoId, candidate)
}
}
val known = MoodRepository.recommendations(mood.id).associateBy { it.videoId }
val moodText = listOfNotNull(mood.query, mood.description, mood.name)
.joinToString(" ")
val upserts = candidates.values
.mapNotNull { candidate ->
val existing = known[candidate.videoId]
// Accepted or rejected tracks keep their decision and are never
// offered again; only last_seen is refreshed by the upsert.
val matched = existing?.matchedSongId ?: matchSong(candidate, folders, user.name)
if (existing == null && matched != null && matched in seedIds) {
return@mapNotNull null
}
UpsertCandidate(
candidate = candidate,
matchedSongId = matched,
score = scoreCandidate(candidate, seedArtists, moodText, matched != null),
)
}
.sortedWith(compareByDescending<UpsertCandidate> { it.score }.thenBy { it.candidate.title })
.take(mood.targetSize.coerceIn(10, 60))
.map {
RecommendationUpsert(
videoId = it.candidate.videoId,
title = it.candidate.title,
artist = it.candidate.artist,
album = it.candidate.album,
duration = it.candidate.duration,
thumbnail = it.candidate.thumbnail,
matchedSongId = it.matchedSongId,
)
}
val stored = MoodRepository.storeRecommendations(mood.id, upserts, now.toString())
MoodRepository.markDiscovered(mood.id)
log.info("YT discovery for mood {} stored {} recommendations", mood.name, stored)
return DiscoveryOutcome(
stored = stored,
matched = upserts.count { it.matchedSongId != null },
)
}
private fun matchSong(candidate: YtCandidate, folders: Set<Int>, userName: String): String? {
val titleKey = matchKey(candidate.title)
if (titleKey.isEmpty()) return null
return LibraryRepository
.searchSongs(candidate.title, MATCH_LIMIT, 0, folders, userName)
.firstOrNull { song ->
matchKey(song.title) == titleKey && artistMatches(candidate.artist, song.displayArtist)
}
?.id
}
private data class UpsertCandidate(
val candidate: YtCandidate,
val matchedSongId: String?,
val score: Double,
)
private companion object {
const val QUERY_LIMIT = 10
const val MATCH_LIMIT = 25
}
}
internal fun isFresh(lastDiscoveryAt: String?, refreshHours: Int, now: Instant): Boolean {
if (lastDiscoveryAt == null) return false
val at = runCatching { Instant.parse(lastDiscoveryAt) }.getOrNull() ?: return false
return at.plus(refreshHours.toLong().coerceAtLeast(1), ChronoUnit.HOURS).isAfter(now)
}
internal fun isMusicLength(seconds: Int?): Boolean =
seconds == null || seconds in 45..(20 * 60)
internal fun buildQueries(mood: MoodRecord, seedArtists: List<String>): List<String> {
val queries = LinkedHashSet<String>()
mood.query?.trim()?.takeIf { it.isNotEmpty() }?.let(queries::add)
seedArtists.map { it.trim() }.filter { it.isNotEmpty() }.distinct().take(2).forEach {
queries += it
}
mood.genres.map { it.trim() }.filter { it.isNotEmpty() }.take(2).forEach {
queries += "$it mix"
}
if (queries.isEmpty()) queries += mood.name
return queries.distinctBy { it.lowercase() }.take(4)
}
internal fun scoreCandidate(
candidate: YtCandidate,
seedArtists: List<String>,
moodText: String,
matched: Boolean,
): Double {
var score = 0.0
val artist = matchKey(candidate.artist)
val seedKeys = seedArtists.map { matchKey(it) }.filter { it.isNotEmpty() }
when {
artist.isNotEmpty() && artist in seedKeys -> score += 3.0
artist.isNotEmpty() && seedKeys.any { artist.contains(it) || it.contains(artist) } -> score += 1.5
}
val text = matchKey(moodText)
if (text.isNotEmpty() && matchKey(candidate.title).contains(text)) score += 1.0
if (matched) score += 1.5
if ((candidate.duration ?: 0) > 8 * 60) score -= 1.0
return score
}
private val BRACKETS = Regex("\\([^)]*\\)|\\[[^\\]]*\\]")
private val NON_WORD = Regex("[^\\p{L}\\p{N} ]+")
private val SPACES = Regex("\\s+")
private val FEATURING = Regex("\\b(feat|ft|featuring)\\b.*$", RegexOption.IGNORE_CASE)
/** Loose key for comparing YT titles and artist names with library tags. */
internal fun matchKey(value: String): String =
IdGenerator.normalize(value)
.replace(BRACKETS, " ")
.replace(FEATURING, " ")
.replace(NON_WORD, " ")
.replace(SPACES, " ")
.trim()
internal fun artistMatches(a: String, b: String): Boolean {
val left = matchKey(a)
val right = matchKey(b)
if (left.isEmpty() || right.isEmpty()) return false
return left == right || left.contains(right) || right.contains(left)
}
@@ -0,0 +1,130 @@
package win.taptap.musicmanager.discovery
import java.util.concurrent.TimeUnit
import kotlin.concurrent.thread
import kotlinx.serialization.json.Json
import kotlinx.serialization.json.JsonObject
import kotlinx.serialization.json.JsonPrimitive
import kotlinx.serialization.json.contentOrNull
import kotlinx.serialization.json.jsonArray
import kotlinx.serialization.json.jsonObject
/** One YouTube search hit, already split into an artist and a track title. */
data class YtCandidate(
val videoId: String,
val title: String,
val artist: String,
val album: String? = null,
val duration: Int? = null,
val thumbnail: String? = null,
)
fun interface YtSearch {
fun search(query: String, limit: Int): List<YtCandidate>
}
/**
* Searches YouTube through `yt-dlp`. Flat playlist output avoids resolving every
* video, which keeps a discovery run to a single request per query.
*/
class YtDlpSearch(
private val executable: String,
private val timeoutSeconds: Long = 30,
) : YtSearch {
override fun search(query: String, limit: Int): List<YtCandidate> {
val process = ProcessBuilder(
executable,
"--flat-playlist",
"--dump-single-json",
"--no-warnings",
"--ignore-errors",
"--no-cache-dir",
"--socket-timeout",
"10",
"ytsearch${limit.coerceIn(1, 25)}:$query",
)
.redirectErrorStream(true)
.start()
val output = StringBuilder()
val reader = thread(isDaemon = true, name = "yt-dlp-search") {
process.inputStream.bufferedReader().useLines { lines ->
lines.forEach { output.appendLine(it) }
}
}
if (!process.waitFor(timeoutSeconds, TimeUnit.SECONDS)) {
process.destroyForcibly()
reader.join(2_000)
throw IllegalStateException("yt-dlp timed out after ${timeoutSeconds}s")
}
reader.join(2_000)
if (process.exitValue() != 0) {
val detail = output.toString().take(400).trim()
throw IllegalStateException("yt-dlp exited with ${process.exitValue()}: $detail")
}
return parseYtSearchOutput(output.toString())
}
}
internal fun parseYtSearchOutput(json: String): List<YtCandidate> {
if (json.isBlank()) return emptyList()
val root = runCatching { Json.parseToJsonElement(json).jsonObject }.getOrNull() ?: return emptyList()
val entries = runCatching { root["entries"]?.jsonArray }.getOrNull() ?: return emptyList()
return entries.mapNotNull { element ->
val entry = runCatching { element.jsonObject }.getOrNull() ?: return@mapNotNull null
val videoId = entry.string("id")?.takeIf { it.isNotBlank() } ?: return@mapNotNull null
val rawTitle = entry.string("title")?.takeIf { it.isNotBlank() } ?: return@mapNotNull null
val channel = entry.string("channel") ?: entry.string("uploader").orEmpty()
val (artist, title) = splitYtTitle(rawTitle, channel)
YtCandidate(
videoId = videoId,
title = title.ifEmpty { rawTitle },
artist = artist.ifEmpty { "Unknown artist" },
album = entry.string("album"),
duration = entry.number("duration")?.toInt(),
thumbnail = entry.thumbnail(),
)
}
}
private const val NOISE_WORDS = "official|lyrics?|audio|video|visuali[sz]er|mv|hd|4k|remaster(?:ed)?"
private val TITLE_NOISE = Regex(
"\\((?:$NOISE_WORDS)[^)]*\\)" +
"|\\[(?:$NOISE_WORDS)[^\\]]*\\]" +
"|\\s*\\|\\s*(?:$NOISE_WORDS).*$",
RegexOption.IGNORE_CASE,
)
/** "Artist - Track (Official Video)" becomes ("Artist", "Track"). */
internal fun splitYtTitle(rawTitle: String, channel: String): Pair<String, String> {
val cleaned = rawTitle.replace(TITLE_NOISE, "").trim()
for (separator in listOf(" - ", " – ", " — ")) {
val index = cleaned.indexOf(separator)
if (index > 0) {
val artist = cleaned.substring(0, index).trim()
val title = cleaned.substring(index + separator.length).trim()
if (artist.isNotEmpty() && title.isNotEmpty()) return artist to title
}
}
val artist = channel.removeSuffix(" - Topic").trim()
return artist to cleaned
}
private fun JsonObject.string(key: String): String? =
(this[key] as? JsonPrimitive)?.contentOrNull
private fun JsonObject.number(key: String): Double? =
(this[key] as? JsonPrimitive)?.contentOrNull?.toDoubleOrNull()
private fun JsonObject.thumbnail(): String? {
val thumbnails = runCatching { get("thumbnails")?.jsonArray }.getOrNull()
val last = thumbnails
?.mapNotNull { runCatching { it.jsonObject.string("url") }.getOrNull() }
?.lastOrNull()
return last ?: string("thumbnail")
}
@@ -1,5 +1,7 @@
package win.taptap.musicmanager.radio package win.taptap.musicmanager.radio
import java.time.Instant
import java.time.temporal.ChronoUnit
import java.util.UUID import java.util.UUID
import kotlin.math.max import kotlin.math.max
import kotlin.random.Random import kotlin.random.Random
@@ -140,15 +142,27 @@ class RadioService {
} }
} }
val recent = RadioRepository.recentSongIds(
user.name,
Instant.now().minus(RECENT_DAYS, ChronoUnit.DAYS),
)
// In a normal-sized library, skip songs from the last week entirely.
// Small libraries keep them as low-scored fallbacks instead of going silent.
if (candidates.size > limit * 3) {
candidates.keys.removeAll(recent - seedIds)
}
val affinity = RadioRepository.affinity(user.name, candidates.keys) val affinity = RadioRepository.affinity(user.name, candidates.keys)
val scored = candidates.values.map { song -> val scored = candidates.values.map { song ->
song to score( song to scoreCandidate(
song = song, song = song,
seedIds = seedIds, seedIds = seedIds,
seedGenres = seedGenres, seedGenres = seedGenres,
seedArtistIds = seedArtistIds, seedArtistIds = seedArtistIds,
query = query, query = query,
affinity = affinity[song.id]?.score ?: 0.0, affinity = affinity[song.id]?.score ?: 0.0,
recent = recent,
jitter = random.nextDouble(0.0, 2.0),
) )
} }
@@ -159,50 +173,69 @@ class RadioService {
return picks return picks
} }
private fun score( internal fun scoreCandidate(
song: SongRecord, song: SongRecord,
seedIds: Set<String>, seedIds: Set<String>,
seedGenres: List<String>, seedGenres: List<String>,
seedArtistIds: List<String>, seedArtistIds: List<String>,
query: String?, query: String?,
affinity: Double, affinity: Double,
recent: Set<String> = emptySet(),
jitter: Double = 0.0,
): Double { ): Double {
var score = random.nextDouble(0.0, 2.0) var score = jitter
if (song.id in seedIds) score += 10.0 if (song.id in seedIds) score += 10.0
if (song.artistId != null && song.artistId in seedArtistIds) score += 3.0 if (song.artistId != null && song.artistId in seedArtistIds) score += 3.0
score += song.genres.count { candidate -> score += song.genres.count { candidate ->
seedGenres.any { it.equals(candidate, ignoreCase = true) } seedGenres.any { it.equals(candidate, ignoreCase = true) }
} * 1.5 } * 1.5
if (query != null && song.title.contains(query, ignoreCase = true)) score += 2.0 if (query != null && song.title.contains(query, ignoreCase = true)) score += 2.0
score += affinity.coerceIn(-6.0, 6.0) score += affinity.coerceIn(-8.0, 8.0)
if (song.id in recent) score -= RECENT_PENALTY
return score return score
} }
/** /**
* Picks the best remaining candidate, preferring a different artist than the * Picks the best remaining candidate while keeping artists and albums apart.
* previous track so a batch does not become one album on repeat. * An artist may return after [artistSpacing] other tracks, an album after
* [albumSpacing]. Fallbacks relax the constraints rather than stalling.
*/ */
private fun pickDiverse( internal fun pickDiverse(
scored: List<Pair<SongRecord, Double>>, scored: List<Pair<SongRecord, Double>>,
limit: Int, limit: Int,
artistSpacing: Int = 2,
albumSpacing: Int = 3,
): List<SongRecord> { ): List<SongRecord> {
val remaining = scored val remaining = scored
.sortedWith(compareByDescending<Pair<SongRecord, Double>> { it.second }) .sortedWith(compareByDescending<Pair<SongRecord, Double>> { it.second })
.toMutableList() .toMutableList()
val picked = ArrayList<SongRecord>(limit) val picked = ArrayList<SongRecord>(limit)
var lastArtist: String? = null val artists = ArrayDeque<String>()
val albums = ArrayDeque<String>()
while (picked.size < limit && remaining.isNotEmpty()) { while (picked.size < limit && remaining.isNotEmpty()) {
val different = remaining.indexOfFirst { it.first.displayArtist != lastArtist } var index = remaining.indexOfFirst { (song, _) ->
val index = if (different >= 0) different else 0 song.displayArtist !in artists && song.albumId !in albums
}
if (index < 0) {
index = remaining.indexOfFirst { (song, _) -> song.displayArtist !in artists }
}
if (index < 0) index = 0
val song = remaining.removeAt(index).first val song = remaining.removeAt(index).first
picked += song picked += song
lastArtist = song.displayArtist
artists.addLast(song.displayArtist)
if (artists.size > artistSpacing) artists.removeFirst()
albums.addLast(song.albumId)
if (albums.size > albumSpacing) albums.removeFirst()
} }
return picked return picked
} }
private companion object { private companion object {
const val POOL_LIMIT = 400 const val POOL_LIMIT = 400
const val RECENT_DAYS = 7L
const val RECENT_PENALTY = 6.0
} }
} }
@@ -53,6 +53,9 @@ import win.taptap.musicmanager.db.DatabaseHandle
import win.taptap.musicmanager.db.SongArtistsTable import win.taptap.musicmanager.db.SongArtistsTable
import win.taptap.musicmanager.db.SongGenresTable import win.taptap.musicmanager.db.SongGenresTable
import win.taptap.musicmanager.db.SongsTable import win.taptap.musicmanager.db.SongsTable
import win.taptap.musicmanager.discovery.DiscoveryService
import win.taptap.musicmanager.discovery.YtCandidate
import win.taptap.musicmanager.discovery.YtSearch
import win.taptap.musicmanager.radio.RadioService import win.taptap.musicmanager.radio.RadioService
import win.taptap.musicmanager.scanner.CoverArtService import win.taptap.musicmanager.scanner.CoverArtService
import win.taptap.musicmanager.scanner.ScanStatus import win.taptap.musicmanager.scanner.ScanStatus
@@ -240,6 +243,107 @@ class ApiRoutesTest {
} }
} }
@Test
fun `yt recommendations can be refreshed accepted and rejected`() {
seededContext().use { seed ->
val results = listOf(
YtCandidate(
videoId = "v-match",
title = "Soft Light",
artist = "Beta",
duration = 180,
thumbnail = "https://img.example/v-match.jpg",
),
YtCandidate("v-new", "Fresh Track", "New Artist", duration = 200),
YtCandidate("v-reject", "Loud", "Gamma", duration = 120),
YtCandidate("v-long", "Late Night Set", "Some DJ", duration = 3_600),
)
val search = YtSearch { _, _ -> results }
testApplication {
setupApp(seed, search)
val token = client.token("alice", "alice-pw")
val created = client.post("/api/v1/moods") {
bearer(token)
contentType(ContentType.Application.Json)
setBody(
"""{"name":"Focus","genres":["Ambient"],"seedSongIds":["${seed.aliceAmbient1}"]}""",
)
}
val moodId = parse(created.bodyAsText())["mood"]!!.jsonObject.string("id")
val refreshed = client.post("/api/v1/moods/$moodId/recommendations/refresh") {
bearer(token)
}
assertEquals(HttpStatusCode.OK, refreshed.status)
val refreshedBody = refreshed.bodyAsText()
val recommendations = parse(refreshedBody)["recommendations"]!!.jsonArray
.map { it.jsonObject }
// Long DJ sets are not music candidates.
assertFalse(recommendations.any { it.string("videoId") == "v-long" })
assertEquals(3, recommendations.size)
val match = recommendations.first { it.string("videoId") == "v-match" }
assertEquals(
seed.aliceAmbient2,
(match["matchedSong"] as? JsonObject)?.string("id"),
)
assertEquals("https://img.example/v-match.jpg", match.string("thumbnail"))
val unmatched = recommendations.first { it.string("videoId") == "v-new" }
assertTrue(unmatched["matchedSong"] !is JsonObject)
// A recommendation already in the library becomes a seed at once.
val accepted = client.post("/api/v1/moods/$moodId/recommendations/v-match/accept") {
bearer(token)
}
assertEquals(HttpStatusCode.OK, accepted.status)
val acceptedBody = parse(accepted.bodyAsText())
assertTrue(acceptedBody["seedAdded"]!!.jsonPrimitive.content.toBoolean())
assertEquals("accepted", acceptedBody["recommendation"]!!.jsonObject.string("state"))
assertEquals(2, acceptedBody["detail"]!!.jsonObject["mood"]!!.jsonObject.int("seedCount"))
// Unknown tracks are accepted for the future downloader.
val acceptedNew = client.post("/api/v1/moods/$moodId/recommendations/v-new/accept") {
bearer(token)
}
val acceptedNewBody = parse(acceptedNew.bodyAsText())
assertFalse(acceptedNewBody["seedAdded"]!!.jsonPrimitive.content.toBoolean())
assertEquals(
"accepted",
acceptedNewBody["recommendation"]!!.jsonObject.string("state"),
)
val rejected = client.post("/api/v1/moods/$moodId/recommendations/v-reject/reject") {
bearer(token)
}
assertEquals(
"rejected",
parse(rejected.bodyAsText())["recommendation"]!!.jsonObject.string("state"),
)
// Decisions survive a refresh: nothing is resurrected.
val again = client.post("/api/v1/moods/$moodId/recommendations/refresh") {
bearer(token)
}
val states = parse(again.bodyAsText())["recommendations"]!!.jsonArray
.map { it.jsonObject }
.associate { it.string("videoId") to it.string("state") }
assertEquals("accepted", states["v-match"])
assertEquals("accepted", states["v-new"])
assertEquals("rejected", states["v-reject"])
// Recommendations are private to the mood owner.
val bob = client.token("bob", "bob-pw")
val denied = client.post("/api/v1/moods/$moodId/recommendations/refresh") {
bearer(bob)
}
assertEquals(HttpStatusCode.NotFound, denied.status)
}
}
}
@Test @Test
fun `a session is scoped to its owner`() { fun `a session is scoped to its owner`() {
seededContext().use { seed -> seededContext().use { seed ->
@@ -267,7 +371,10 @@ class ApiRoutesTest {
} }
} }
private fun ApplicationTestBuilder.setupApp(seed: Seed) { private fun ApplicationTestBuilder.setupApp(
seed: Seed,
ytSearch: YtSearch = YtSearch { _, _ -> emptyList() },
) {
application { application {
install(ContentNegotiation) { install(ContentNegotiation) {
json(Json { ignoreUnknownKeys = true; encodeDefaults = true }) json(Json { ignoreUnknownKeys = true; encodeDefaults = true })
@@ -288,6 +395,7 @@ class ApiRoutesTest {
coverArtService = seed.coverArt, coverArtService = seed.coverArt,
transcoder = FfmpegTranscoder("ffmpeg"), transcoder = FfmpegTranscoder("ffmpeg"),
radioService = RadioService(), radioService = RadioService(),
discoveryService = DiscoveryService(ytSearch),
startScan = { false }, startScan = { false },
scanStatus = { ScanStatus(false, 0) }, scanStatus = { ScanStatus(false, 0) },
) )
@@ -0,0 +1,113 @@
package win.taptap.musicmanager.discovery
import java.time.Instant
import kotlin.test.Test
import kotlin.test.assertEquals
import kotlin.test.assertFalse
import kotlin.test.assertTrue
import win.taptap.musicmanager.db.MoodRecord
class DiscoveryServiceTest {
@Test
fun `parses flat playlist output and splits artist from title`() {
val json = """
{"id":"search","entries":[
{"id":"a","title":"Alpha - Focus One (Official Video)","duration":123.0,
"channel":"Alpha","thumbnails":[{"url":"http://t/1.jpg"},{"url":"http://t/2.jpg"}]},
{"id":"b","title":"Some Song","duration":61,"channel":"Some Artist - Topic"},
{"id":"c","title":"Loud | Official Audio","duration":90,"channel":"Gamma"}
]}
""".trimIndent()
val parsed = parseYtSearchOutput(json)
assertEquals(3, parsed.size)
assertEquals(
YtCandidate("a", "Focus One", "Alpha", null, 123, "http://t/2.jpg"),
parsed[0],
)
assertEquals(YtCandidate("b", "Some Song", "Some Artist", null, 61, null), parsed[1])
assertEquals(YtCandidate("c", "Loud", "Gamma", null, 90, null), parsed[2])
}
@Test
fun `malformed search output yields no candidates`() {
assertTrue(parseYtSearchOutput("").isEmpty())
assertTrue(parseYtSearchOutput("not json").isEmpty())
assertTrue(parseYtSearchOutput("""{"entries": "nope"}""").isEmpty())
}
@Test
fun `music length keeps songs and unknown durations but drops long mixes`() {
assertTrue(isMusicLength(null))
assertTrue(isMusicLength(45))
assertTrue(isMusicLength(200))
assertFalse(isMusicLength(30))
assertFalse(isMusicLength(21 * 60))
}
@Test
fun `match keys ignore brackets punctuation and featuring credits`() {
assertEquals(matchKey("Focus One"), matchKey("Focus One (Official Video)"))
assertEquals(matchKey("Focus One"), matchKey("Focus One [Remastered]"))
assertEquals(matchKey("Alpha"), matchKey("Alpha feat. Beta"))
assertTrue(artistMatches("Alpha - Topic", "Alpha"))
assertTrue(artistMatches("Alpha & Friends", "Alpha"))
assertFalse(artistMatches("Beta", "Alpha"))
}
@Test
fun `freshness follows the mood refresh window`() {
val now = Instant.parse("2026-11-15T10:00:00Z")
assertFalse(isFresh(null, 24, now))
assertTrue(isFresh("2026-11-15T01:00:00Z", 24, now))
assertFalse(isFresh("2026-11-13T01:00:00Z", 24, now))
}
@Test
fun `queries combine text seed artists and genres`() {
val mood = mood(query = "late night", genres = listOf("Ambient", "Jazz"))
val queries = buildQueries(mood, listOf("Alpha", "Beta", "Alpha"))
assertEquals(listOf("late night", "Alpha", "Beta", "Ambient mix"), queries)
}
@Test
fun `queries fall back to the mood name`() {
assertEquals(listOf("Focus"), buildQueries(mood(), emptyList()))
}
@Test
fun `seed artists and library matches score higher`() {
val candidate = YtCandidate("v", "Focus Two", "Beta", duration = 200)
val plain = scoreCandidate(candidate, listOf("Alpha"), "focus", matched = false)
val seedArtist = scoreCandidate(candidate, listOf("Beta"), "focus", matched = false)
val matched = scoreCandidate(candidate, listOf("Alpha"), "focus", matched = true)
assertTrue(seedArtist > plain)
assertTrue(matched > plain)
}
private fun mood(
query: String? = null,
genres: List<String> = emptyList(),
) = MoodRecord(
id = "mo-1",
owner = "alice",
name = "Focus",
description = null,
query = query,
genres = genres,
targetSize = 30,
refreshHours = 24,
ytEnabled = true,
lastDiscoveryAt = null,
createdAt = "2026-01-01T00:00:00Z",
changedAt = "2026-01-01T00:00:00Z",
lastError = null,
seedCount = 0,
newRecommendations = 0,
)
}
@@ -0,0 +1,137 @@
package win.taptap.musicmanager.radio
import kotlin.test.Test
import kotlin.test.assertEquals
import kotlin.test.assertTrue
import win.taptap.musicmanager.db.AffinityRecord
import win.taptap.musicmanager.db.SongRecord
class RadioServiceTest {
private val service = RadioService()
@Test
fun `recently played songs are penalized`() {
val song = song("s-1")
val fresh = service.scoreCandidate(
song, emptySet(), emptyList(), emptyList(), null, affinity = 0.0, jitter = 0.0,
)
val recent = service.scoreCandidate(
song, emptySet(), emptyList(), emptyList(), null, affinity = 0.0,
recent = setOf("s-1"), jitter = 0.0,
)
assertEquals(0.0, fresh)
assertEquals(-6.0, recent)
}
@Test
fun `skips outweigh completions`() {
assertEquals(1.0, AffinityRecord(completions = 1, skips = 0).score)
assertEquals(-2.0, AffinityRecord(completions = 1, skips = 1).score)
assertEquals(-5.0, AffinityRecord(completions = 1, skips = 2).score)
}
@Test
fun `seeds and matching genres are boosted`() {
val seeded = service.scoreCandidate(
song("s-1", genres = listOf("Ambient")),
seedIds = setOf("s-1"),
seedGenres = listOf("Ambient"),
seedArtistIds = emptyList(),
query = null,
affinity = 0.0,
jitter = 0.0,
)
val plain = service.scoreCandidate(
song("s-2"),
seedIds = setOf("s-1"),
seedGenres = listOf("Ambient"),
seedArtistIds = emptyList(),
query = null,
affinity = 0.0,
jitter = 0.0,
)
assertEquals(11.5, seeded)
assertEquals(0.0, plain)
}
@Test
fun `picks keep artists and albums apart`() {
val scored = listOf(
song("s1", artist = "A", albumId = "al1") to 0.0,
song("s2", artist = "B", albumId = "al2") to 0.0,
song("s3", artist = "C", albumId = "al3") to 0.0,
song("s4", artist = "D", albumId = "al4") to 0.0,
song("s5", artist = "A", albumId = "al5") to 0.0,
song("s6", artist = "B", albumId = "al6") to 0.0,
)
val picked = service.pickDiverse(scored, limit = 6)
assertEquals(6, picked.size)
picked.indices.forEach { index ->
val artists = picked.subList(index, minOf(index + 3, picked.size)).map { it.displayArtist }
assertEquals(artists.size, artists.distinct().size, "artist spacing at $index")
val albums = picked.subList(index, minOf(index + 4, picked.size)).map { it.albumId }
assertEquals(albums.size, albums.distinct().size, "album spacing at $index")
}
}
@Test
fun `picks relax spacing when the pool is too small`() {
val scored = listOf(
song("s1", artist = "A", albumId = "al1") to 0.0,
song("s2", artist = "A", albumId = "al1") to 0.0,
)
val picked = service.pickDiverse(scored, limit = 2)
assertEquals(listOf("s1", "s2"), picked.map { it.id })
}
@Test
fun `higher scores are picked first`() {
val scored = listOf(
song("low", artist = "A", albumId = "al1") to 1.0,
song("high", artist = "B", albumId = "al2") to 9.0,
)
assertTrue(service.pickDiverse(scored, limit = 2).first().id == "high")
}
private fun song(
id: String,
artist: String = "Artist $id",
albumId: String = "album-$id",
genres: List<String> = emptyList(),
) = SongRecord(
id = id,
albumId = albumId,
albumName = "Album",
artistId = "ar-$artist",
artistName = artist,
displayArtist = artist,
title = "Title $id",
trackNumber = 1,
discNumber = 1,
year = 2026,
duration = 180,
bitRate = 320,
size = 1_000,
suffix = "mp3",
contentType = "audio/mpeg",
coverArt = null,
folderId = 1,
relativePath = "$id.mp3",
created = "2026-01-01T00:00:00Z",
updatedAt = "2026-01-01T00:00:00Z",
starredAt = null,
rating = null,
playCount = 0,
lastPlayedAt = null,
artists = emptyList(),
genres = genres,
)
}