fix: harden scans, downloads, sync and radio edge cases
Scanner and library: - Keep the last library when a configured folder cannot be walked; the previous code deleted every song, cascading away mood seeds and progress. - Re-adding a file now clears its tombstone, and a re-deletion refreshes the timestamp instead of keeping the old one. - Tombstone timestamps are unique per row, and delta pages continue from the older of the song/tombstone page ends. - Snapshot pagination accepts a stable revision for later pages so songs inserted mid-sync are not skipped forever. - Reject "%" and "_" searches that normalize to an empty LIKE needle. Downloads and discovery: - Download promotion uses the deterministic path id, one folder per track so covers do not overwrite each other, removes failed promotions, and restarts the scan after a running one finishes. - Stale downloading jobs are recovered; reject cancels queued/in-flight jobs; markDone only owns downloading rows; rejecting no longer seeds the track. - Reuse an already completed download of the same video for another mood. - yt-dlp stdout and stderr are separated, empty discovery results no longer prune undecided recommendations, and filenames are capped by UTF-8 bytes. API and data: - Malformed or missing JSON returns 400 instead of 500; optional session bodies use receiveNullable. - Playback events must belong to the session, known song and open session. - Radio refills the candidate pool after recent-song pruning. - Seed percent is required in MoodRepository.update. - Normalize legacy V4 liked/downloaded recommendation states before V5, with an upgrade regression test. CI: check out without persisting credentials and require tests before chart publishing.
This commit is contained in:
1 parent
c75f2971b4
commit
4be502bb4e
23 files changed
+693
-54
No files matched your search
@@ -23,6 +23,8 @@ jobs:
|
||||
image: gitea.tap-tap.win/taptap/audiomanager-ci:2
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
persist-credentials: false
|
||||
- name: Test
|
||||
run: gradle test --console=plain
|
||||
- name: Smoke test
|
||||
@@ -34,11 +36,14 @@ jobs:
|
||||
image: gitea.tap-tap.win/taptap/audiomanager-ci:2
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
persist-credentials: false
|
||||
- name: Lint and render the chart
|
||||
run: scripts/check-chart.sh
|
||||
|
||||
chart-push:
|
||||
needs:
|
||||
- test
|
||||
- chart
|
||||
if: github.event_name == 'push'
|
||||
runs-on: ubuntu-latest
|
||||
@@ -50,6 +55,8 @@ jobs:
|
||||
CHARTS_TOKEN: ${{ secrets.AUDIOMANAGER_CHARTS_TOKEN }}
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
persist-credentials: false
|
||||
- name: Package and publish the chart
|
||||
run: scripts/push-chart.sh
|
||||
|
||||
|
||||
@@ -186,7 +186,7 @@ Song objects contain deterministic IDs (`sg-…`), album/artist names, duration,
|
||||
- `POST /moods/{id}/recommendations/{videoId}/accept` — accept; the library match becomes a seed immediately.
|
||||
- `POST /moods/{id}/recommendations/{videoId}/reject` — reject; the candidate is not offered again.
|
||||
|
||||
Accepted recommendations without a library match are queued for download (`download.state` = `queued|downloading|done|failed` in the recommendation). The worker writes MP3s into `<music folder>/AudioManager Imports`, embeds the video thumbnail as cover art, runs a scan, matches the new song, adds it as a seed and updates the recommendation.
|
||||
Accepted recommendations without a library match are queued for download (`download.state` = `queued|downloading|done|failed` in the recommendation). The worker writes MP3s into `<music folder>/AudioManager Imports/<artist>/<title>/`, embeds the video thumbnail as cover art, runs a scan, matches the new song by its deterministic path, adds it as a seed and updates the recommendation. The nested layout keeps per-track covers from overwriting each other.
|
||||
|
||||
### Radio sessions
|
||||
|
||||
@@ -299,7 +299,7 @@ helmCharts:
|
||||
- name: audiomanager
|
||||
repo: https://gitea.tap-tap.win/TapTap/Charts/raw/branch/main
|
||||
releaseName: audiomanager
|
||||
version: 0.2.0-sha.<commit>
|
||||
version: 0.2.4-sha.<commit>
|
||||
namespace: audiomanager
|
||||
valuesFile: values.yaml
|
||||
```
|
||||
@@ -315,7 +315,7 @@ The chart expects:
|
||||
- a `users.yaml` Secret (Vault-managed) referenced by `users.existingSecret`;
|
||||
- a writable music hostPath when automatic downloads are enabled. The chart runs a `prepare-imports` init container (`downloads.prepare`) that creates `<music>/AudioManager Imports` and gives it to UID 10001. If the host denies the write, jobs fail immediately with the underlying error instead of retrying;
|
||||
- a `library.yaml` ConfigMap;
|
||||
- read-only hostPaths for music and long-form;
|
||||
- a writable music hostPath for imports (or `downloads.enabled=false`) and a read-only long-form hostPath;
|
||||
- a registry pull secret for the private image.
|
||||
|
||||
Ingress host: `subsonic.tap-tap.win` (the historical name is kept, but the server no longer speaks the Subsonic protocol).
|
||||
@@ -335,7 +335,7 @@ The client lives in `TapTap/AudioManagerClient` and is built with Kotlin Multipl
|
||||
- **Desktop (JVM)**: shared Compose UI and a JVM audio backend.
|
||||
- **Web (Wasm)**: the same Compose UI, HTML audio, tokenized stream URLs.
|
||||
|
||||
The shared module owns the API client, auth store, SQLDelight offline database, mood radio queue, and platform player abstractions. Downloads are original MP3s from `/stream?download=true`. Web is streaming-only (the browser owns its cache).
|
||||
The shared module owns the API client, auth store, JSON offline cache, mood radio queue, and platform player abstractions. Downloads are original MP3s from `/stream?download=true`. Web is streaming-only (the browser owns its cache).
|
||||
|
||||
## Testing
|
||||
|
||||
|
||||
@@ -3,6 +3,8 @@ package win.taptap.musicmanager.api
|
||||
import io.ktor.http.HttpHeaders
|
||||
import io.ktor.http.HttpStatusCode
|
||||
import io.ktor.server.application.ApplicationCall
|
||||
import io.ktor.server.plugins.BadRequestException
|
||||
import io.ktor.server.plugins.ContentTransformationException
|
||||
import io.ktor.server.request.receive
|
||||
import io.ktor.server.response.respond
|
||||
import io.ktor.util.AttributeKey
|
||||
@@ -94,6 +96,10 @@ suspend inline fun <reified T : Any> ApplicationCall.receiveJson(): T =
|
||||
receive<T>()
|
||||
} catch (e: SerializationException) {
|
||||
throw ApiException(HttpStatusCode.BadRequest, "invalid_body", "Invalid JSON body")
|
||||
} catch (e: ContentTransformationException) {
|
||||
throw ApiException(HttpStatusCode.BadRequest, "invalid_body", "Invalid JSON body")
|
||||
} catch (e: BadRequestException) {
|
||||
throw ApiException(HttpStatusCode.BadRequest, "invalid_body", "Invalid JSON body")
|
||||
} catch (e: IllegalArgumentException) {
|
||||
throw ApiException(HttpStatusCode.BadRequest, "invalid_body", e.message ?: "Invalid body")
|
||||
}
|
||||
|
||||
@@ -43,17 +43,35 @@ fun Route.libraryRoutes(
|
||||
val now = Instant.now().toString()
|
||||
val since = call.request.queryParameters["since"]?.takeIf { it.isNotBlank() }
|
||||
val cursor = call.request.queryParameters["cursor"]?.takeIf { it.isNotBlank() }
|
||||
// Snapshot pagination must keep the timestamp of the first page. Using
|
||||
// each page's own time could hide a song inserted between pages, whose
|
||||
// id sorts before the cursor but whose updatedAt predates the final
|
||||
// revision (so no later delta would fetch it).
|
||||
val revision = call.request.queryParameters["revision"]?.takeIf { it.isNotBlank() } ?: now
|
||||
|
||||
if (since != null) {
|
||||
val changed = LibraryRepository.changedSongs(since, folders, limit, user.name)
|
||||
val deleted = LibraryRepository.tombstones(since, limit).map { it.entityId }
|
||||
val complete = changed.size < limit && deleted.size < limit
|
||||
val deletedRows = LibraryRepository.tombstones(since, limit)
|
||||
val deleted = deletedRows.map { it.entityId }
|
||||
val complete = changed.size < limit && deletedRows.size < limit
|
||||
|
||||
// Either list can fill a page. Continue from the older of the two
|
||||
// page ends, so a full tombstone page is not lost when fewer songs
|
||||
// changed. Re-reading a few rows is harmless; skipping them is not.
|
||||
val nextSince = when {
|
||||
changed.size == limit && deletedRows.size == limit ->
|
||||
listOf(changed.last().updatedAt, deletedRows.last().deletedAt).minOrNull()
|
||||
changed.size == limit -> changed.last().updatedAt
|
||||
deletedRows.size == limit -> deletedRows.last().deletedAt
|
||||
else -> null
|
||||
}
|
||||
|
||||
call.respond(
|
||||
LibraryPage(
|
||||
songs = changed.map { it.toDto() },
|
||||
deletedSongIds = deleted,
|
||||
revision = if (complete) now else changed.lastOrNull()?.updatedAt ?: now,
|
||||
nextSince = if (changed.size == limit) changed.last().updatedAt else null,
|
||||
revision = if (complete) now else nextSince ?: now,
|
||||
nextSince = nextSince,
|
||||
),
|
||||
)
|
||||
return@get
|
||||
@@ -63,7 +81,7 @@ fun Route.libraryRoutes(
|
||||
call.respond(
|
||||
LibraryPage(
|
||||
songs = songs.map { it.toDto() },
|
||||
revision = now,
|
||||
revision = revision,
|
||||
nextCursor = if (songs.size == limit) songs.last().id else null,
|
||||
),
|
||||
)
|
||||
|
||||
@@ -183,6 +183,9 @@ fun Route.moodRoutes(
|
||||
|
||||
val matched = recommendation.matchedSongId?.let { songId ->
|
||||
LibraryRepository.song(songId, user.musicFolderIds(library), user.name)
|
||||
} ?: DownloadRepository.completedSongForVideo(recommendation.videoId)?.let { songId ->
|
||||
// The same video may already have been downloaded for another mood.
|
||||
LibraryRepository.song(songId, user.musicFolderIds(library), user.name)
|
||||
}
|
||||
val seedAdded = matched != null &&
|
||||
MoodRepository.addSeed(mood.id, user.name, matched.id, "recommendation")
|
||||
@@ -208,6 +211,7 @@ fun Route.moodRoutes(
|
||||
val videoId = call.parameters["videoId"] ?: badRequest("Video id is required")
|
||||
MoodRepository.recommendation(mood.id, videoId) ?: notFound("Recommendation")
|
||||
MoodRepository.setRecommendationState(mood.id, videoId, "rejected")
|
||||
DownloadRepository.cancel(mood.id, videoId, "Recommendation rejected")
|
||||
call.respond(decision(mood.id, videoId, user, library, seedAdded = false))
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
package win.taptap.musicmanager.api
|
||||
|
||||
import io.ktor.http.HttpStatusCode
|
||||
import io.ktor.server.request.receiveNullable
|
||||
import io.ktor.server.response.respond
|
||||
import io.ktor.server.routing.Route
|
||||
import io.ktor.server.routing.delete
|
||||
@@ -37,8 +38,7 @@ fun Route.sessionRoutes(
|
||||
post("/moods/{id}/sessions") {
|
||||
val user = call.requireApiUser(users)
|
||||
val mood = ownedMoodFor(call.parameters["id"], user)
|
||||
val request = runCatching { call.receiveJson<StartSessionRequest>() }
|
||||
.getOrDefault(StartSessionRequest())
|
||||
val request = call.receiveNullable<StartSessionRequest>() ?: StartSessionRequest()
|
||||
val limit = request.limit ?: DEFAULT_BATCH
|
||||
|
||||
val started = radioService.start(mood, user, library, limit)
|
||||
@@ -81,8 +81,7 @@ fun Route.sessionRoutes(
|
||||
val session = ownedSession(call.parameters["id"], user)
|
||||
if (session.closedAt != null) conflict("Session is closed")
|
||||
val mood = ownedMoodFor(session.moodId, user)
|
||||
val request = runCatching { call.receiveJson<NextTracksRequest>() }
|
||||
.getOrDefault(NextTracksRequest())
|
||||
val request = call.receiveNullable<NextTracksRequest>() ?: NextTracksRequest()
|
||||
|
||||
val songs = radioService.next(
|
||||
mood = mood,
|
||||
|
||||
@@ -62,12 +62,33 @@ object DatabaseFactory {
|
||||
}
|
||||
}
|
||||
|
||||
normalizeLegacyMoodStates(dataSource)
|
||||
val database = Database.connect(dataSource)
|
||||
migrate(dataSource)
|
||||
|
||||
return DatabaseHandle(dataSource, database)
|
||||
}
|
||||
|
||||
/**
|
||||
* V4 allowed `liked` and `downloaded` recommendations, but its CHECK
|
||||
* constraint rejects the V5 `accepted` state, so an in-place rename is not
|
||||
* possible. Move those rows back to `new` (allowed by both schemas) before
|
||||
* Flyway runs. The table is absent on a fresh database and already uses the
|
||||
* new states after V5, so both are harmless no-ops.
|
||||
*/
|
||||
private fun normalizeLegacyMoodStates(dataSource: HikariDataSource) {
|
||||
dataSource.connection.use { connection ->
|
||||
runCatching {
|
||||
connection.createStatement().use { statement ->
|
||||
statement.executeUpdate(
|
||||
"UPDATE mood_recommendations " +
|
||||
"SET state = 'new' WHERE state IN ('liked', 'downloaded')",
|
||||
)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fun migrate(dataSource: HikariDataSource) {
|
||||
Flyway.configure()
|
||||
.dataSource(dataSource)
|
||||
@@ -75,4 +96,5 @@ object DatabaseFactory {
|
||||
.load()
|
||||
.migrate()
|
||||
}
|
||||
|
||||
}
|
||||
@@ -3,6 +3,7 @@ package win.taptap.musicmanager.db
|
||||
import java.time.Instant
|
||||
import org.jetbrains.exposed.sql.SortOrder
|
||||
import org.jetbrains.exposed.sql.SqlExpressionBuilder.eq
|
||||
import org.jetbrains.exposed.sql.SqlExpressionBuilder.less
|
||||
import org.jetbrains.exposed.sql.and
|
||||
import org.jetbrains.exposed.sql.insertIgnore
|
||||
import org.jetbrains.exposed.sql.selectAll
|
||||
@@ -73,6 +74,7 @@ object DownloadRepository {
|
||||
|
||||
/** Takes the oldest queued job and marks it as downloading. */
|
||||
fun claimNext(): DownloadJobRecord? = transaction {
|
||||
recoverStale()
|
||||
val row = DownloadJobsTable.selectAll()
|
||||
.where { DownloadJobsTable.state eq "queued" }
|
||||
.orderBy(DownloadJobsTable.createdAt to SortOrder.ASC)
|
||||
@@ -91,7 +93,12 @@ object DownloadRepository {
|
||||
|
||||
fun markDone(id: String, songId: String) {
|
||||
transaction {
|
||||
DownloadJobsTable.update({ DownloadJobsTable.id eq id }) {
|
||||
// Only finish a job that is still ours. A reject or recovery may
|
||||
// have changed the row while the downloader was running.
|
||||
DownloadJobsTable.update({
|
||||
(DownloadJobsTable.id eq id) and
|
||||
(DownloadJobsTable.state eq "downloading")
|
||||
}) {
|
||||
it[state] = "done"
|
||||
it[this.songId] = songId
|
||||
it[error] = null
|
||||
@@ -100,6 +107,53 @@ object DownloadRepository {
|
||||
}
|
||||
}
|
||||
|
||||
/** Rejects must stop a queued or in-flight download. */
|
||||
fun cancel(moodId: String, videoId: String, reason: String) {
|
||||
transaction {
|
||||
DownloadJobsTable.update({
|
||||
(DownloadJobsTable.moodId eq moodId) and
|
||||
(DownloadJobsTable.videoId eq videoId) and
|
||||
(DownloadJobsTable.state inList listOf("queued", "downloading"))
|
||||
}) {
|
||||
it[state] = "failed"
|
||||
it[error] = reason.take(1000)
|
||||
it[updatedAt] = Instant.now().toString()
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* A pod restart or an uncaught bookkeeping failure can leave jobs in
|
||||
* `downloading`. Requeue them after a generous timeout so they are not lost.
|
||||
*/
|
||||
private fun recoverStale(now: Instant = Instant.now()) {
|
||||
val cutoff = now.minusSeconds(STALE_DOWNLOAD_SECONDS).toString()
|
||||
val stale = DownloadJobsTable.selectAll()
|
||||
.where {
|
||||
(DownloadJobsTable.state eq "downloading") and
|
||||
(DownloadJobsTable.updatedAt less cutoff)
|
||||
}
|
||||
.toList()
|
||||
|
||||
stale.forEach { row ->
|
||||
val id = row[DownloadJobsTable.id]
|
||||
val attempts = row[DownloadJobsTable.attempts]
|
||||
if (attempts >= MAX_ATTEMPTS) {
|
||||
DownloadJobsTable.update({ DownloadJobsTable.id eq id }) {
|
||||
it[state] = "failed"
|
||||
it[error] = "Download was interrupted and could not be resumed"
|
||||
it[updatedAt] = now.toString()
|
||||
}
|
||||
} else {
|
||||
DownloadJobsTable.update({ DownloadJobsTable.id eq id }) {
|
||||
it[state] = "queued"
|
||||
it[error] = "Download was interrupted; retrying"
|
||||
it[updatedAt] = now.toString()
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fun markFailed(id: String, error: String) {
|
||||
transaction {
|
||||
DownloadJobsTable.update({ DownloadJobsTable.id eq id }) {
|
||||
@@ -136,6 +190,19 @@ object DownloadRepository {
|
||||
.associate { it[DownloadJobsTable.videoId] to it.toRecord() }
|
||||
}
|
||||
|
||||
/** A video already downloaded for any mood, if the current user can see it. */
|
||||
fun completedSongForVideo(videoId: String): String? = transaction {
|
||||
DownloadJobsTable.selectAll()
|
||||
.where {
|
||||
(DownloadJobsTable.videoId eq videoId) and
|
||||
(DownloadJobsTable.state eq "done")
|
||||
}
|
||||
.orderBy(DownloadJobsTable.updatedAt to SortOrder.DESC)
|
||||
.limit(1)
|
||||
.singleOrNull()
|
||||
?.get(DownloadJobsTable.songId)
|
||||
}
|
||||
|
||||
fun forRecommendation(userName: String, moodId: String, videoId: String): DownloadJobRecord? =
|
||||
transaction {
|
||||
DownloadJobsTable.selectAll()
|
||||
@@ -148,6 +215,9 @@ object DownloadRepository {
|
||||
?.toRecord()
|
||||
}
|
||||
|
||||
private const val STALE_DOWNLOAD_SECONDS = 30 * 60L
|
||||
private const val MAX_ATTEMPTS = 3
|
||||
|
||||
private fun org.jetbrains.exposed.sql.ResultRow.toRecord() = DownloadJobRecord(
|
||||
id = this[DownloadJobsTable.id],
|
||||
userName = this[DownloadJobsTable.userName],
|
||||
|
||||
@@ -658,6 +658,9 @@ object LibraryRepository {
|
||||
if (folderIds.isEmpty()) return@read emptyList()
|
||||
|
||||
val pattern = likePattern(query)
|
||||
// "%" or "_" alone normalizes to an empty LIKE needle and would match
|
||||
// the whole library.
|
||||
if (pattern == "%%") return@read emptyList()
|
||||
val rows = SongsTable.selectAll()
|
||||
.where {
|
||||
(SongsTable.searchText like pattern) and
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
package win.taptap.musicmanager.db
|
||||
|
||||
import java.time.Instant
|
||||
import org.jetbrains.exposed.sql.SqlExpressionBuilder.eq
|
||||
import org.jetbrains.exposed.sql.and
|
||||
import org.jetbrains.exposed.sql.deleteWhere
|
||||
import org.jetbrains.exposed.sql.insert
|
||||
@@ -138,6 +139,12 @@ object MediaRepository {
|
||||
}
|
||||
}
|
||||
|
||||
// A re-added song must not stay tombstoned for clients that still
|
||||
// remember the older revision.
|
||||
LibraryTombstonesTable.deleteWhere {
|
||||
it.run { (entity eq "song") and (entityId eq track.songId) }
|
||||
}
|
||||
|
||||
SongArtistsTable.deleteWhere { it.run { SongArtistsTable.songId eq track.songId } }
|
||||
track.artists.forEachIndexed { index, artist ->
|
||||
SongArtistsTable.insertIgnore {
|
||||
@@ -171,12 +178,23 @@ object MediaRepository {
|
||||
.map { it[SongsTable.id] }
|
||||
.filterNot { it in seenSongIds }
|
||||
|
||||
val now = Instant.now().toString()
|
||||
obsolete.forEach { songId ->
|
||||
val base = Instant.now()
|
||||
obsolete.forEachIndexed { index, songId ->
|
||||
// Unique timestamps let timestamp cursors advance even when a
|
||||
// scan deletes hundreds of songs at once.
|
||||
val timestamp = base.plusNanos(index.toLong()).toString()
|
||||
LibraryTombstonesTable.insertIgnore {
|
||||
it[entity] = "song"
|
||||
it[entityId] = songId
|
||||
it[deletedAt] = now
|
||||
it[deletedAt] = timestamp
|
||||
}
|
||||
// insertIgnore keeps the old timestamp on a re-deletion, which
|
||||
// would make the second deletion invisible to synced clients.
|
||||
LibraryTombstonesTable.update({
|
||||
(LibraryTombstonesTable.entity eq "song") and
|
||||
(LibraryTombstonesTable.entityId eq songId)
|
||||
}) {
|
||||
it[deletedAt] = timestamp
|
||||
}
|
||||
SongsTable.deleteWhere { it.run { SongsTable.id eq songId } }
|
||||
}
|
||||
|
||||
@@ -137,7 +137,7 @@ object MoodRepository {
|
||||
targetSize: Int,
|
||||
refreshHours: Int,
|
||||
ytEnabled: Boolean,
|
||||
seedPercent: Int = 50,
|
||||
seedPercent: Int,
|
||||
): MoodRecord? = transaction {
|
||||
val exists = MoodsTable.selectAll()
|
||||
.where { (MoodsTable.id eq id) and (MoodsTable.owner eq owner) }
|
||||
@@ -351,15 +351,18 @@ object MoodRepository {
|
||||
}
|
||||
|
||||
/** Links a downloaded recommendation to the scanned song and accepts it. */
|
||||
fun attachDownloadedSong(moodId: String, videoId: String, songId: String) {
|
||||
transaction {
|
||||
fun attachDownloadedSong(moodId: String, videoId: String, songId: String): Boolean = transaction {
|
||||
val row = MoodRecommendationsTable.selectAll()
|
||||
.where {
|
||||
(MoodRecommendationsTable.moodId eq moodId) and
|
||||
(MoodRecommendationsTable.videoId eq videoId)
|
||||
}
|
||||
.singleOrNull()
|
||||
?: return@transaction
|
||||
?: return@transaction false
|
||||
// The user may have rejected the track while it was downloading.
|
||||
if (row[MoodRecommendationsTable.state] == "rejected") {
|
||||
return@transaction false
|
||||
}
|
||||
MoodRecommendationsTable.update({
|
||||
(MoodRecommendationsTable.moodId eq moodId) and
|
||||
(MoodRecommendationsTable.videoId eq videoId)
|
||||
@@ -369,7 +372,7 @@ object MoodRepository {
|
||||
it[timesAdded] = row[MoodRecommendationsTable.timesAdded] + 1
|
||||
it[lastSeen] = Instant.now().toString()
|
||||
}
|
||||
}
|
||||
true
|
||||
}
|
||||
|
||||
fun markDiscovered(id: String) {
|
||||
|
||||
@@ -144,6 +144,17 @@ object RadioRepository {
|
||||
(PlaybackSessionsTable.userName eq userName)
|
||||
}
|
||||
.singleOrNull() ?: return@transaction false
|
||||
if (session[PlaybackSessionsTable.closedAt] != null) return@transaction false
|
||||
|
||||
// Only tracks the server actually queued in this session may change
|
||||
// affinity. Arbitrary/deleted ids used to violate the affinity FK.
|
||||
val queued = SessionItemsTable.selectAll()
|
||||
.where {
|
||||
(SessionItemsTable.sessionId eq sessionId) and
|
||||
(SessionItemsTable.songId eq songId)
|
||||
}
|
||||
.any()
|
||||
if (!queued) return@transaction false
|
||||
|
||||
val now = Instant.now().toString()
|
||||
SessionItemsTable.update({
|
||||
|
||||
@@ -91,6 +91,12 @@ class DiscoveryService(
|
||||
}
|
||||
}
|
||||
|
||||
if (candidates.isEmpty()) {
|
||||
val message = "YouTube returned no usable results"
|
||||
MoodRepository.markError(mood.id, message)
|
||||
return DiscoveryOutcome(0, 0, error = message)
|
||||
}
|
||||
|
||||
val known = MoodRepository.recommendations(mood.id).associateBy { it.videoId }
|
||||
val moodText = listOfNotNull(mood.query, mood.description, mood.name)
|
||||
.joinToString(" ")
|
||||
@@ -216,5 +222,8 @@ internal fun artistMatches(a: String, b: String): Boolean {
|
||||
val left = matchKey(a)
|
||||
val right = matchKey(b)
|
||||
if (left.isEmpty() || right.isEmpty()) return false
|
||||
return left == right || left.contains(right) || right.contains(left)
|
||||
if (left == right) return true
|
||||
// Short names like "A" used to match every artist containing the letter.
|
||||
if (left.length < 3 || right.length < 3) return false
|
||||
return " $left ".contains(" $right ") || " $right ".contains(" $left ")
|
||||
}
|
||||
@@ -44,24 +44,34 @@ class YtDlpSearch(
|
||||
"10",
|
||||
"ytsearch${limit.coerceIn(1, 25)}:$query",
|
||||
)
|
||||
.redirectErrorStream(true)
|
||||
// Warnings and ERROR lines must not be mixed into stdout: the JSON
|
||||
// parser would fail and look like an empty result set.
|
||||
.redirectErrorStream(false)
|
||||
.start()
|
||||
|
||||
val output = StringBuilder()
|
||||
val reader = thread(isDaemon = true, name = "yt-dlp-search") {
|
||||
val errors = StringBuilder()
|
||||
val reader = thread(isDaemon = true, name = "yt-dlp-search-stdout") {
|
||||
process.inputStream.bufferedReader().useLines { lines ->
|
||||
lines.forEach { output.appendLine(it) }
|
||||
}
|
||||
}
|
||||
val errorReader = thread(isDaemon = true, name = "yt-dlp-search-stderr") {
|
||||
process.errorStream.bufferedReader().useLines { lines ->
|
||||
lines.forEach { errors.appendLine(it) }
|
||||
}
|
||||
}
|
||||
|
||||
if (!process.waitFor(timeoutSeconds, TimeUnit.SECONDS)) {
|
||||
process.destroyForcibly()
|
||||
reader.join(2_000)
|
||||
errorReader.join(2_000)
|
||||
throw IllegalStateException("yt-dlp timed out after ${timeoutSeconds}s")
|
||||
}
|
||||
reader.join(2_000)
|
||||
errorReader.join(2_000)
|
||||
if (process.exitValue() != 0) {
|
||||
val detail = output.toString().take(400).trim()
|
||||
val detail = (errors.toString().ifBlank { output.toString() }).take(400).trim()
|
||||
throw IllegalStateException("yt-dlp exited with ${process.exitValue()}: $detail")
|
||||
}
|
||||
return parseYtSearchOutput(output.toString())
|
||||
|
||||
@@ -16,6 +16,7 @@ import win.taptap.musicmanager.db.LibraryRepository
|
||||
import win.taptap.musicmanager.db.MoodRepository
|
||||
import win.taptap.musicmanager.discovery.artistMatches
|
||||
import win.taptap.musicmanager.discovery.matchKey
|
||||
import win.taptap.musicmanager.util.IdGenerator
|
||||
|
||||
fun interface AudioDownloader {
|
||||
/** Downloads [videoId] and returns the resulting MP3 file. */
|
||||
@@ -72,10 +73,17 @@ class DownloadService(
|
||||
val working = tempDir.resolve(job.id)
|
||||
Files.createDirectories(working)
|
||||
|
||||
var promoted: Path? = null
|
||||
try {
|
||||
val downloaded = downloader.download(job.videoId, job.title, job.artist, working)
|
||||
val extension = downloaded.fileName.toString().substringAfterLast('.', "mp3")
|
||||
val target = uniqueTarget(importDir, job.artist, job.title, extension)
|
||||
// One folder per artist/title keeps the scanner's album fallback
|
||||
// unique, so two downloads never overwrite each other's cover.
|
||||
val targetDir = importDir
|
||||
.resolve(sanitize(job.artist))
|
||||
.resolve(sanitize(job.title))
|
||||
Files.createDirectories(targetDir)
|
||||
val target = uniqueTarget(targetDir, job.title, extension)
|
||||
try {
|
||||
Files.move(downloaded, target, StandardCopyOption.REPLACE_EXISTING)
|
||||
} catch (e: IOException) {
|
||||
@@ -84,15 +92,33 @@ class DownloadService(
|
||||
e,
|
||||
)
|
||||
}
|
||||
promoted = target
|
||||
log.info("Downloaded {} to {}", job.videoId, target)
|
||||
|
||||
awaitScan()
|
||||
|
||||
val songId = matchSong(job, folders.keys, user.name)
|
||||
// The scanner derives the song id from the folder and relative
|
||||
// path. Prefer that exact id: metadata matching could otherwise
|
||||
// attach the download to an older track with the same title.
|
||||
val relativePath = folder.path.relativize(target).joinToString("/") { it.toString() }
|
||||
val expectedId = IdGenerator.songId(folder.id, relativePath)
|
||||
val songId = LibraryRepository.song(expectedId, folders.keys, user.name)?.id
|
||||
?: matchSong(job, folders.keys, user.name)
|
||||
?: throw IllegalStateException("Downloaded file was not indexed by the scanner")
|
||||
DownloadRepository.markDone(job.id, songId)
|
||||
MoodRepository.attachDownloadedSong(job.moodId, job.videoId, songId)
|
||||
val attached = MoodRepository.attachDownloadedSong(job.moodId, job.videoId, songId)
|
||||
if (attached) {
|
||||
MoodRepository.addSeed(job.moodId, job.userName, songId, "yt")
|
||||
promoted = null
|
||||
} else {
|
||||
log.info("Recommendation {} was rejected; keeping the file but not seeding", job.videoId)
|
||||
promoted = null
|
||||
DownloadRepository.cancel(job.moodId, job.videoId, "Recommendation rejected")
|
||||
}
|
||||
} catch (t: Throwable) {
|
||||
// A failed verification must not leave a duplicate for the retry.
|
||||
promoted?.let { target -> runCatching { Files.deleteIfExists(target) } }
|
||||
throw t
|
||||
} finally {
|
||||
runCatching { working.toFile().deleteRecursively() }
|
||||
}
|
||||
@@ -115,9 +141,23 @@ class DownloadService(
|
||||
}
|
||||
|
||||
private fun awaitScan() {
|
||||
// If another scan is already running, wait for that one instead.
|
||||
startScan()
|
||||
// The downloaded file must be covered by a scan that starts *after* it
|
||||
// was moved. If a scan is already running, wait for it and then start a
|
||||
// fresh one; otherwise a slow scan could finish without seeing the file
|
||||
// and the job would download the same track again on retry.
|
||||
val deadline = System.nanoTime() + SCAN_TIMEOUT_MINUTES * 60_000_000_000L
|
||||
var attempts = 0
|
||||
while (!startScan()) {
|
||||
attempts += 1
|
||||
if (attempts >= MAX_SCAN_START_ATTEMPTS) {
|
||||
throw IllegalStateException("Could not start a library scan")
|
||||
}
|
||||
awaitScanToFinish(deadline)
|
||||
}
|
||||
awaitScanToFinish(deadline)
|
||||
}
|
||||
|
||||
private fun awaitScanToFinish(deadline: Long) {
|
||||
while (isScanning()) {
|
||||
if (System.nanoTime() > deadline) {
|
||||
throw IllegalStateException("Library scan did not finish in time")
|
||||
@@ -136,8 +176,8 @@ class DownloadService(
|
||||
?.id
|
||||
}
|
||||
|
||||
private fun uniqueTarget(dir: Path, artist: String, title: String, extension: String): Path {
|
||||
val base = "${sanitize(artist)} - ${sanitize(title)}"
|
||||
private fun uniqueTarget(dir: Path, title: String, extension: String): Path {
|
||||
val base = sanitize(title)
|
||||
val suffix = extension.ifBlank { "mp3" }.take(10)
|
||||
var candidate = dir.resolve("$base.$suffix")
|
||||
var counter = 2
|
||||
@@ -171,14 +211,31 @@ class DownloadService(
|
||||
}
|
||||
|
||||
private fun sanitize(value: String): String =
|
||||
truncateUtf8(
|
||||
value.replace(Regex("[\\\\/:*?\"<>|\\p{Cntrl}]"), "_")
|
||||
.replace(Regex("\\s+"), " ")
|
||||
.trim()
|
||||
.take(120)
|
||||
.ifEmpty { "track" }
|
||||
.ifEmpty { "track" },
|
||||
MAX_NAME_BYTES,
|
||||
)
|
||||
|
||||
/** Linux NAME_MAX is 255 bytes, not characters; CJK titles are 3x. */
|
||||
private fun truncateUtf8(value: String, maxBytes: Int): String {
|
||||
var bytes = 0
|
||||
val result = StringBuilder()
|
||||
for (character in value) {
|
||||
val length = character.toString().toByteArray(Charsets.UTF_8).size
|
||||
if (bytes + length > maxBytes) break
|
||||
result.append(character)
|
||||
bytes += length
|
||||
}
|
||||
return result.toString().ifEmpty { "track" }
|
||||
}
|
||||
|
||||
private companion object {
|
||||
const val IMPORT_DIR = "AudioManager Imports"
|
||||
const val SCAN_TIMEOUT_MINUTES = 10L
|
||||
const val MAX_SCAN_START_ATTEMPTS = 5
|
||||
const val MAX_NAME_BYTES = 120
|
||||
}
|
||||
}
|
||||
@@ -36,25 +36,34 @@ class YtDlpDownloader(
|
||||
targetDir.resolve("%(id)s.%(ext)s").toString(),
|
||||
"https://www.youtube.com/watch?v=$videoId",
|
||||
)
|
||||
.redirectErrorStream(true)
|
||||
.redirectErrorStream(false)
|
||||
.start()
|
||||
|
||||
val output = StringBuilder()
|
||||
val reader = thread(isDaemon = true, name = "yt-dlp-download") {
|
||||
val errors = StringBuilder()
|
||||
val reader = thread(isDaemon = true, name = "yt-dlp-download-stdout") {
|
||||
process.inputStream.bufferedReader().useLines { lines ->
|
||||
lines.forEach { output.appendLine(it) }
|
||||
}
|
||||
}
|
||||
val errorReader = thread(isDaemon = true, name = "yt-dlp-download-stderr") {
|
||||
process.errorStream.bufferedReader().useLines { lines ->
|
||||
lines.forEach { errors.appendLine(it) }
|
||||
}
|
||||
}
|
||||
|
||||
if (!process.waitFor(timeoutSeconds, TimeUnit.SECONDS)) {
|
||||
process.destroyForcibly()
|
||||
reader.join(2_000)
|
||||
errorReader.join(2_000)
|
||||
throw IllegalStateException("yt-dlp download timed out after ${timeoutSeconds}s")
|
||||
}
|
||||
reader.join(2_000)
|
||||
errorReader.join(2_000)
|
||||
if (process.exitValue() != 0) {
|
||||
val detail = (errors.toString().ifBlank { output.toString() }).take(400).trim()
|
||||
throw IllegalStateException(
|
||||
"yt-dlp exited with ${process.exitValue()}: ${output.toString().take(400).trim()}",
|
||||
"yt-dlp exited with ${process.exitValue()}: $detail",
|
||||
)
|
||||
}
|
||||
|
||||
|
||||
@@ -129,18 +129,7 @@ class RadioService(
|
||||
// Small libraries can exhaust the mood. Replay tracks the session already
|
||||
// heard instead of returning silence, but never repeat the current batch.
|
||||
if (candidates.size < limit) {
|
||||
LibraryRepository.randomSongs(
|
||||
size = max(limit * 10, 200),
|
||||
genre = null,
|
||||
fromYear = null,
|
||||
toYear = null,
|
||||
folderIds = folders,
|
||||
userName = user.name,
|
||||
).forEach { song ->
|
||||
if (song.id !in excluded) {
|
||||
candidates.putIfAbsent(song.id, song)
|
||||
}
|
||||
}
|
||||
refill(candidates, folders, user.name, excluded, sessionSongs, emptySet(), limit)
|
||||
}
|
||||
|
||||
val recent = RadioRepository.recentSongIds(
|
||||
@@ -152,6 +141,14 @@ class RadioService(
|
||||
if (candidates.size > limit * 3) {
|
||||
candidates.keys.removeAll(recent - seedIds)
|
||||
}
|
||||
// Pruning can empty the pool even though the library has songs. Prefer
|
||||
// non-recent tracks, then fall back to replay if the library is tiny.
|
||||
if (candidates.size < limit) {
|
||||
refill(candidates, folders, user.name, excluded, sessionSongs, recent, limit)
|
||||
if (candidates.size < limit) {
|
||||
refill(candidates, folders, user.name, excluded, emptySet(), emptySet(), limit)
|
||||
}
|
||||
}
|
||||
|
||||
val affinity = RadioRepository.affinity(user.name, candidates.keys)
|
||||
val scored = candidates.values.map { song ->
|
||||
@@ -181,6 +178,29 @@ class RadioService(
|
||||
return picks
|
||||
}
|
||||
|
||||
private fun refill(
|
||||
candidates: LinkedHashMap<String, SongRecord>,
|
||||
folders: Set<Int>,
|
||||
userName: String,
|
||||
excluded: Set<String>,
|
||||
avoidSession: Set<String>,
|
||||
avoidRecent: Set<String>,
|
||||
limit: Int,
|
||||
) {
|
||||
LibraryRepository.randomSongs(
|
||||
size = max(limit * 10, 200),
|
||||
genre = null,
|
||||
fromYear = null,
|
||||
toYear = null,
|
||||
folderIds = folders,
|
||||
userName = userName,
|
||||
).forEach { song ->
|
||||
if (song.id !in excluded && song.id !in avoidSession && song.id !in avoidRecent) {
|
||||
candidates.putIfAbsent(song.id, song)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
internal fun scoreCandidate(
|
||||
song: SongRecord,
|
||||
seedIds: Set<String>,
|
||||
|
||||
@@ -49,6 +49,7 @@ class MediaScanner(
|
||||
var count = 0L
|
||||
val seenSongIds = mutableSetOf<String>()
|
||||
var error: String? = null
|
||||
var folderFailed = false
|
||||
|
||||
try {
|
||||
library.folders.forEach { folder ->
|
||||
@@ -58,11 +59,22 @@ class MediaScanner(
|
||||
status.set(ScanStatus(scanning = true, count = count))
|
||||
}
|
||||
} catch (e: Exception) {
|
||||
folderFailed = true
|
||||
if (error == null) {
|
||||
error = "Could not scan folder ${folder.id} (${folder.path}): " +
|
||||
(e.message ?: e::class.simpleName)
|
||||
}
|
||||
log.error("Could not scan folder ${folder.id} (${folder.path})", e)
|
||||
}
|
||||
}
|
||||
|
||||
if (folderFailed) {
|
||||
// An unmounted or temporarily unreadable folder must not be
|
||||
// mistaken for deleted files. Keep the last known library.
|
||||
log.warn("Skipping orphan cleanup because at least one folder failed to scan")
|
||||
} else {
|
||||
repository.deleteSongsNotIn(seenSongIds)
|
||||
}
|
||||
repository.rebuildAggregates()
|
||||
} catch (t: Throwable) {
|
||||
error = t.message ?: t::class.simpleName
|
||||
|
||||
@@ -53,6 +53,7 @@ import win.taptap.musicmanager.db.ConfigRepository
|
||||
import win.taptap.musicmanager.db.DatabaseFactory
|
||||
import win.taptap.musicmanager.db.DatabaseHandle
|
||||
import win.taptap.musicmanager.db.DownloadRepository
|
||||
import win.taptap.musicmanager.db.LibraryTombstonesTable
|
||||
import win.taptap.musicmanager.db.SongArtistsTable
|
||||
import win.taptap.musicmanager.db.SongGenresTable
|
||||
import win.taptap.musicmanager.db.SongsTable
|
||||
@@ -81,6 +82,13 @@ class ApiRoutesTest {
|
||||
assertEquals("alice", ok["user"]!!.jsonObject.string("name"))
|
||||
assertEquals(listOf(1), ok["user"]!!.jsonObject["musicFolders"]!!.jsonArray.map { it.jsonPrimitive.int })
|
||||
|
||||
val malformed = client.post("/api/v1/moods") {
|
||||
bearer(parse(client.login("alice", "alice-pw").bodyAsText()).string("token"))
|
||||
contentType(ContentType.Application.Json)
|
||||
setBody("not-json")
|
||||
}
|
||||
assertEquals(HttpStatusCode.BadRequest, malformed.status)
|
||||
|
||||
val bad = client.login("alice", "wrong")
|
||||
assertEquals(HttpStatusCode.Unauthorized, bad.status)
|
||||
assertEquals("invalid_credentials", parse(bad.bodyAsText())["error"]!!.jsonObject.string("code"))
|
||||
@@ -110,6 +118,15 @@ class ApiRoutesTest {
|
||||
assertFalse(seed.bobSong in ids)
|
||||
assertTrue(page.string("revision").isNotBlank())
|
||||
|
||||
// A snapshot revision is echoed so multi-page syncs share one cutoff.
|
||||
val snapshot = "2026-01-01T00:00:00Z"
|
||||
val paged = parseBody(
|
||||
client.get("/api/v1/library/songs?scope=music&limit=1&revision=$snapshot") {
|
||||
bearer(token)
|
||||
}.bodyAsText(),
|
||||
)
|
||||
assertEquals(snapshot, paged.string("revision"))
|
||||
|
||||
val longform = parseBody(
|
||||
client.get("/api/v1/library/songs?scope=longform") {
|
||||
bearer(token)
|
||||
@@ -131,6 +148,54 @@ class ApiRoutesTest {
|
||||
|
||||
val forbidden = client.get("/api/v1/songs/${seed.bobSong}/stream?token=$token")
|
||||
assertEquals(HttpStatusCode.NotFound, forbidden.status)
|
||||
|
||||
// A wildcard-only search must not return the whole library.
|
||||
val wildcard = parseBody(
|
||||
client.get("/api/v1/library/search?q=%25&scope=music") { bearer(token) }.bodyAsText(),
|
||||
)
|
||||
assertTrue(wildcard["songs"]!!.jsonArray.isEmpty())
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@Test
|
||||
fun `delta sync continues when only tombstones fill the page`() {
|
||||
seededContext().use { seed ->
|
||||
testApplication {
|
||||
setupApp(seed)
|
||||
val token = client.token("alice", "alice-pw")
|
||||
val since = Instant.now().plusSeconds(1)
|
||||
transaction(seed.db.database) {
|
||||
listOf("gone-1" to 2L, "gone-2" to 3L).forEach { (id, offset) ->
|
||||
LibraryTombstonesTable.insert {
|
||||
it[entity] = "song"
|
||||
it[entityId] = id
|
||||
it[deletedAt] = since.plusSeconds(offset).toString()
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
val first = parseBody(
|
||||
client.get("/api/v1/library/songs?limit=1&since=$since") {
|
||||
bearer(token)
|
||||
}.bodyAsText(),
|
||||
)
|
||||
assertEquals(
|
||||
listOf("gone-1"),
|
||||
first["deletedSongIds"]!!.jsonArray.map { it.jsonPrimitive.content },
|
||||
)
|
||||
val next = first["nextSince"]!!.jsonPrimitive.content
|
||||
|
||||
val second = parseBody(
|
||||
client.get("/api/v1/library/songs?limit=1&since=$next") {
|
||||
bearer(token)
|
||||
}.bodyAsText(),
|
||||
)
|
||||
assertEquals(
|
||||
listOf("gone-2"),
|
||||
second["deletedSongIds"]!!.jsonArray.map { it.jsonPrimitive.content },
|
||||
)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -189,6 +254,22 @@ class ApiRoutesTest {
|
||||
assertFalse(queue.any { it.string("id") == seed.bobSong })
|
||||
assertTrue(queue.all { it.string("folderId") == "1" })
|
||||
|
||||
// Unknown ids must not break event recording with a foreign-key error.
|
||||
val unknownEvent = client.post("/api/v1/sessions/$sessionId/events") {
|
||||
bearer(token)
|
||||
contentType(ContentType.Application.Json)
|
||||
setBody("""{"events":[{"songId":"sg-does-not-exist","type":"completed"}]}""")
|
||||
}
|
||||
assertEquals(HttpStatusCode.OK, unknownEvent.status)
|
||||
assertEquals(0, parse(unknownEvent.bodyAsText()).int("accepted"))
|
||||
|
||||
val badNext = client.post("/api/v1/sessions/$sessionId/next") {
|
||||
bearer(token)
|
||||
contentType(ContentType.Application.Json)
|
||||
setBody("not-json")
|
||||
}
|
||||
assertEquals(HttpStatusCode.BadRequest, badNext.status)
|
||||
|
||||
val second = client.post("/api/v1/sessions/$sessionId/next") {
|
||||
bearer(token)
|
||||
contentType(ContentType.Application.Json)
|
||||
|
||||
@@ -2,6 +2,9 @@ package win.taptap.musicmanager.db
|
||||
|
||||
import java.nio.file.Files
|
||||
import java.nio.file.Path
|
||||
import java.sql.DriverManager
|
||||
import org.flywaydb.core.Flyway
|
||||
import org.flywaydb.core.api.MigrationVersion
|
||||
import java.time.Instant
|
||||
import kotlin.test.Test
|
||||
import kotlin.test.assertEquals
|
||||
@@ -43,6 +46,58 @@ class DatabaseFactoryTest {
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@Test
|
||||
fun `upgrades legacy liked and downloaded recommendations`() {
|
||||
val config = appConfig()
|
||||
Files.createDirectories(config.dataDir)
|
||||
val url = "jdbc:sqlite:${config.databasePath.toAbsolutePath()}"
|
||||
|
||||
Flyway.configure()
|
||||
.dataSource(url, null, null)
|
||||
.locations("classpath:db/migration")
|
||||
.target(MigrationVersion.fromVersion("4"))
|
||||
.load()
|
||||
.migrate()
|
||||
|
||||
DriverManager.getConnection(url).use { connection ->
|
||||
connection.createStatement().use { statement ->
|
||||
statement.executeUpdate(
|
||||
"INSERT INTO users (name, is_admin, updated_at) VALUES ('alice', 1, 'now')",
|
||||
)
|
||||
statement.executeUpdate(
|
||||
"INSERT INTO playlists (id, owner, name, created_at, changed_at) " +
|
||||
"VALUES ('pl-1', 'alice', 'Mood', 'now', 'now')",
|
||||
)
|
||||
statement.executeUpdate(
|
||||
"INSERT INTO moods (playlist_id) VALUES ('pl-1')",
|
||||
)
|
||||
statement.executeUpdate(
|
||||
"INSERT INTO mood_recommendations " +
|
||||
"(playlist_id, video_id, title, artist, state, first_seen, last_seen) " +
|
||||
"VALUES ('pl-1', 'v-1', 'Title', 'Artist', 'liked', 'now', 'now')",
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
val db = DatabaseFactory.connect(config)
|
||||
try {
|
||||
val state = db.dataSource.connection.use { connection ->
|
||||
connection.createStatement().use { statement ->
|
||||
statement.executeQuery(
|
||||
"SELECT state FROM mood_recommendations WHERE video_id = 'v-1'",
|
||||
).use { result ->
|
||||
result.next()
|
||||
result.getString(1)
|
||||
}
|
||||
}
|
||||
}
|
||||
assertEquals("new", state)
|
||||
} finally {
|
||||
db.close()
|
||||
}
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `syncs folders and users idempotently`() {
|
||||
val db = DatabaseFactory.connect(appConfig())
|
||||
|
||||
@@ -55,6 +55,7 @@ class DiscoveryServiceTest {
|
||||
assertTrue(artistMatches("Alpha - Topic", "Alpha"))
|
||||
assertTrue(artistMatches("Alpha & Friends", "Alpha"))
|
||||
assertFalse(artistMatches("Beta", "Alpha"))
|
||||
assertFalse(artistMatches("A", "ABBA"))
|
||||
}
|
||||
|
||||
@Test
|
||||
|
||||
@@ -6,10 +6,13 @@ import java.nio.file.Path
|
||||
import java.time.Instant
|
||||
import kotlin.test.Test
|
||||
import kotlin.test.assertEquals
|
||||
import kotlin.test.assertNotEquals
|
||||
import kotlin.test.assertNotNull
|
||||
import kotlin.test.assertTrue
|
||||
import org.jetbrains.exposed.sql.SqlExpressionBuilder.eq
|
||||
import org.jetbrains.exposed.sql.insert
|
||||
import org.jetbrains.exposed.sql.insertIgnore
|
||||
import org.jetbrains.exposed.sql.update
|
||||
import org.jetbrains.exposed.sql.transactions.transaction
|
||||
import org.junit.jupiter.api.io.TempDir
|
||||
import win.taptap.musicmanager.config.AppConfig
|
||||
@@ -24,6 +27,7 @@ import win.taptap.musicmanager.db.ArtistsTable
|
||||
import win.taptap.musicmanager.db.ConfigRepository
|
||||
import win.taptap.musicmanager.db.DatabaseFactory
|
||||
import win.taptap.musicmanager.db.DatabaseHandle
|
||||
import win.taptap.musicmanager.db.DownloadJobsTable
|
||||
import win.taptap.musicmanager.db.DownloadRepository
|
||||
import win.taptap.musicmanager.db.MoodRepository
|
||||
import win.taptap.musicmanager.db.RecommendationUpsert
|
||||
@@ -100,9 +104,84 @@ class DownloadServiceTest {
|
||||
assertTrue(MoodRepository.seedSongIds(mood.id).contains(songId))
|
||||
|
||||
val imports = library.folders.first().path.resolve("AudioManager Imports")
|
||||
val files = Files.list(imports).use { stream -> stream.toList() }
|
||||
val files = Files.walk(imports).use { stream ->
|
||||
stream.filter { Files.isRegularFile(it) }.toList()
|
||||
}
|
||||
assertEquals(1, files.size)
|
||||
assertEquals("Die drei ___ - Neue Folge.mp3", files.first().fileName.toString())
|
||||
assertEquals("Neue Folge.mp3", files.first().fileName.toString())
|
||||
assertEquals("Neue Folge", files.first().parent.fileName.toString())
|
||||
assertEquals("Die drei ___", files.first().parent.parent.fileName.toString())
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@Test
|
||||
fun `downloaded file is promoted instead of an older same-title song`() {
|
||||
withDatabase { handle, library, users ->
|
||||
val mood = MoodRepository.create(
|
||||
owner = "alice",
|
||||
name = "Drei ???",
|
||||
description = null,
|
||||
query = null,
|
||||
genres = emptyList(),
|
||||
targetSize = 30,
|
||||
refreshHours = 24,
|
||||
ytEnabled = true,
|
||||
seedSongIds = emptyList(),
|
||||
)
|
||||
MoodRepository.storeRecommendations(
|
||||
mood.id,
|
||||
listOf(
|
||||
RecommendationUpsert(
|
||||
videoId = "v-dup",
|
||||
title = "Neue Folge",
|
||||
artist = "Die drei ???",
|
||||
album = null,
|
||||
duration = 300,
|
||||
thumbnail = null,
|
||||
matchedSongId = null,
|
||||
),
|
||||
),
|
||||
"2026-01-01T00:00:00Z",
|
||||
)
|
||||
DownloadRepository.enqueue("alice", mood.id, "v-dup", "Neue Folge", "Die drei ???")
|
||||
|
||||
val oldId = insertSong(
|
||||
handle,
|
||||
"Die drei ???",
|
||||
"Neue Folge",
|
||||
"Old/Die drei ??? - Neue Folge.mp3",
|
||||
)
|
||||
var newId: String? = null
|
||||
val service = DownloadService(
|
||||
users = users,
|
||||
library = library,
|
||||
downloader = AudioDownloader { videoId, _, _, dir ->
|
||||
val file = dir.resolve("$videoId.mp3")
|
||||
Files.write(file, "fake-mp3".toByteArray())
|
||||
file
|
||||
},
|
||||
tempDir = tempDir.resolve("downloads"),
|
||||
startScan = {
|
||||
if (newId == null) {
|
||||
newId = insertSong(
|
||||
handle,
|
||||
"Die drei ???",
|
||||
"Neue Folge",
|
||||
"AudioManager Imports/Die drei ___/Neue Folge/Neue Folge.mp3",
|
||||
)
|
||||
}
|
||||
true
|
||||
},
|
||||
isScanning = { false },
|
||||
)
|
||||
|
||||
assertTrue(service.processOne())
|
||||
|
||||
val job = assertNotNull(DownloadRepository.forRecommendation("alice", mood.id, "v-dup"))
|
||||
assertEquals("done", job.state)
|
||||
assertEquals(newId, job.songId)
|
||||
assertNotEquals(oldId, job.songId)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -144,6 +223,78 @@ class DownloadServiceTest {
|
||||
}
|
||||
|
||||
|
||||
|
||||
@Test
|
||||
fun `stale downloading jobs are recovered`() {
|
||||
withDatabase { handle, library, users ->
|
||||
val mood = MoodRepository.create(
|
||||
owner = "alice",
|
||||
name = "Recovery",
|
||||
description = null,
|
||||
query = null,
|
||||
genres = emptyList(),
|
||||
targetSize = 30,
|
||||
refreshHours = 24,
|
||||
ytEnabled = true,
|
||||
seedSongIds = emptyList(),
|
||||
)
|
||||
DownloadRepository.enqueue("alice", mood.id, "v-stale", "Stale", "Nobody")
|
||||
val job = assertNotNull(DownloadRepository.forRecommendation("alice", mood.id, "v-stale"))
|
||||
transaction(handle.database) {
|
||||
DownloadJobsTable.update({ DownloadJobsTable.id eq job.id }) {
|
||||
it[state] = "downloading"
|
||||
it[updatedAt] = Instant.now().minusSeconds(3600).toString()
|
||||
}
|
||||
}
|
||||
|
||||
val claimed = assertNotNull(DownloadRepository.claimNext())
|
||||
assertEquals(job.id, claimed.id)
|
||||
assertEquals("downloading", claimed.state)
|
||||
}
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `failed verification removes the promoted file instead of duplicating it`() {
|
||||
withDatabase { _, library, users ->
|
||||
val mood = MoodRepository.create(
|
||||
owner = "alice",
|
||||
name = "Broken match",
|
||||
description = null,
|
||||
query = null,
|
||||
genres = emptyList(),
|
||||
targetSize = 30,
|
||||
refreshHours = 24,
|
||||
ytEnabled = true,
|
||||
seedSongIds = emptyList(),
|
||||
)
|
||||
DownloadRepository.enqueue("alice", mood.id, "v-nomatch", "No Match", "Nobody")
|
||||
val service = DownloadService(
|
||||
users = users,
|
||||
library = library,
|
||||
downloader = AudioDownloader { videoId, _, _, dir ->
|
||||
val file = dir.resolve("$videoId.mp3")
|
||||
Files.write(file, "fake-mp3".toByteArray())
|
||||
file
|
||||
},
|
||||
tempDir = tempDir.resolve("downloads"),
|
||||
startScan = { true },
|
||||
isScanning = { false },
|
||||
)
|
||||
|
||||
repeat(3) { assertTrue(service.processOne()) }
|
||||
|
||||
val job = assertNotNull(DownloadRepository.forRecommendation("alice", mood.id, "v-nomatch"))
|
||||
assertEquals("failed", job.state)
|
||||
val imports = library.folders.first().path.resolve("AudioManager Imports")
|
||||
if (Files.isDirectory(imports)) {
|
||||
val files = Files.walk(imports).use { stream ->
|
||||
stream.filter { Files.isRegularFile(it) }.toList()
|
||||
}
|
||||
assertTrue(files.isEmpty(), "leftover promoted files: $files")
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `permission errors fail without burning retries`() {
|
||||
withDatabase { _, library, users ->
|
||||
@@ -205,10 +356,14 @@ class DownloadServiceTest {
|
||||
}
|
||||
}
|
||||
|
||||
private fun insertSong(handle: DatabaseHandle, artistName: String, title: String) {
|
||||
private fun insertSong(
|
||||
handle: DatabaseHandle,
|
||||
artistName: String,
|
||||
title: String,
|
||||
relativePath: String = "AudioManager Imports/$artistName - $title.mp3",
|
||||
): String {
|
||||
val artistId = IdGenerator.artistId(artistName)
|
||||
val albumId = IdGenerator.albumId(artistName, "Downloads")
|
||||
val relativePath = "AudioManager Imports/$artistName - $title.mp3"
|
||||
val songId = IdGenerator.songId(1, relativePath)
|
||||
val now = Instant.now().toString()
|
||||
|
||||
@@ -258,5 +413,6 @@ class DownloadServiceTest {
|
||||
it[genre] = "Hörspiel"
|
||||
}
|
||||
}
|
||||
return songId
|
||||
}
|
||||
}
|
||||
@@ -24,6 +24,7 @@ import win.taptap.musicmanager.db.ArtistsTable
|
||||
import win.taptap.musicmanager.db.ConfigRepository
|
||||
import win.taptap.musicmanager.db.DatabaseFactory
|
||||
import win.taptap.musicmanager.db.DatabaseHandle
|
||||
import win.taptap.musicmanager.db.LibraryTombstonesTable
|
||||
import win.taptap.musicmanager.db.MediaRepository
|
||||
import win.taptap.musicmanager.db.SongGenresTable
|
||||
import win.taptap.musicmanager.db.SongsTable
|
||||
@@ -137,6 +138,73 @@ class MediaScannerTest {
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@Test
|
||||
fun `an unreadable folder does not delete its songs`() {
|
||||
TestAudio.assumeFfmpeg()
|
||||
val db = DatabaseFactory.connect(appConfig())
|
||||
try {
|
||||
val library = library()
|
||||
ConfigRepository.sync(library, users())
|
||||
|
||||
val file = music.resolve("Artist/Album/song.mp3")
|
||||
TestAudio.createMp3(file)
|
||||
|
||||
val scanner = MediaScanner(library, CoverArtService(tempDir.resolve("data")))
|
||||
assertEquals(1L, scanner.scanBlocking().count)
|
||||
assertEquals(1L, MediaRepository.countSongs())
|
||||
|
||||
// Simulate an unmounted or temporarily unreadable folder.
|
||||
music.toFile().deleteRecursively()
|
||||
|
||||
val broken = scanner.scanBlocking()
|
||||
assertNotNull(broken.error)
|
||||
assertEquals(1L, MediaRepository.countSongs())
|
||||
} finally {
|
||||
db.close()
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@Test
|
||||
fun `re-adding a deleted file clears its tombstone`() {
|
||||
TestAudio.assumeFfmpeg()
|
||||
val db = DatabaseFactory.connect(appConfig())
|
||||
try {
|
||||
val library = library()
|
||||
ConfigRepository.sync(library, users())
|
||||
val file = music.resolve("Artist/Album/song.mp3")
|
||||
TestAudio.createMp3(file)
|
||||
val scanner = MediaScanner(library, CoverArtService(tempDir.resolve("data")))
|
||||
scanner.scanBlocking()
|
||||
val songId = IdGenerator.songId(1, "Artist/Album/song.mp3")
|
||||
|
||||
Files.delete(file)
|
||||
scanner.scanBlocking()
|
||||
transaction(db.database) {
|
||||
assertEquals(
|
||||
1L,
|
||||
LibraryTombstonesTable.selectAll()
|
||||
.where { LibraryTombstonesTable.entityId eq songId }
|
||||
.count(),
|
||||
)
|
||||
}
|
||||
|
||||
TestAudio.createMp3(file)
|
||||
scanner.scanBlocking()
|
||||
transaction(db.database) {
|
||||
assertEquals(
|
||||
0L,
|
||||
LibraryTombstonesTable.selectAll()
|
||||
.where { LibraryTombstonesTable.entityId eq songId }
|
||||
.count(),
|
||||
)
|
||||
}
|
||||
} finally {
|
||||
db.close()
|
||||
}
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `unchanged files are skipped on a second scan`() {
|
||||
TestAudio.assumeFfmpeg()
|
||||
|
||||
Reference in new issue
Block a user