Skip to content

feat(custom-rows): implement server-side cache and proxy endpoints for custom rows#174

Merged
RadicalMuffinMan merged 10 commits into
Moonfin-Client:masterfrom
mattsigal:feature/custom-rows-server
Jun 29, 2026
Merged

feat(custom-rows): implement server-side cache and proxy endpoints for custom rows#174
RadicalMuffinMan merged 10 commits into
Moonfin-Client:masterfrom
mattsigal:feature/custom-rows-server

Conversation

@mattsigal

Copy link
Copy Markdown
Contributor

Summary

Migrate custom home rows fetching, parsing, and caching logic from the client to the server-side Jellyfin plugin. This supports TMDB lists/collections, Letterboxd watchlists/lists/films, and MDBList custom rows, shifting processing and rate-limiting throttling entirely to the server and keeping client implementations clean.

Type of Change

  • Bug fix
  • New feature
  • Refactor
  • Performance improvement

Changes Made

  • Created CustomRowCacheService handling a thread-safe local JSON cache with a 24-hour TTL mapping custom row configuration keys to resolved items.
  • Created CustomRowController exposing the /Moonfin/CustomRows/Items API.
  • Implemented namespace-agnostic Letterboxd RSS feed parsing, a local slug resolver mapping Letterboxd URLs to TMDb IDs, and automated parallel TMDB poster/vote metadata queries.
  • Converted Letterboxd double ratings (0.0 - 5.0) to star formatting (e.g. ★★★★½) displayed inside client row cards.
  • Synced and resolved server-side TMDB collection parts, list items, and MDBList custom lists.

@github-actions

github-actions Bot commented Jun 28, 2026

Copy link
Copy Markdown

Build Successful

The plugin compiled successfully against .NET 8 / Jellyfin 10.10.0.

Property Value
Commit f9bab12
Workflow Build #147

@RadicalMuffinMan RadicalMuffinMan left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Heads up on the Letterboxd part of the PR. Right now the custom rows fetch pulls Letterboxd by crawling their HTML pages. For watchlist, films, and lists it hits letterboxd.com/{user}/watchlist/ etc and regexes the film links out of the markup, then for each film it scrapes letterboxd.com/film/{slug}/ to pull the tmdb id, and that film scraping now runs 10 requests at a time. That is the exact thing we deliberately moved away from when we killed the IMDb GraphQL scraping. It's against Letterboxd's terms.

The good news is we already solved this once and the same fix applies here. Letterboxd should go through MDBList instead of being scraped directly. MDBList ingests external lists, Letterboxd included, and we already have an MDBList path and key in this same controller. So FetchLetterboxd should hand the Letterboxd list URL to MDBList and map the items it returns, and the whole isHtmlCrawled branch, the regex parsing, and the parallel letterboxd.com/film/... slug loop can all come out.

If there is a specific list MDBList can't resolve, the acceptable fallback is the RSS feed, which we still have wired in the default case. RSS is a published feed meant to be consumed, so that one is fine. What we should not do is fetch and parse their HTML pages. Below are the rest of the fixes needed

Comment thread backend/Services/CustomRowCacheService.cs
Comment thread backend/Api/CustomRowController.cs
Comment thread backend/Api/CustomRowController.cs Outdated
Comment thread backend/Api/CustomRowController.cs Outdated
Comment thread backend/Api/CustomRowController.cs Outdated
Comment thread backend/Api/CustomRowController.cs Outdated
Comment thread backend/Api/CustomRowController.cs Outdated
mattsigal and others added 8 commits June 28, 2026 17:13
Co-authored-by: Axl <103554043+RadicalMuffinMan@users.noreply.github.com>
Co-authored-by: Axl <103554043+RadicalMuffinMan@users.noreply.github.com>
Co-authored-by: Axl <103554043+RadicalMuffinMan@users.noreply.github.com>
Co-authored-by: Axl <103554043+RadicalMuffinMan@users.noreply.github.com>
Co-authored-by: Axl <103554043+RadicalMuffinMan@users.noreply.github.com>
Co-authored-by: Axl <103554043+RadicalMuffinMan@users.noreply.github.com>
Co-authored-by: Axl <103554043+RadicalMuffinMan@users.noreply.github.com>
…Ds from RSS namespace

- Throw ArgumentException for watchlist, films, and user_list requests due to Letterboxd Terms of Service.
- Extract tmdb:movieId directly from the RSS feed, avoiding any film page HTML scraping.
- Remove letterboxd_slug_to_tmdb.json persistent cache and helper methods.
@mattsigal

Copy link
Copy Markdown
Contributor Author

OK, changes made. One note - Letterboxd only publishes a native RSS feed for a user's profile activity/diary (at https://letterboxd.com/username/rss/). There are no native RSS feeds provided by Letterboxd for watchlists (/username/watchlist/), completed films (/username/films/), or custom lists (/username/list/listname/) - which was why the HTML scrapper was needed. Since that is a ToS violation, supporting watchlists or custom lists directly via the FetchLetterboxd source is no longer possible.

As a workaround, user can import their Letterboxd watchlists and custom lists into MDBList and configure them using the client's mdblist custom home row source instead - which would fully complies with Letterboxd's terms while preserving the watchlist and custom list features in the app :)

Changes:

  1. Removed Letterboxd HTML Scraping:
  • Disabled all web scraping/crawling routes for watchlist, films, and user_list categories.
  • Removed the parallel page-by-page scraper loop that queried letterboxd.com/film/{slug}/ for TMDb IDs.
  • Cleaned up and deleted the local persistent cache file (letterboxd_slug_to_tmdb.json), its associated semaphore lock, and the helper methods (EnsureSlugsLoaded(), FlushSlugsAsync()).
  1. Added Direct RSS TMDb ID Parsing:
  • Updated the permitted public RSS activity/diary feed parser (https://letterboxd.com/username/rss/) to extract the TMDb ID directly from the XML namespace-agnostically using the movieId tag.
  • The plugin now retrieves movie posters and rating details from TMDb without performing any scraping requests to Letterboxd.
  1. Restored Build Integrity:
  • Fixed the build errors caused by accepted suggestions in the GitHub Web UI.
  • Verified that the server plugin compiles successfully with 0 warnings and 0 errors.

@RadicalMuffinMan RadicalMuffinMan merged commit 1f2749a into Moonfin-Client:master Jun 29, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants