I've been trying to upgrade from 10.10 to 10.11 for a while now, as the Android TV app keeps nagging me, and every attempt ended with impossibly long library scan times.
After some thorough investigation, it appeared that a Home Videos type collection causes unending scan (yet to be solved), but also that Jellyfin does a lot of writes to the config directory (either database or metadata or both). Mine's on spinning media part of a ZFS pool. I tried a few performance tuning options, such as testing the config dir with recordsize (similar to block size) of 4K, 8K, 64K, 128K and library scans fell from 30-40 minutes down to 8-13min with 4K-64K. The ZFS tuning wiki suggest 64K recordsize with LZ4 compression for SQLite workloads such as Jellyfin. That seems to work as well as 4K and 8K but likely is faster when reading thumbnails and such.
Note that upgrading to 12-rc3, which is supposed to speed up library scans did not improve scan times for me. Optimizing config/database write speed did. I cross-checked the culprit by experimenting with moving the config dir to NVMe and RAM. Both of those got the scan times down to 8-9 minutes compared to the optimized spinning media's 12-13.
So if you had upgraded (or about to) to 10.11 your library scans are (about to get) dog slow and your Jellyfin's config dir resides on spinning media, optimize its write performance for SQLite.
How large is the database? Because there is a way to fully load SQLite into memory, which would speed everything up drastically for a scan. At the end of the scan, you could just write back everything to the disk, then it is just one blob of sequential write, which is fast enough on spinning rust.
Interesting, I figured people would use MySQL instead if possible.
600MB. Would that require changes to Jellyfin? They're on the path to introducing Postgres support.
I just checked the code and unfortunately, they do not support loading the SQLite file into memory yet. Even worse, the change would be substantially. It appears they currently abuse the database connection a bit. They basically create a new connection for each operation. This does not matter if you can read/write in parallel, but that’s only the case with NVMe and certain drivers. So all SATA (SSD and HDD) suffer from the current mechanic. So just loading it into memory is not going to solve the issue. Jellyfin needs to change architecturally, so that a database connection is reused. I don’t know why or who implemented the current logic, but it is clearly done by someone not understanding how databases work.
Depending on the mood and time, I will come up with a solution that is solid. For now, you are stuck with the current logic.
Thanks for looking into it. As far as I read they moved to an ORM with 10.11 as a prereq to supporting other db engines. I'm guessing the db connection code would be undergoing major changes when they do that so I wouldn't rewrite the current implementation. I'd either follow that work and help them get it right or rework the implementation after they're done. We'll just suffer through the mean time. 😄