From bc1e3d78d3ce0f115db1ab9b49b20daa46567167 Mon Sep 17 00:00:00 2001 From: thethreemagi Date: Fri, 31 Jul 2026 19:35:52 +0100 Subject: [PATCH] =?UTF-8?q?migration:=20new=20sequence=20=E2=80=94=20D:=20?= =?UTF-8?q?is=20the=20backup,=20copy-off=20to=20media=5Fpc,=20format=20bef?= =?UTF-8?q?ore=20first=20start?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit No Windows-side robocopy. Everything on D: (PMS dir, Calibre, Nikola iPad) is copied to the NAS media_pc share from Linux, Media/ staged via the SSD, D: formatted ext4 BEFORE Plex first starts, thumbnails live on /srv/data. Fixes the M3 restore path that pointed at the unmounted Share. Per Mike 2026-07-31. --- migration/README.md | 286 +++++++++++++++++++++++++------------------- 1 file changed, 163 insertions(+), 123 deletions(-) diff --git a/migration/README.md b/migration/README.md index 7895941..7d639f7 100644 --- a/migration/README.md +++ b/migration/README.md @@ -6,6 +6,12 @@ already-generated preview cache**, rebuilding fresh costs one to three weeks of saturated NAS link and permanently loses every manual match fix. Migration is the cheaper and lower-risk option. +**Revised 2026-07-31:** no Windows-side robocopy, `D:` itself is the backup +until everything on it is copied to the NAS `media_pc` share from Linux, and +the 337 GB `Media/` cache lives on the 3 TB ext4 disk — **not the SSD** — from +day one. `D:` is therefore formatted **before** Plex first starts, not after +the soak. + ## What survives Watch history and resume points across all items, play counts, ratings, @@ -14,6 +20,9 @@ choices, added-at dates (so Recently Added stays meaningful), the 337 GB preview cache, and **the server identity** — shared users stay invited, clients do not need to re-add the server, and there is no claim step at all. +Calibre (25.4 GB) and `Nikola iPad` (1.2 GB) also survive — they move to the +NAS `media_pc` share. Nothing on `D:` is abandoned. + Plex's own article calls cross-OS moves "possible but not officially supported and not covered here." That is a documentation gap, not a technical barrier. The database is platform-independent; what Plex declines to help with is @@ -25,176 +34,207 @@ mapping the settings, which `gen-preferences.ps1` does. |---|---| | Source | PMS **1.43.3.10828**, Windows 10, data dir `D:\Plex\Local Data` | | Server identity | `machineIdentifier eb69ec8a…` — preserved, not regenerated | -| Data directory | ~374 GB total → ~364 GB after pruning | +| Data directory | ~374 GB total → ~364 GB after pruning (~27 GB minus `Media/`) | | Library | 1,360 movies · 23,493 episodes · 295 other video · 20,798 audio | | Path rows | 80,126 `media_parts` · 14,837 `media_streams` · 11 → 9 roots | -| Target layout | Whole data dir on the 500 GB SSD (~75 GB free) | -| NAS mounts | 6 shares at `/mnt/nas/` | +| Target layout | DB + `Metadata/` on the SSD · `Media/` (337 GB) on the 3 TB ext4 disk | +| Backup | Full copy on NAS `media_pc` (write share), made from Linux before `D:` is formatted | +| NAS mounts | 6 library shares read-only + `media_pc` read-write, at `/mnt/nas/` | --- ## M0 — Prepare, while Windows is still running -**Disable automatic trash emptying first.** This is step 1 of Plex's own -procedure and it matters more here than usual: the registry confirms -`autoEmptyTrash = 1`, and the new NAS account is read-write. A missing mount at -scan time would mean Plex deletes library records it has permission to delete. -Settings → Library → untick *Empty trash automatically after every scan*. +**Disable automatic trash emptying first.** The registry confirms +`autoEmptyTrash = 1`. The new NAS account is read-only on the library shares, +so Plex can no longer delete *files* — but a missing mount at scan time would +still purge library *records*. Settings → Library → untick *Empty trash +automatically after every scan*. -Generate the Linux settings file from the registry: +Generate the Linux settings file from the registry. **Write it to `D:`, not +`C:` — `C:` is the install target and gets wiped:** ```powershell -powershell -ExecutionPolicy Bypass -File .\gen-preferences.ps1 -Out C:\mbx\Preferences.xml -[xml](Get-Content -Raw C:\mbx\Preferences.xml) | Out-Null; 'XML OK' +powershell -ExecutionPolicy Bypass -File .\gen-preferences.ps1 -Out D:\mbx\Preferences.xml +[xml](Get-Content -Raw D:\mbx\Preferences.xml) | Out-Null; 'XML OK' ``` **Stop Plex Media Server completely** — tray icon → Exit, then confirm no -`Plex Media Server.exe` remains. The database must not be open when copied, or -the WAL will be mid-flight and the copy inconsistent. +`Plex Media Server.exe` remains. This is the database's final clean shutdown; +a copy taken mid-WAL is inconsistent and you will not find out until much +later. Do not start Plex on Windows again after this. -Copy the data directory to the NAS. Skip what must not travel: +There is **no robocopy here**. `D:` is untouched by the install and is the +backup until M2 copies it off. -```powershell -robocopy "D:\Plex\Local Data\Plex Media Server" "\\korval\Share\backups\mediabox\PMS" ` - /E /COPY:DAT /R:2 /W:5 /MT:16 ` - /XD "Cache" "Codecs" "Drivers" "Crash Reports" "Logs" "Updates" ` - /LOG:C:\mbx\pms-copy.log /TEE -``` - -`Codecs` and `Drivers` are **Windows binaries** — copying them to Linux breaks -transcoding. `Cache` is regenerable. Excluding them also trims the copy. - -> **GATE M0** — robocopy reports 0 failures. `Plug-in Support\Databases\`, -> `Metadata\` and `Media\` are all present on the NAS. **Do not proceed on a -> copy you have not listed.** +> **GATE M0** — `D:\mbx\Preferences.xml` exists, parses, and contains +> `MachineIdentifier`. Plex is fully exited. --- -## M1 — Dry run, before anything is destroyed +## M1 — Install Linux -The whole migration can be rehearsed while Windows still works. This is the -step that makes the rest safe. - -On arrsstack (or any Linux host with the NAS mounted), take **only** the -database and Preferences — a couple of GB, not 364 — and run the remap: - -```bash -sudo ./migrate-db.sh /path/to/copy/com.plexapp.plugins.library.db -``` - -Then start a throwaway Plex against it **with no internet access**, so it -cannot contact plex.tv and collide with the live server that shares its -identity: - -```bash -docker network create --internal plex-dryrun -docker run --rm -d --name plex-dryrun --network plex-dryrun \ - -v /path/to/dryrun-config:/config \ - -v "/mnt/nas/media:/mnt/nas/media:ro" \ - lscr.io/linuxserver/plex:latest -``` - -> **GATE M1** — browse the unclaimed server directly and confirm: Movies shows -> **1,360**, TV Shows shows **23,493 episodes**, watch state and resume points -> are present, and a title's file path resolves. If this passes, the migration -> is proven. If it fails you have lost an afternoon and nothing else. - ---- - -## M2 — Install Linux - -Follow the main `runbook.md` phases 2–5 (BIOS, USB, install, first boot). The +Follow the main `runbook.md` phases 2–8 (BIOS — the iGPU toggle is mandatory, +Quick Sync is the transcode device — then USB, install, first boot, NAS +credentials and the seven mounts, shell-mcp cutover, `vainfo`). The autoinstall targets Disk 0 by serial `1808AE802176` and leaves the 3 TB disk untouched. -Write the NAS credentials and bring up the six mounts: - -```bash -printf 'username=mediabox\npassword=\ndomain=WORKGROUP\n' \ - | sudo tee /etc/cifs/korval.cred >/dev/null -sudo chmod 600 /etc/cifs/korval.cred -sudo systemctl daemon-reload -sudo /srv/mediabox-bootstrap/scripts/20-cifs.sh --verify -``` - -> **GATE M2** — all six mounts report `[ok]`, and `stat -c '%u:%g' /mnt/nas/media` -> returns `3000:3000`. Wrong ownership means Plex hits permission errors on -> every file. +> **GATE M1** — runbook gates 4–8 all pass. In particular: all seven mounts +> `[ok]`, the six library shares refuse writes, `media_pc` accepts them, and +> `vainfo` lists H.264/HEVC entrypoints. --- -## M3 — Restore and remap +## M2 — Copy everything off `D:` + +Mount the NTFS disk read-only and copy all of it to the NAS write share: ```bash -sudo install -d -o 3000 -g 3000 /srv/plex/config -sudo rsync -a --info=progress2 \ - "/mnt/nas/Share/backups/mediabox/PMS/" \ - "/srv/plex/config/Library/Application Support/Plex Media Server/" +DEV=/dev/$(lsblk -dno NAME,SERIAL | awk '$2=="WD-WCC7K3JAEDR3"{print $1}') +sudo mkdir -p /mnt/dwin +sudo mount -t ntfs3 -o ro "${DEV}1" /mnt/dwin # partition number per lsblk -sudo install -o 3000 -g 3000 -m 0644 /path/to/Preferences.xml \ +# 1) Full PMS backup — excludes match the old robocopy list +sudo rsync -a --info=progress2 \ + --exclude='Cache/' --exclude='Codecs/' --exclude='Drivers/' \ + --exclude='Crash Reports/' --exclude='Logs/' --exclude='Updates/' \ + "/mnt/dwin/Plex/Local Data/Plex Media Server/" \ + "/mnt/nas/media_pc/backups/mediabox/PMS/" + +# 2) The generated settings file +sudo rsync -a "/mnt/dwin/mbx/" "/mnt/nas/media_pc/backups/mediabox/mbx/" + +# 3) Everything else worth keeping — nothing on D: is abandoned +sudo rsync -a --info=progress2 "/mnt/dwin/Calibre/" "/mnt/nas/media_pc/Calibre/" +sudo rsync -a --info=progress2 "/mnt/dwin/Nikola iPad/" "/mnt/nas/media_pc/Nikola iPad/" +``` + +`Codecs` and `Drivers` are **Windows binaries** — carrying them to Linux +breaks transcoding. `Cache` is regenerable. + +Expect this to take a long time — `Metadata/` and `Media/` are millions of +small files, and small-file writes over CIFS are slow. Plan on hours, possibly +overnight. It only has to happen once. + +> **GATE M2** — rsync exits 0 on all four copies. Spot-check on the NAS that +> `backups/mediabox/PMS/Plug-in Support/Databases/`, `Metadata/` and `Media/` +> are present and that `Calibre/` and `Nikola iPad/` file counts look right. +> **Do not proceed to M3 on a copy you have not listed.** + +*Optional rehearsal:* before M3, run `migrate-db.sh` on a scratch copy of the +database from `/mnt/dwin` and start a throwaway Plex against it on an +`--internal` Docker network (no internet, so it cannot collide with the +server identity). Browse it and confirm counts and watch state. This proves +the migration while `D:` is still intact. + +--- + +## M3 — Stage, format `D:`, place the data + +`Media/` cannot stay on the SSD (337 GB, Mike's call 2026-07-31) and cannot +live on the NAS (network cost + RAID space on disposable cache). Its home is +the 3 TB disk — which currently holds the source data. So: stage through the +SSD, format, move. + +```bash +# 1) Restore the config (everything EXCEPT Media/) to its real home on the SSD +sudo install -d -o 3000 -g 3000 /srv/plex/config +sudo rsync -a --info=progress2 --exclude='Media/' \ + "/mnt/dwin/Plex/Local Data/Plex Media Server/" \ + "/srv/plex/config/Library/Application Support/Plex Media Server/" \ + --exclude='Cache/' --exclude='Codecs/' --exclude='Drivers/' \ + --exclude='Crash Reports/' --exclude='Logs/' --exclude='Updates/' + +sudo install -o 3000 -g 3000 -m 0644 /mnt/dwin/mbx/Preferences.xml \ "/srv/plex/config/Library/Application Support/Plex Media Server/Preferences.xml" +# 2) Stage Media/ on the SSD — temporary, ~337 GB into ~400 GB free. Check first: +df -h / +sudo rsync -a --info=progress2 \ + "/mnt/dwin/Plex/Local Data/Plex Media Server/Media/" \ + /srv/plex-media-staging/ + +# 3) Unmount and format D: — BY SERIAL, never /dev/sdX +sudo umount /mnt/dwin +lsblk -o NAME,SERIAL,SIZE,FSTYPE +DEV=/dev/$(lsblk -dno NAME,SERIAL | awk '$2=="WD-WCC7K3JAEDR3"{print $1}') +echo "$DEV" # sanity-check this before continuing +sudo wipefs -a "$DEV" +sudo parted -s "$DEV" mklabel gpt mkpart primary ext4 0% 100% +sudo mkfs.ext4 -L mediabox-data "${DEV}1" +UUID=$(blkid -s UUID -o value "${DEV}1") +echo "UUID=$UUID /srv/data ext4 defaults,nofail 0 2" | sudo tee -a /etc/fstab +sudo mkdir -p /srv/data && sudo systemctl daemon-reload && sudo mount /srv/data + +# 4) Move Media/ to its permanent home (fast local move, no NAS round trip) +sudo install -d -o 3000 -g 3000 /srv/data/plex-media +sudo rsync -a --info=progress2 --remove-source-files /srv/plex-media-staging/ /srv/data/plex-media/ +sudo find /srv/plex-media-staging -type d -empty -delete +sudo chown -R 3000:3000 /srv/plex/config /srv/data/plex-media +``` + +The staging copy through the SSD is deliberate: local disk-to-disk at full +SATA speed instead of a 337 GB round trip through gigabit CIFS. The NAS copy +from M2 is the *backup*, not the restore source — unless something here fails, +in which case it is exactly that. + +> **GATE M3** — `df -h /srv/data` shows ~2.7 TB, survives a reboot. +> `/srv/data/plex-media` and the config tree are owned `3000:3000`. The +> staging directory is gone and the SSD is back to mostly free. + +--- + +## M4 — Remap and first start + +```bash sudo /srv/mediabox-bootstrap/migration/migrate-db.sh \ "/srv/plex/config/Library/Application Support/Plex Media Server/Plug-in Support/Databases/com.plexapp.plugins.library.db" -sudo chown -R 3000:3000 /srv/plex/config -``` - -`migrate-db.sh` keeps a `.pre-remap` rollback copy, asserts that no Windows -paths remain, that the counts moved 1:1, that the "Dr. Korval" tag and the four -Liaden book summaries are **untouched**, and finally stats 200 random remapped -paths against the real filesystem. - -> **GATE M3** — `migrate-db.sh` exits 0 with the sample check reporting -> `200/200 sampled files resolve on disk`. That last line is the one that -> matters; SQL running without error proves nothing. - ---- - -## M4 — First start - -```bash sudo RENDER_GID=$(stat -c '%g' /dev/dri/renderD128) \ /srv/mediabox-bootstrap/scripts/50-plex.sh ``` -No claim token. The server already has its identity. +`migrate-db.sh` keeps a `.pre-remap` rollback copy, asserts that no Windows +paths remain, that the counts moved 1:1, that the "Dr. Korval" tag and the +four Liaden book summaries are **untouched**, and finally stats 200 random +remapped paths against the real filesystem. -> **GATE M4** — `plex.thewichersfamily.com` loads and shows *Magi Plex* with -> your libraries. Watch history and Continue Watching are intact. A remote -> client that already had this server still sees it without re-adding. +No claim token. The server already has its identity. The compose bind-mounts +`/srv/data/plex-media` over `.../Plex Media Server/Media`, so the migrated +thumbnails are in place from the first start. + +`scripts/60-media-relocate.sh` is **no longer needed** — `Media/` is born on +`/srv/data`. What still matters from it is the verification: after first +start, force thumbnail generation on one title and confirm the write lands +under `/srv/data/plex-media` — that is the `EXDEV` cross-filesystem question, +which can only be tested, not reasoned about. + +> **GATE M4** — `migrate-db.sh` exits 0 with `200/200 sampled files resolve on +> disk`. `plex.thewichersfamily.com` loads and shows *Magi Plex* with Movies +> 1,360 and 23,493 episodes; watch history and Continue Watching intact; a +> remote client that already had this server still sees it without re-adding. > Play something and confirm `intel_gpu_top` shows the video engine working. +> The one-title thumbnail test writes under `/srv/data/plex-media`. --- ## M5 — Soak -One to two weeks on the supported layout. Confirm mounts survive a NAS reboot, -the GPU survives a kernel update, and the box comes back **unattended** from a -real power cut. That last one is the acceptance test for "headless". +One to two weeks. Confirm mounts survive a NAS reboot, the GPU survives a +kernel update, and the box comes back **unattended** from a real power cut. +That last one is the acceptance test for "headless". ---- - -## M6 — Relocate the preview cache (optional, post-soak) - -Only after M5. Moves the 337 GB `Media/` directory to the 3 TB ext4 disk so -future thumbnail growth (~28 MB per content-hour) stops consuming the SSD. - -```bash -sudo /srv/mediabox-bootstrap/scripts/60-media-relocate.sh -``` - -Plex does **not** support relocating this directory. The script performs the -move, enables the bind mount, and then forces thumbnail generation on a single -title to prove writes succeed across the filesystem boundary — the one failure -mode that cannot be reasoned about, only tested. Rollback is moving it back. +**Leave the NAS `media_pc` backup in place** — it is the durable copy of the +old server. Prune it later if the space is ever needed; that is a deliberate +decision, not a cleanup step. --- ## If something goes wrong -Before M2, everything is reversible — Windows still boots. After M2 the -recovery path is *reinstall*, not *roll back*: the USB rebuilds the box in -about twenty minutes and the data directory is still on the NAS. The database -copy on the NAS outlives every step here, and `migrate-db.sh` leaves a -`.pre-remap` copy beside every database it touches. +Before M3 step 3 (the format), `D:` is intact and everything is reversible on +the data side — Windows itself is gone after M1, but a reinstall from the USB +takes ~20 minutes and every copy still exists. After the format, the NAS +`media_pc` backup is the recovery source: it outlives every step here, and +`migrate-db.sh` leaves a `.pre-remap` copy beside every database it touches.