{"id":18,"date":"2026-03-26T16:02:31","date_gmt":"2026-03-26T16:02:31","guid":{"rendered":"https:\/\/kr0311.com\/projects\/?p=18"},"modified":"2026-03-26T21:32:13","modified_gmt":"2026-03-26T21:32:13","slug":"the-day-12000-photos-fought-back","status":"publish","type":"post","link":"https:\/\/kr0311.com\/projects\/the-day-12000-photos-fought-back\/","title":{"rendered":"The Day 12,000 Photos Fought Back"},"content":{"rendered":"\n

I didn\u2019t plan for this to turn into a full system stress test.

What started as a simple job \u2013 organising an iCloud photo dump \u2013 quickly escalated into one of the most revealing (and slightly chaotic) moments I\u2019ve had with my homelab.

This post sits later in my homelab journey, once everything was already built and running smoothly. And honestly\u2026 it ended up being the perfect accidental test of the entire stack.<\/p>\n\n\n\n

\ud83e\uddf1 The Setup<\/h2>\n\n\n\n

At this point in my setup:
– Proxmox host (R730) running multiple VMs
– TrueNAS handling storage (ZFS)
– A Lubuntu VM used as a utility machine<\/p>\n\n\n\n

All running locally on the same host \u2014 no external bottlenecks, no excuses.<\/p>\n\n\n\n

<\/p>\n\n\n\n

\ud83c\udfaf The Task<\/h2>\n\n\n\n

The task was simple (or so I thought):<\/p>\n\n\n\n

Take ~12,000 photos and videos dumped into a single folder and organise them into a clean structure:<\/p>\n\n\n\n

Sorted by: YYYY\/MM<\/pre>\n\n\n\n
Nice and tidy. Easy job\u2026 right?<\/p>\n\n\n\n
<\/p>\n\n\n\n
\u26a0\ufe0f The First Mistake<\/h2>\n\n\n\n
I opened Rapid Photo Downloader and immediately selected the source folder\u2026<\/p>\n\n\n\n
Bad idea.

It instantly started scanning all 12,000 files, generating thumbnails, and confidently creating folder structures I never asked for.

Before I\u2019d even touched the settings.

At this point, I wasn\u2019t organising files anymore \u2014 I was watching chaos unfold in real time.

This resulted in a mess of incorrectly named folders that I then had to clean up manually.<\/p>\n\n\n\n
<\/p>\n\n\n\n
\ud83d\udd27 Reset & Proper Approach<\/h2>\n\n\n\n
After admitting defeat (and undoing the damage via CLI), I tried again \u2014 properly this time.<\/p>\n\n\n\n
    \n
  1. Set destination first<\/li>\n\n\n\n
  2. Configure folder structure (Year\/Month)<\/li>\n\n\n\n
  3. Disable thumbnails generation<\/li>\n\n\n\n
  4. THEN select source<\/li>\n<\/ol>\n\n\n\n
    Amazing what happens when you do things in the right order.<\/p>\n\n\n\n
    This made a huge difference in both performance and stability.<\/p>\n\n\n\n
    After fixing the setup, the structure looked like this:<\/p>\n\n\n\n
    \"\"<\/figure>\n\n\n\n
    \ud83d\ude80 When It Started Working… Properly<\/h2>\n\n\n\n
    As soon as the job started correctly, things got interesting.<\/p>\n\n\n\n
    Both the processing VM and storage VM started ramping up almost immediately.

    The Lubuntu VM handling the sorting began chewing through memory and CPU, while TrueNAS kicked into gear, aggressively caching and handling I\/O like it had something to prove.

    Lubuntu VM under load on the first attempt (before I gave it 12GB RAM):<\/p>\n\n\n\n
    \"\"<\/figure>\n\n\n\n
    \u2699\ufe0f What Was Actually Happening<\/h2>\n\n\n\n
    This workload turned out to be far more demanding than expected:

    – Thousands of small file operations
    – Metadata (EXIF) parsing per file
    – Constant read\/write operations
    – Directory creation and file moves

    Individually, each operation is tiny.

    At this scale? Not so much.

    This is pretty much a worst-case scenario for storage systems \u2014 lots of small, repetitive operations that don\u2019t scale nicely.<\/p>\n\n\n\n
    <\/p>\n\n\n\n
    \ud83d\udcbe TrueNAS Behaviour<\/h2>\n\n\n\n
    At the same time, TrueNAS started ramping up.

    RAM usage shot up close to 100%, which might look alarming at first \u2014 but this is exactly what you want.
    ZFS uses available memory for ARC cache, improving performance during heavy workloads.
    Disk I\/O spiked as files were being read, moved, and rewritten across the dataset.

    TrueNAS under heavy load:<\/p>\n\n\n\n
    \"\"<\/figure>\n\n\n\n
    <\/p>\n\n\n\n
    \ud83e\udde0 Meanwhile\u2026 The Host<\/h2>\n\n\n\n
    And then there\u2019s the part that made me laugh.

    Despite all of this activity\u2026
    The Proxmox host itself barely broke a sweat.
    CPU usage hovered around ~9%, and overall system resources were largely untouched.
    It was basically sat there like:

    \u201cIs that it?\u201d

    This highlighted an important point:
    The workload wasn\u2019t limited by compute power \u2014 it was limited by I\/O and per-file operations.

    Meanwhile, the host:<\/p>\n\n\n\n
    \"\"<\/figure>\n\n\n\n
    <\/p>\n\n\n\n
    \ud83e\udde9 Conclusion<\/h2>\n\n\n\n
    What started as a simple photo organisation task turned into a full-stack test of my homelab.

    It validated:
    – VM resource allocation
    – Storage performance under load
    – ZFS caching behaviour
    – Overall system headroom

    And more importantly\u2026

    It proved the setup could handle real-world workloads without falling over \u2014 even if the VMs themselves looked like they were fighting for their lives at times.<\/p>\n\n\n\n
    Also, a valuable lesson was learned:<\/p>\n\n\n\n
    \ud83d\udc49 Configure first
    \ud83d\udc49 Select 12,000 files second<\/p>\n\n\n\n
    <\/p>\n","protected":false},"excerpt":{"rendered":"
    What started as a simple photo organisation task quickly turned into a full system stress test across my homelab stack.<\/p>\n","protected":false},"author":1,"featured_media":35,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[7],"tags":[8,12,10,13,9,15,16,11,18,14],"class_list":["post-18","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-experiments-incidents","tag-data-management","tag-file-organisation","tag-homelab","tag-linux","tag-photo-management","tag-proxmox","tag-real-world-test","tag-truenas","tag-unexpected-issues","tag-zfs"],"_links":{"self":[{"href":"https:\/\/kr0311.com\/projects\/wp-json\/wp\/v2\/posts\/18","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/kr0311.com\/projects\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/kr0311.com\/projects\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/kr0311.com\/projects\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/kr0311.com\/projects\/wp-json\/wp\/v2\/comments?post=18"}],"version-history":[{"count":5,"href":"https:\/\/kr0311.com\/projects\/wp-json\/wp\/v2\/posts\/18\/revisions"}],"predecessor-version":[{"id":31,"href":"https:\/\/kr0311.com\/projects\/wp-json\/wp\/v2\/posts\/18\/revisions\/31"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/kr0311.com\/projects\/wp-json\/wp\/v2\/media\/35"}],"wp:attachment":[{"href":"https:\/\/kr0311.com\/projects\/wp-json\/wp\/v2\/media?parent=18"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/kr0311.com\/projects\/wp-json\/wp\/v2\/categories?post=18"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/kr0311.com\/projects\/wp-json\/wp\/v2\/tags?post=18"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}