#181

Every site in the LocalAd network serves images and video. PetBlip has member pet galleries and stream photo posts. ForumLA has thread image attachments. VolusiaMarket has business listing photos. The stream on every property shows media inline. At some point that media has to live somewhere — on disk, in a database, with a URL that works. For a long time, each site solved that problem independently. This is the story of how that changed, and where it's going.

The Problem: Every Site Had Its Own Media Situation

Before vc1 existed as a concept, media storage was handled at the application level. WordPress sites used the WordPress media library. Custom applications uploaded directly to their own server's filesystem. Each site had its own upload folder, its own database table tracking what was there, its own admin tool for managing it.

This worked until the sites started needing to share. A photo uploaded to PetBlip can't appear in a VolusiaMarket business listing without re-uploading it. A product image used in a stream post can't serve the same file to an ad system without duplicating it. Every site independently storing the same media meant the same file living in multiple places with no relationship between them — no provenance, no reuse, no central accounting of what existed or where.

The more immediate problem was video. The LocalAd stream on ForumLA was built to support YouTube embeds, but native video upload was always on the roadmap. Storing raw video on the same server handling web requests is a bad architecture — video files are large, transcoding is CPU-heavy, and you don't want a 300MB iPhone upload competing with page requests on the same box. That problem required a dedicated machine.

vc1: The Physical Machine

vc1 is a dedicated server on the LAN segment at 10.0.1.160, sitting alongside the other internal machines (ai1, the NAS, the mail server) behind the Frankenrouter. It runs Ubuntu with nginx, PHP 8.3-FPM, MariaDB 10.11, and ImageMagick. Media files live on a mounted storage volume — /mnt/photos/ for images and /mnt/video/ for video content.

It is not publicly accessible directly. All public-facing access to vc1 media goes through video.localad.cc, a domain that routes through the Frankenrouter's HAProxy layer to vc1's nginx. This keeps the media server off the public internet while making its content available to all the sites that need it.

Phase 1: The Photo Gallery

vc1's first production use was the PetBlip photo gallery. The original problem was simple: 600+ dog photos sitting in a zip file needed to be organized, stored, and displayed on petblip.com. That required building the initial upload infrastructure from scratch.

The first schema lived in a database called localad_photos with tables for members, dogs, galleries, and photos. The upload endpoint lived at /var/www/photo-gallery/api/upload.php. The flow was: PetBlip's ser2 server receives a file from a member's browser, resizes it to three sizes using PHP's GD library, then POSTs all three sizes plus a shared secret to vc1 via cURL. vc1 stores the files on disk under /mnt/photos/ and writes the paths to the database.

This worked and it shipped. The public pet gallery at PetBlip went live with over 400 photos in a masonry grid with lightbox. Member profiles got their own gallery pages. The stream integration allowed photos posted in the activity feed to route through vc1 for storage and serve back through video.localad.cc.

A separate localad_video database handled the video side — its own tables (videos, encode_queue, api_clients, api_keys) managing a basic video pipeline. Video files landed in /mnt/video/streams/.

It was functional. It was also fragmented. Two separate databases, two separate schemas, no unified concept of a "media asset" that could span both.

The Problem with the Photo Gallery Approach

As more sites came online and vc1 started serving ForumLA thread images in addition to PetBlip gallery photos, the limitations of the original approach became structural problems.

Member ID drift. vc1 auto-incremented its own members.id independently of ser2's users.id. A PetBlip user with ID 7 on ser2 might map to member ID 8 on vc1 if the tables got out of sync. This produced a real bug where uploads from one user ended up attributed to a completely different member's gallery. Fixed with a source_site column added to the members table, but it was a symptom of a deeper design problem: the media server was trying to maintain its own user identity layer instead of treating the originating site as the authority on user identity.

No cross-site concept. If the same image needed to appear on VolusiaMarket and PetBlip, there was no mechanism for that. Each site uploaded independently. The same file could live on vc1 twice with no relationship between the two copies.

Path coupling. Applications stored file paths directly in their databases. If a file moved, if the directory structure changed, if a derivative size was renamed — every application that stored that path broke. There was no abstraction layer between "the file" and "where the file is."

No provenance. The localad_photos schema had no record of which site uploaded an asset, from which feature, by which user on that site. An admin looking at the database couldn't answer "where did this image come from?" without cross-referencing application logs.

The Architecture Decision: A Unified Media Identity Layer

The decision was to replace the fragmented approach with a single abstraction: every media asset gets a unique ID, and applications reference that ID — never a file path. The ID is the stable identity. Paths, derivatives, storage locations — those are implementation details that can change without breaking any application that uses the asset.

The new schema lives in a separate database: localad_media. The old localad_photos and localad_video databases remain untouched — the legacy content in them lacks the provenance metadata the new system requires, and migrating them automatically would produce low-quality records. They'll be retired gradually as their content either gets re-registered through the new system or becomes irrelevant.

The new schema has three core tables:

vc1_media — The Asset Registry

Every media asset that enters vc1 gets a row in vc1_media. The primary key is a generated media_id in the format vc1_YYYYMMDD_NNNNNN — for example, vc1_20260529_000001. This ID is what every application stores. Not a path. Not an integer. A stable string identifier that can be resolved through the API.

Key columns:

  • media_id — the stable external identifier
  • media_type — image or video
  • asset_class — what kind of asset: media, content, product, advertisement, or profile
  • source_site — which site uploaded it (petblip, forumla, marketsfl, etc.)
  • source_module — which feature on that site (stream, gallery, thread, listing)
  • source_record_id — the ID of the originating record on the source site, enabling reverse lookup
  • uploader_user_id — the user ID from the source site (the source site is the authority on user identity)
  • original_filename, mime_type, file_size, width, height
  • status — pending, ready, failed, archived

Currently 32 assets are registered in this table: 24 images and 8 videos. The registry is growing as new uploads come through the retrofitted upload pipeline.

vc1_derivatives — Size Variants

Every image asset gets derivative sizes generated automatically. Rather than storing derivative paths in vc1_media (the old approach with thumb_small_path and thumb_medium_path columns), derivatives live in their own table keyed by asset_id.

The locked derivative size standard:

  • thumb — 150×150 (square crop)
  • medium — 600px wide
  • large — 1200px wide
  • banner — 1200×400 (landscape crop)
  • square — 600×600 (square crop)

When an application asks for a medium version of asset vc1_20260529_000001, it calls the API with the asset ID and size name. The API checks vc1_derivatives for a ready record, returns the path. If the derivative doesn't exist yet, it returns the original and queues a background job to generate it. The application never sees a broken image and never waits on derivative generation.

vc1_jobs — The Work Queue

Anything that takes time — generating derivatives, transcoding video, batch operations — goes through the job queue rather than happening synchronously during an upload request. A user uploading a 300MB iPhone video should get an immediate success response. The actual transcoding work happens asynchronously.

The jobs table (vc1_jobs) tracks each unit of work through its lifecycle: queued → claimed → processing → done (or failed). The worker_id column records which worker machine claimed a job, preventing two workers from processing the same job simultaneously. The payload and result columns are JSON, allowing arbitrary parameters and output metadata per job type.

The worker architecture is designed for AceMagic mini PCs as dedicated processing nodes. These are cheap, low-power machines purpose-built for the transcoding workload — vc1 receives and registers, the workers chew through the queue independently. Multiple workers can poll the same queue simultaneously without coordination because the claim step uses an atomic UPDATE to prevent race conditions.

The API Abstraction Layer

The key architectural principle driving the new system: applications never query vc1_media directly and never store file paths.

Every interaction goes through the API at video.localad.cc/api/. An application that wants the medium version of an asset calls:

GET /api/media?id=vc1_20260529_000040&size=medium

The API resolves the derivative, returns the URL. The application doesn't know the path. It doesn't know the storage layout. It doesn't know if the file is on /mnt/photos/ or somewhere else. That's entirely vc1's concern. The moment applications store file paths in their own databases, any storage reorganization breaks them. The ID is the contract. Everything else is implementation.

This is the lesson learned the hard way from the legacy system: if PetBlip queries vc1_media directly instead of through an API, the abstraction layer is gone the moment a column is renamed.

The Reuse Velocity Insight

Early in the architecture planning for the new system, there was a temptation to frame the value proposition as a "relationship graph" — a map of which assets appear on which sites, which features reference which images. That framing is technically interesting but it's not the immediate value.

The immediate value is reuse velocity: can an image uploaded to PetBlip show up on VolusiaMarket without re-uploading it? Can a product photo serve the ad system, the blog post, and the stream wall from one source of truth? The relationship graph is a byproduct of solving that problem, not the goal. The goal is one upload, referenced everywhere, with full provenance on the original.

The media_usage table — a ledger of every time an asset is attached or referenced by an application — captures the relationship as a side effect of normal application behavior, not as a primary operation. It gets written when ForumLA attaches an image to a thread, when VolusiaMarket links a photo to a listing. Not at upload time. Populating it at upload time would make every record meaningless because the asset hasn't been used anywhere yet.

vc1-admin: The Unified Interface

One of the frustrations driving the architecture work was the cycle of building separate admin tools. Before the unified system, there were five separate interfaces: a photo gallery admin, a video admin, a media-admin.php general tool, the old per-site admin panels. Each rebuild took weeks. The goal of vc1-admin is to build one interface, correctly, and not rebuild it.

The planned vc1-admin structure covers:

  • Asset Explorer — browse all registered assets, filter by site/type/class/status, view individual asset detail with derivative status
  • Orphan Resolver — the legacy localad_photos database had 135 files on disk with no database record (disk orphans) and records with no corresponding file. The orphan resolver surfaces and resolves these
  • Jobs Dashboard — queue depth, worker status, per-worker activity, failure log
  • Sites Overview — per-site upload volume, storage consumption, asset counts
  • Upload API management — API key registry for each site

Current State and What's Next

As of the current build:

  • localad_media database is live on vc1 with vc1_media, vc1_derivatives, vc1_jobs, media_usage, and stream_events tables created and indexed
  • 32 assets registered in vc1_media (24 images, 8 videos) from current upload activity
  • The generate_media_id() function is live and producing IDs in vc1_YYYYMMDD_NNNNNN format
  • The job worker framework is proven — the claim/process/complete loop has been tested with the queue cycling correctly through states
  • Legacy localad_photos (497 photos) and localad_video remain running in parallel, untouched
  • The old thumb_small_path and thumb_medium_path columns in vc1_media are pending removal once upload.php is retrofitted to write to vc1_derivatives instead
  • vc1-admin shell is the immediate next build milestone

The principle going forward: lock it down, back it up, stop rebuilding what works. The vc1 architecture is defined. The schema is set. New features — AceMagic video workers, derivative engine, site expansion to VolusiaMarket and MarketsFL — build on top of what's there. They don't replace it.

The wall display system and ad development come after vc1-admin is solid. Getting distracted by new features before the foundation is stable is what created the rebuild cycle in the first place. vc1-admin ships, the legacy orphan situation gets cleaned up, storage gets expanded, and then the wall and ad systems have a real media layer to build on.

Log in or register to reply.