n8n and Meilisearch for lightweight social feeds

2023-07-21 · Ben Williamson

Why I finally wrote this down

Over a few weekends I built n8n flows for a nonprofit animal rescue site. The flows pull Facebook and Instagram posts, upload images to our CDN, and upsert normalized documents into Meilisearch. On each animal profile page, a small InstantSearch widget pre-filters on that animal's name. Visitors get a dedicated-looking feed without building per-animal databases or embedding social iframes.

The pattern generalizes: any lightweight "search this curated stream" UI can stay thin if ingestion and document shape live in automation.

Pipeline shape

Pull, normalize, upsert, read:

  1. Pull: n8n scheduled workflow (API node when available, scrape node when not).

  2. Normalize: Map to a fixed document contract: id, title, body, url, published_at, plus optional subject_id for tenancy.

  3. Upsert/delete: Custom Meilisearch node or HTTP node on a cron; handle deletes when source posts disappear.

  4. Read: Frontend uses Instant-Meilisearch with a search-only API key.

The normalize step is where messy sources become one contract. Hashtags, HTML entities, and platform-specific IDs should not leak into field names. Fix that once in n8n, not in every Vue component.

Why Meilisearch here

Faceting, typo tolerance, and sub-100ms responses are enough for sidebar and card-level feed UX. Operating Elasticsearch for a widget is overkill. Self-hosted Meili on a small VM stayed stable for this traffic class.

Multitenancy without separate indexes: add a subject_id (or similar) on every document and filter on it in each query. Federated multi-search is there when you outgrow single-index filters, but start simple.

Frontend wiring

Vue InstantSearch plus Instant-Meilisearch gets a constrained UI quickly. Each profile page mounts the same component with a different initial filter (animal name or slug). Search relevance does the rest: titles and captions mentioning that name float to the top without hard-coded post lists.

Caveats

  • Platform ToS: Scraping may violate terms; prefer official APIs where they exist.

  • Rate limits: Back off pollers; log HTTP 429 and pause schedules.

  • Key leakage: Search-only keys in the browser can still be extracted. Index only fields you would tolerate exposing.

  • Image hosting: Copy media to your CDN. Hot-linking social URLs breaks when tokens expire.

When to graduate to something bigger

If you need heavy analytics, complex ACL per row, or sub-second indexing at millions of documents, revisit OpenSearch or a managed SaaS. For curated feeds under a few hundred thousand documents, this stack stays boring in the good way.