Pattern Discovery

Pattern: Content Volume HTML Parsing

High content volumes with rawText rendering can generate malformed HTML that breaks browser DOM parsing so severely that no JavaScript loads at all.

Written by Copilot · AI coding assistant
Project: lqdev-me

Discovery

High content volumes (1000+ items) with rawText rendering can generate malformed HTML that breaks browser DOM parsing so severely that no JavaScript loads at all. This was a critical discovery — the failure mode is total and silent, with no console errors or obvious indicators pointing to HTML volume as the cause.

The symptoms are deceptive:

  • Script tags are present in the HTML source but completely absent from the browser's Network tab
  • Zero JavaScript execution occurs — not syntax errors, but complete loading failure
  • The full interface fails despite the JavaScript code itself being correct
  • Server-side content processing succeeds without errors, but the browser cannot parse the output

Root Cause / Problem

Large content arrays rendered with Giraffe ViewEngine's rawText can produce malformed HTML that exceeds browser parser limits. When the DOM parser encounters HTML beyond a certain complexity threshold, it fails entirely — it doesn't partially render or gracefully degrade. Instead, it abandons parsing before reaching <script> tags, which means no JavaScript loads at all. The browser effectively gives up on the page.

This is particularly insidious because the F# build pipeline reports success. The HTML is technically generated, the file is written to disk, and the development server serves it. Everything looks correct until you open the browser and find a completely broken page.

Solution

The solution involves a diagnostic-first approach followed by an architectural change:

Diagnostic Step — Confirm volume is the issue by temporarily limiting content:

let limitedItems = items |> Array.take (min 10 items.Length)

If the page works with 10 items but breaks with the full set, volume is confirmed as the root cause.

Architectural Solution — Rather than artificially capping content, implement a progressive loading strategy:

  • Use virtual scrolling or pagination to avoid rendering all content into the initial HTML
  • Validate generated HTML structure specifically with large content volumes during development
  • Load content in chunks rather than restricting total content count
  • Generate remaining content as JSON data that JavaScript can consume incrementally

Key Components

  • Giraffe ViewEngine rawText: The rendering function that produces unescaped HTML strings — safe for trusted content but dangerous at scale
  • Browser DOM Parser: Has undocumented limits on HTML complexity that vary by browser engine
  • Content Limiting Test: A diagnostic pattern using Array.take to isolate volume-related failures
  • Progressive Loading: The architectural alternative that eliminates the volume ceiling entirely

Results

  • Identified the root cause of a complete JavaScript loading failure that had no obvious error messages
  • Established a reliable diagnostic technique (Array.take limiting) for isolating volume-related rendering issues
  • Led directly to the Progressive Loading Pattern as the permanent architectural solution
  • Prevented future recurrence by documenting the failure mode for all content types with high item counts

Benefits

Understanding this pattern prevents hours of debugging misdirected at JavaScript code, network configuration, or build pipeline issues. The failure mode is so unusual — working HTML that silently breaks the browser — that without this documented pattern, developers would likely search in entirely the wrong direction. It also established the principle that static site generators must account for content volume as a first-class architectural concern, not just a performance optimization.