toolup-forgetoolup-forge
GitHub

Phase 80 — Narrative ⊕ PublicRendering integration

Phase 80 — Narrative ⊕ PublicRendering integration

Status: Shipped. Scope: Schema-breaking extension to the Narrative package; new ContentBody.Narrative variant in PublicRendering; new layout / prerender / structured-data / RSS helpers; new Atom renderer.

What changes

1. InlineSpan (BREAKING)

Adds three cases:

New case Shape Purpose
Link Link of href: string * spans: InlineSpan list Inline anchor. spans are the visible content (so a link can wrap emphasised / coded / metric runs); href is the URL.
Image Image of src: string * alt: string * title: string option Inline image. alt is mandatory (plaintext / RSS / a11y fallback).
Br Br Hard line break inside a paragraph or bullet.

Every existing pattern-match on InlineSpan (including external INarrativeRenderer implementations) gains an incomplete-match warning until the new cases are handled.

2. NarrativeElement (BREAKING)

Adds three cases:

New case Shape Purpose
Heading Heading of level: int * spans: InlineSpan list Sub-section heading (H3–H6 — the document title is implicit H1, the section heading is H2). Levels are clamped to 3..6.
CodeBlock CodeBlock of language: string option * content: string Fenced code block. language is an optional syntax-highlighter hint surfaced as class="language-fsharp" in HTML and a ```fsharp fence in markdown.
Blockquote Blockquote of citation: string option * spans: InlineSpan list Block quotation (testimonials / pull quotes). Semantically distinct from Callout (severity-keyed advisories).

⚠️ Blockquote collides with NarrativeMarkdown.AdmonitionStyle.Blockquote in scope. Inside NarrativeMarkdown.fs, the inner match on AdmonitionStyle now qualifies as | AdmonitionStyle.Blockquote -> and the outer match qualifies as | NarrativeElement.Blockquote(citation, spans) ->. Consumers do not need to update their own match patterns — the qualification is only required where both types are in scope.

3. NarrativeDocument (BREAKING)

Adds two optional fields:

New field Type Purpose
Lang string option BCP-47 language tag ("en-GB", "fr"). Drives <html lang="..."> and og:locale.
CanonicalUrl string option Canonical absolute URL. Drives <link rel="canonical">, og:url, and Atom <link rel="alternate">.

Every record-literal construction of NarrativeDocument ({ Title = ...; Subtitle = ...; Sections = ...; Provenance = ... }) must add the two new fields:

let sectionDoc: NarrativeDocument = {
    Title = e.Document.Title
    Subtitle = e.Document.Subtitle
    Sections = [ section ]
    Provenance = e.Document.Provenance
    Lang = e.Document.Lang             // NEW
    CanonicalUrl = e.Document.CanonicalUrl  // NEW
}

Internal SDK consumers (NarrativeTools.fs, KnowledgeBase/Server/Api/Narrative.fs) are updated as part of this phase.

Narrative.create in NarrativeBuilder.fs initialises both fields to None, so consumers using the builder pipeline are unaffected. Two new pipeline helpers:

let doc =
    Narrative.create "Quarterly outlook"
    |> Narrative.subtitle "Q3 FY26"
    |> Narrative.withLang "en-GB"
    |> Narrative.withCanonicalUrl "https://example.com/q3-fy26"
    |> Narrative.section ...

4. ContentBody (BREAKING, PublicRendering only)

Adds Narrative of NarrativeDocument:

type ContentBody =
    | Markdown of source: string
    | Html of fragment: string
    | Narrative of document: NarrativeDocument   // NEW

Pages whose body is a typed NarrativeDocument — programmatic pages, AI-emitted pages, analytical posts whose body uses the Narrative element set natively — now have a first-class path through the PublicRendering pipeline. Layouts inspect Body and dispatch.

New surface

Pure rendering

  • NarrativeHtml.RenderOptions — ImageLoading (Eager / Lazy → emits loading="lazy"), ExternalLinkRel (auto-applied rel="noopener nofollow" shape for absolute-URL links), HeadingLevelOffset (for nested embedding), EmitSectionAnchors (toggle the <section> wrapper). Defaults preserve byte-for-byte historical output (GP 11). Use NarrativeHtml.renderWith options doc; NarrativeHtml.render doc still calls defaults.
  • NarrativeHtml.tableOfContents — <nav class="narrative-toc"> listing sections (and optionally H3/H4 headings within them) as anchor links to section.Id. Pure string output.
  • NarrativeAtom — Atom 1.0 renderer. renderEntry doc for a single <entry>, renderFeed feedTitle feedSelfUrl feedAlternateUrl docs for a complete <feed>. Registered in NarrativeRenderers.defaults under application/atom+xml.

PublicRendering layout helpers

  • NarrativeLayout.renderBody — projects a PublicPage body into a Giraffe.ViewEngine XmlNode. Markdown / Html bodies pass through; Narrative bodies render through NarrativeHtml.render.
  • NarrativeLayout.articleJsonLd — schema.org Article JSON-LD for the page (Some when body is Narrative + has Provenance).
  • NarrativeLayout.prerenderMeta — PrerenderMeta derived from the Narrative body, ready for ClientConfig.PrerenderRoutes.
  • NarrativeLayout.tableOfContents — <nav> ToC as an XmlNode. None for non-Narrative bodies.
  • NarrativeLayout.headTags — bundled <head> SEO fragment: <link rel="canonical">, JSON-LD, Open Graph, Twitter card. Empty list for non-Narrative bodies, so layouts can unconditionally yield! NarrativeLayout.headTags page inside head [ ... ].

Structured-data helpers (extends StructuredDataHelpers.fs)

  • articleFromNarrative page doc — schema.org Article JSON-LD blob (string option).
  • openGraphFromNarrative page doc — (property, content) list of Open Graph meta tags.
  • twitterCardFromNarrative page doc — (name, content) list of Twitter card meta tags.

Prerender bridge

  • NarrativePrerender.fromDocument — NarrativeDocument → PrerenderMeta (uses synthesised page shape).
  • NarrativePrerender.fromPage page doc — frontmatter-aware variant.

Diff to apply (consumer side)

A. Pattern-matches on InlineSpan / NarrativeElement

Add cases for the new variants. Example for a custom INarrativeRenderer:

let private renderSpan (span: InlineSpan) =
    match span with
    | Text t -> ...
    | Emphasis t -> ...
    | Strong t -> ...
    | Metric(label, value) -> ...
    | Code t -> ...
    // NEW cases:
    | Link(href, spans) -> ...
    | Image(src, alt, title) -> ...
    | Br -> ...

let private renderElement = function
    | Paragraph spans -> ...
    | Heading(level, spans) -> ...           // NEW
    | BulletList items -> ...
    | OrderedList items -> ...
    | KeyValueGrid pairs -> ...
    | Table(columns, rows) -> ...
    | Callout(severity, spans) -> ...
    | CodeBlock(language, content) -> ...    // NEW
    | Blockquote(citation, spans) -> ...     // NEW
    | Divider -> ...

B. Record-literal NarrativeDocument constructors

Add the two new fields (typically Lang = None; CanonicalUrl = None for analytical narratives that don't need them):

let doc: NarrativeDocument = {
    Title = ...
    Subtitle = ...
    Sections = [...]
    Provenance = ...
    Lang = None              // NEW
    CanonicalUrl = None      // NEW
}

Consumers using Narrative.create |> Narrative.section ... need no change.

C. Layout authors adopting Narrative-bodied pages (additive)

let myLayout (page: PublicPage) : XmlNode =
    html [ _lang (page.Body |> function | Narrative d -> d.Lang |> Option.defaultValue "en" | _ -> "en") ] [
        head [] [
            title [] [ encodedText page.Title ]
            yield! NarrativeLayout.headTags page   // canonical + JSON-LD + OG + Twitter
            // ... your own <head> additions
        ]
        body [] [
            // ... your own header / nav
            NarrativeLayout.renderBody page
            match NarrativeLayout.tableOfContents false page with
            | Some toc -> aside [ _class "toc" ] [ toc ]
            | None -> rawText ""
            // ... your own footer
        ]
    ]

Verification

  1. dotnet build ToolUp.Forge.sln — clean. All four core renderers (HTML / Markdown / Plaintext / Feliz client) + Atom renderer + internal consumers (AI tool registry, KB ingestion) compile.
  2. Byte-for-byte parity check — render a pre-Phase-80 NarrativeDocument (no Link / Image / Br / Heading / CodeBlock / Blockquote / Lang / CanonicalUrl) through NarrativeHtml.render / NarrativeMarkdown.render / NarrativePlaintext.render. Output must match the pre-Phase-80 byte-for-byte (GP 11).
  3. PublicRendering layout smoke test — a PublicPage with Body = Narrative doc rendered through a layout calling NarrativeLayout.renderBody produces an <article> with the document content; NarrativeLayout.headTags adds canonical + OG + Twitter + JSON-LD when provenance is present.
  4. Atom feed verification — NarrativeAtom.renderFeed produces a parseable Atom 1.0 document (verify with xmllint --noout feed.xml or a feed validator).

Rollback

Revert the commit. The schema additions are all backwards-incompatible — there is no graceful runtime rollback because record-literal constructors and pattern-matches against the extended types will fail to compile against the pre-Phase-80 shape. If a downstream consumer cannot adopt the schema bump on its own timeline, pin to the prior forge SDK version (ToolUp.Sdk.Version prior to this phase's release) until they catch up.

Why this lands together

The five schema additions (Link / Image / Heading / CodeBlock / Blockquote / Br on InlineSpan-and-Element + Lang / CanonicalUrl on Document) all require the same migration shape — every external INarrativeRenderer impl recompiles, every record-literal constructor recompiles. Bundling them as one phase costs one migration doc + one adoption-matrix row + one consumer-side audit. Splitting them into N phases pays N×each.

The PublicRendering integration (ContentBody.Narrative variant + layout helpers + Atom renderer + RenderOptions + ToC + OG/Twitter helpers + prerender bridge) is the user-facing capability that motivates the schema changes — together they make NarrativeDocument a credible marketing-page primitive, programmatic page source, and AI-emitted-page target, not just an analytics output shape.

Phase 80a — publish_narrative AI tool + content-negotiated export

Status: Shipped. Scope: Wires the two highest-leverage substrate pieces called out in Phase 80's "honest gap" so the AI authoring loop and multi-channel distribution are end-to-end.

What changes (additive — no consumer migration required)

1. INarrativePagePublisher substrate seam (Platform.Server)

New interface in INarrativeStore.fs:

type NarrativePublishOutcome =
    | PublishSucceeded of slug: string
    | PublishFailed of reason: string

type INarrativePagePublisher =
    abstract member PublishAsync:
        slug: string *
        titleOverride: string option *
        descriptionOverride: string option *
        layoutHint: string option *
        document: NarrativeDocument ->
            Async<NarrativePublishOutcome>

Lives in Platform.Server so both ToolUp.AI.Server (which calls it from the new AI tool) and ToolUp.PublicRendering (which provides the implementation) can reference it without depending on each other.

2. publish_narrative AI tool (AI.Server)

New built-in tool in NarrativeTools.fs, auto-registered by composeWithAI:

  • Parameters: id (NarrativeId GUID), slug, optional title / description / layout / canonicalUrl / lang overrides.
  • Behaviour: fetches the narrative from INarrativeStore (user's scope), applies overrides onto the document, resolves INarrativePagePublisher from DI, calls PublishAsync. Returns the canonical slug on success, error JSON on failure.
  • Graceful degradation: when no INarrativePagePublisher is registered (deployment without PublicRendering, or with PublicRendering disabled), returns {"error":"No INarrativePagePublisher is registered..."} — the tool stays callable but no-ops.

3. PublicRenderingNarrativePagePublisher default impl (PublicRendering)

New file NarrativePagePublisher.fs. Constructs a PublicPage envelope with Body = Narrative document, sets the layout from the caller's hint (falling back to the first-registered layout), writes through IEntityStore<PublicPageEntity>.Save to the _public scope. Registered as a DI singleton by PublicRenderingCompose.run when public rendering is enabled.

4. Content-negotiated export handler (PublicRendering)

New file NarrativeExportHandler.fs. Mounted before the default page handler in the compose chain. Triggered by ?format= query parameter; falls through to the standard HTML page handler when the parameter is absent.

Supported combinations:

Body Supported ?format= Content-Type
Narrative html, md / markdown, txt / plain, atom matches the format
Markdown md / markdown text/markdown; charset=utf-8
Html html text/html; charset=utf-8

Unsupported body/format pairs return 415 with a small JSON body listing the formats available for that body kind.

Compose-time wiring (zero consumer change)

PublicRenderingCompose.run now:

  1. Registers INarrativePagePublisher as a DI singleton (resolves IEntityStore from DI per-request so any decorator the consumer wires participates).
  2. Mounts NarrativeExportHandler between the redirect handler and the page handler.

A deployment that already composed withPublicRendering and now also composes withAI gets publish_narrative and ?format= wiring automatically. No call-site changes.

Diff to apply (consumer side)

None. The Phase 80a surface is purely additive. Consumers that don't want the AI tool can leave their composeWithAI unwired; consumers that don't want the export query can leave ?format= unused. Existing routes continue to serve HTML by default.

Verification

  1. dotnet build ToolUp.Forge.sln — clean. Phase 80a touches three sibling packages (Platform.Server, AI.Server, PublicRendering) plus the workspace adoption matrix.
  2. AI tool registration — composeWithAI registers the new publish_narrative tool. Verify by inspecting AIToolRegistry.GetAll() at startup; it should include the four narrative tools (list_narratives, get_narrative, get_narrative_section, publish_narrative).
  3. End-to-end publish — in a deployment with both AI and PublicRendering composed: ask the assistant to "publish the last narrative I generated at /blog/test". The assistant calls list_narratives → publish_narrative(id, "blog/test"). Subsequent GET /blog/test returns the page.
  4. ?format= export — curl http://your-deploy/blog/test?format=atom returns the page as an Atom entry; ?format=md returns markdown; ?format=html returns the HTML article fragment (not the layout shell).
  5. 415 path — curl http://your-deploy/marketing-page?format=atom (where /marketing-page is a Markdown-bodied page) returns 415 with {"error":"unsupported format...","supported":["md"]}.
  6. Strip-imports — a deployment with PublicRendering disabled (ServerConfig.PublicRendering = NoPublicRendering) gets no INarrativePagePublisher registration. The AI tool's publish_narrative returns the "no publisher registered" error. No runtime cost when unused.

Rollback

Phase 80a is additive — revert the commit. Deployments that already started using publish_narrative would lose the tool but the published pages remain in IEntityStore (unaffected by reverting the publisher).

What this leaves open (productisation, not substrate)

→ All four items below closed in Phase 80b (below). Substrate-side gap section retained for historical reference.

  • Authorisation gating. Closed by withAIPublishEnabled toggle + withAIPublishAuthoriser per-request gate (Phase 80b).
  • Collision policy. Closed by SlugCollisionPolicy parameter on INarrativePagePublisher.PublishAsync + collisionPolicy AI tool parameter (Phase 80b).
  • Layout discovery. Closed by ILayoutCatalog + list_layouts AI tool (Phase 80b).
  • Feed aggregation. Closed by NarrativeFeedHandler + withFeed compose helper (Phase 80b).

Phase 80b — Gating + collision policy + layout discovery + feed aggregation

Status: Shipped. Scope: Closes the four productisation gaps Phase 80a left open. All additive on the substrate side; one behavioural change to the Phase 80a auto-registration noted below.

What changes

1. Compose-time gating for publish_narrative (BEHAVIOURAL — supersedes Phase 80a default)

Phase 80a unconditionally registered INarrativePagePublisher in DI when public rendering was enabled. Phase 80b makes that registration conditional on a compose-time toggle:

let app =
    PublicRenderingServerApp.create ()
    |> PublicRenderingServerApp.withConfig serverConfig
    |> PublicRenderingServerApp.withLayout (LayoutName "default") defaultLayout
    |> PublicRenderingServerApp.withAIPublishEnabled true           // NEW — required to expose publish_narrative
    |> PublicRenderingServerApp.withAIPublishAuthoriser authoriser  // NEW — optional per-request gate
  • withAIPublishEnabled false (the new default) — INarrativePagePublisher is not registered; publish_narrative returns "no publisher registered, or not enabled via withAIPublishEnabled true".
  • withAIPublishEnabled true — publisher is registered; if no authoriser is wired, every AI request that resolves the tool can publish.
  • withAIPublishAuthoriser (fun ctx -> async { return ... }) — per-request gate. false returns cause publish_narrative to refuse with an unauthorised error before the publisher is invoked.

The authoriser signature is HttpContext -> Async<bool>, so deployments can plumb in RBAC role checks, per-team permissions, claim inspection, or any other gating policy without the substrate taking a hard dep on IPermissionStore.

Migration note for Phase 80a adopters: if you started using publish_narrative between Phase 80a and 80b (unlikely — both shipped same session), add withAIPublishEnabled true to your compose root.

2. SlugCollisionPolicy on PublishAsync (BREAKING signature change)

INarrativePagePublisher.PublishAsync gains a collisionPolicy: SlugCollisionPolicy parameter (in fifth position, before document):

type SlugCollisionPolicy =
    | OverwriteExisting     // Phase 80a behaviour
    | RejectIfExists
    | AutoSuffix
  • OverwriteExisting — write regardless; previous version remains in IDataObjectStore history (recoverable via IEntityStore.ListVersions / GetVersion).
  • RejectIfExists — check first; return PublishFailed if any page already lives at the slug.
  • AutoSuffix — try slug, slug-2, slug-3, … up to 100 attempts; PublishSucceeded carries the slug actually used.

The AI tool gains an optional collisionPolicy parameter ("overwrite" / "reject" / "suffix" — default "overwrite" for back-compat).

External INarrativePagePublisher implementations recompile with a missing-parameter error and must update their PublishAsync signatures.

3. ILayoutCatalog + list_layouts AI tool

type ILayoutCatalog =
    abstract member ListLayoutNames: unit -> string list

Lives in Platform.Server alongside INarrativePagePublisher. Default implementation in PublicRendering (PublicRenderingLayoutCatalog) is backed by the layouts registered via PublicRenderingServerApp.withLayout. Registered as a DI singleton by PublicRenderingCompose.run whenever public rendering is enabled (no gating — read-only).

New AI tool list_layouts resolves the catalog and returns the registered names. Empty result + "note":"no layout catalog registered" when called against a deployment without PublicRendering.

Use case: assistant calls list_layouts → picks a sensible layout argument for publish_narrative instead of falling back silently to the first-registered one.

4. NarrativeFeedHandler + withFeed compose helper

let app =
    PublicRenderingServerApp.create ()
    // ...
    |> PublicRenderingServerApp.withFeed {
        NarrativeFeedConfig.defaults with
            Title = "ACME Blog"
            SelfUrl = "/blog.atom"
            AlternateUrl = "https://acme.example/blog"
            Collection = Some "blog"
            MaxEntries = 20
       }

Each registered feed mounts a route handler at its SelfUrl. The handler:

  1. Walks IPublicContentApi.ListPages("") for file-loaded pages.
  2. Walks IEntityStore.ListAll<PublicPageEntity> for entity-store-overlay pages (where publish_narrative writes).
  3. Filters to Narrative-bodied pages matching the optional Collection filter.
  4. Sorts by PublishedAt descending (None last); caps at MaxEntries.
  5. Renders via NarrativeAtom.renderFeed.

Non-Narrative bodies are skipped. For v1 this is acceptable — feed consumers usually want Narrative-shaped structured content. A future Markdown-body-to-feed-entry adapter is non-breaking when added.

Caller can register multiple feeds (one whole-site + one per collection, typically).

Diff to apply (consumer side)

A. External INarrativePagePublisher implementations

Add the collisionPolicy parameter:

// Before (Phase 80a)
member _.PublishAsync(slug, titleOverride, descOverride, layoutHint, doc) = async { ... }

// After (Phase 80b)
member _.PublishAsync(slug, titleOverride, descOverride, layoutHint, collisionPolicy, doc) = async {
    // Honour collisionPolicy:
    //   - OverwriteExisting → write regardless (Phase 80a behaviour)
    //   - RejectIfExists → check existing first, return PublishFailed if occupied
    //   - AutoSuffix → walk slug, slug-2, slug-3, … until free
}

B. Direct INarrativePagePublisher.PublishAsync callers (rare — usually only the AI tool)

Add collisionPolicy to the call site. For back-compat with Phase 80a behaviour pass OverwriteExisting.

C. Compose roots that want to use publish_narrative from AI

Add withAIPublishEnabled true (and optionally withAIPublishAuthoriser):

let app =
    PublicRenderingServerApp.create ()
    |> PublicRenderingServerApp.withConfig serverConfig
    |> PublicRenderingServerApp.withLayout (LayoutName "default") defaultLayout
    |> PublicRenderingServerApp.withAIPublishEnabled true
    // optionally:
    |> PublicRenderingServerApp.withAIPublishAuthoriser (fun ctx -> async {
        // Inspect ctx.User claims, ctx.Items["ToolUp.StorageScope"], etc.
        // Return true for permitted subjects, false otherwise.
        return true
    })

D. Compose roots that want Atom feeds

Add one or more withFeed calls:

let app =
    PublicRenderingServerApp.create ()
    // ...
    |> PublicRenderingServerApp.withFeed { NarrativeFeedConfig.defaults with Title = "..."; SelfUrl = "/feed.atom"; AlternateUrl = "..." }

Verification

  1. dotnet build ToolUp.Forge.sln — clean. Phase 80b touches the same three sibling packages as 80a (Platform.Server, AI.Server, PublicRendering).
  2. Gating off (default) — deployment without withAIPublishEnabled true exposes publish_narrative to AI but every call returns the "not registered" error.
  3. Gating on, no authoriser — every AI user can publish.
  4. Gating on with authoriser refusing — publish_narrative returns the "unauthorised" error before invoking the publisher.
  5. Collision policy reject — publish to occupied slug returns the RejectIfExists error.
  6. Collision policy suffix — publish to occupied slug returns success with a -2 (or higher) suffix in the response.
  7. list_layouts — returns the registered layout names; returns {"layouts":[],"note":"no layout catalog registered"} when PublicRendering isn't wired in.
  8. Feed handler — GET /feed.atom returns Atom XML with entries for every Narrative-bodied page published via publish_narrative (or registered via file-loaded markdown that uses ContentBody.Narrative). Sorted newest first.

Rollback

Phase 80b changes are mostly additive; the breaking item is the PublishAsync signature. Rolling back means:

  • Drop the collisionPolicy parameter from external implementations.
  • Drop the new with* compose helpers from compose roots that adopted them.
  • Phase 80a's unconditional INarrativePagePublisher registration returns (the withAIPublishEnabled gate goes away).

What this leaves truly open

All substrate gaps are now closed. Remaining items are productisation polish:

  • AI authoring UX. The AI assistant emits NarrativeDocuments as opaque records; richer streaming construction (per-section emit, per-element validation) would tighten the author loop.
  • Feed sources beyond Narrative bodies. Markdown/Html bodies are skipped from feeds; a per-body summarisation pass would surface them too.
  • Collection-aware indexing. ListAll<PublicPageEntity> doesn't currently index by Collection; for high-volume sites a denormalised collection index would beat the per-request filter scan.