Engineer by craft. Builder by design.

Writing / AI Chat

Don't parse follow-ups out of the reply

Streaming chat truncates. Markdown JSON fences rot. Suggested next steps belong in a typed side channel.

Back to writing
MCMukul Chugh
2 min read
x

Chat products love "suggested next steps." Teams often stuff them into a fenced JSON blob inside the assistant message. Then the stream truncates, the fence breaks, and the UI shows nothing, or a half-parsed ghost.

Why the JSON-in-markdown pattern breaks

One reply carries two jobs

An answer for a person…

{"action":
Stream cut short
  • Truncation breaks the JSON fence
  • Malformed data reaches the renderer
  • Targets may never have been fetched

The assistant's reply is a stream meant for a human to read, and it gets edited in flight: markdown renderers reflow it, streaming can cut off mid-token, and some UIs summarize or truncate long replies before showing them. A JSON fence surviving all of that intact is a coincidence, not a guarantee. When it breaks, the failure is silent: no error, just a button that never appears or a stray {"action": leaking into the visible text.

There's a second problem underneath the parsing one: a JSON blob the model wrote from scratch has no relationship to what actually happened in the turn. Nothing stops it from suggesting an action against a record the turn never looked up.

A typed side channel

Separate prose from actions

Assistant reply

Natural language, streamed for the reader.

suggest_actions

Required fields, known action, valid targetId.

The server validates the tool call before the UI sees it.

Emit suggestions as a real tool call, not prose:

ts
Download
suggest_actions({
  actions: [
    { label: "Open ticket #4821", action: "open_ticket", targetId: "4821" },
    { label: "Retry the failed sync", action: "retry_sync", targetId: "sync_9f2" }
  ]
})

The server validates the schema before the UI ever sees it: required fields, known action enum, targetId shaped right. A malformed call fails the tool invocation, not the render.

Validate against what the turn actually touched

Ground actions in this turn
IDs fetched, created, or modified
Is targetId in that set?
Yes: show action
No: reject it

The schema check catches shape; a second check catches truth. Keep a set of IDs the turn actually fetched, created, or modified, and reject any suggested action whose targetId isn't in that set. This kills the class of bug where the model suggests acting on something it never actually saw: hallucinated ticket numbers, IDs from a previous turn, records that don't exist.

Fewer, clearer suggestions beat three clones of "look into this." Structured side channels survive streaming. Prose parsers don't.

What should
we make next?

Bring the idea you keep coming back to.
Let’s see where a conversation takes it.

15 minutes.

An idea is enough.