Skip to content

Chat markdown: ASCII-smiley conversion turns :* and :D into emoji, corrupting agent output #402

Description

@sierragolflima

Summary

Markdig's ASCII-smiley conversion is enabled in the shared markdown pipeline. When
the model (or any author) writes a colon immediately followed by * or a capital D
with no intervening space, Markdig interprets it as an emoticon and replaces it with an
emoji. This silently corrupts normal, correct markdown in the MeshWeaver AI chat (and
every other rendered markdown surface): the colon and the following character disappear,
the emoji appears, and any adjacent **bold** breaks because one of its * was eaten.

Reproduction / observed

Model output (correct markdown, colon directly followed by ** or a D-word):

Batch 3 — Erhebungswerte 10000–10102 patchen (13 Knoten):**Batch 4** — …
… (13 Knoten):Die Knoten 10213, 10216–10224 existieren noch nicht.

Rendered in the chat:

Batch 3 — Erhebungswerte 10000–10102 patchen (13 Knoten)😗*Batch 4** — …
… (13 Knoten)😄ie Knoten 10213, 10216–10224 existieren noch nicht.

Expected:

Batch 3 — Erhebungswerte 10000–10102 patchen (13 Knoten): **Batch 4** — …
… (13 Knoten): Die Knoten 10213, 10216–10224 existieren noch nicht.

The mechanics:

  • :* → 😗 (kissing face). The colon and the first * of **Batch 4** are consumed,
    leaving an unbalanced *Batch 4**, so the bold no longer renders.
  • :D → 😄 (grinning face). :Die matches the :D emoticon, leaving a bare ie.

Any colon that lands next to one of Markdig's emoticon trigger characters is exposed:
:*, :D, :P, :p, :O, :o, :/, :\, :(, :) and so on. In agent output —
which is dense with Label: introducing bold text, lists, and capitalized words — these
collisions are common, whereas a deliberately-typed emoticon almost always has a leading
space.

Root cause

src/MeshWeaver.Markdown/MarkdownExtensions.cs:74

.UseEmojiAndSmiley(TableSafeEmojiMapping)

TableSafeEmojiMapping (same file, lines 21–25) keeps every emoji :shortcode: and
every ASCII smiley except the pipe-bearing ones
(:|, :-| → 😐, dropped because they
broke pipe-table alignment delimiters — the 2026-06-13 bug documented in the comment).
That earlier fix treated one symptom of the same feature. The underlying problem is the
ASCII-smiley conversion itself: it produces false positives on ordinary punctuation.

This pipeline is the single markdown pipeline for the whole platform
(MarkdownExtensions.CreateMarkdownPipeline), used by CollaborativeMarkdownView (the
chat renderer), MarkdownViewLogic, MarkdownContent, the content collections, and MAUI —
so the corruption affects all rendered markdown, not only chat.

Proposed fix (recommended)

Disable ASCII-smiley conversion; keep the unambiguous :shortcode: emoji (:smile: → 😄,
:warning:⚠️, …):

// emoji :shortcodes: only — ASCII smileys (:D, :*, :|, …) collide with normal
// colon-adjacent markdown and corrupt output. Real emoji still work via :shortcode:.
.UseEmojiAndSmiley(enableSmileys: false)   // == EmojiMapping.DefaultEmojisOnlyMapping

This removes the entire false-positive class in one line, and lets us delete the
TableSafeEmojiMapping workaround
— the pipe-table :| → 😐 bug is subsumed by the same
change, since :| is an ASCII smiley too. :shortcode: emoji are unaffected.

Trade-off: authors who type :) in a message will no longer get 🙂 auto-converted. In a
technical/document context that is acceptable (and arguably preferable); the emoji
shortcode (:smile:, :wink:) remains available for intentional emoji.

Alternative (narrower, not recommended)

Keep smileys enabled but extend the existing denylist filter to also drop the ambiguous
triggers (:*, :D, :P, :p, :O, :o, :/, :\, …). This preserves :)/:(
but remains whack-a-mole: every emoticon whose trigger can sit against real text is a
latent corruption, and the list has to be maintained by hand.

Scope / impact

  • Affects all rendered markdown; most visible in the AI chat because agent output is
    colon-dense and mixes colons with bold and capitalized words.
  • No data is lost at the source — the stored markdown is correct; only the render is
    wrong — so the fix is render-only and retroactively corrects all existing content.

Acceptance criteria

  • Rendering (13 Knoten):**Batch 4** produces (13 Knoten): followed by a bold
    Batch 4 — no 😗, bold intact.
  • Rendering (13 Knoten):Die Knoten … produces the literal text — no 😄.
  • Rendering a right-aligned pipe table (|---:|) still renders as a table — no 😐
    (the earlier regression stays fixed).
  • :smile: / :warning: :shortcode: emoji still convert.
  • Add a unit test in the markdown test project covering the three cases above.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions