mdformat breaks Hugo shortcodes by escaping `<`

[jamesbraza]

Describe the bug

context

Running mdformat on a Markdown file with a Hugo shortcode.

expectation

I expected mdformat to not break the shortcode.

bug

Instead mdformat is breaking the shortcode by escaping < to \<.

problem

This breaks the shortcode for Hugo users.

Relevant Hugo comment: #112 (comment)

Reproduce the bug

{{< youtube -dJolYw8tnk >}}

That links to this YouTube video. mdformat corrects this to

{{\< youtube -dJolYw8tnk >}}

List your environment

mdformat==0.7.16
mdformat-frontmatter==2.0.1

[Oliver-Habersetzer]

I agree this can be annoying. Also "[" and "]" are escaped - so I can't use this for Obsidian. I might make a PR so we're able to configure this behavior.

EDIT by @hukkin:
Obsidian is off-topic for this discussion, but there is mdformat_obsidian that may work for you.

[BartKeulen]

I just want to add another case. I am using mkdocs with PyMarkdown Extension Snippets; useful for embedding content directly in your documentation pages.

A document is included using the following syntax:

--8<-- "filename.ext"

mdformat changes it to:

--8\<-- "filename.ext"

The behavior does not happen inside a code block, the following code is not changed:

\```yaml
--8<-- "filename.yml"
\```

(had to add the escape characters to display it kind of correctly, lol).

EDIT by @hukkin:
mkdocs is off-topic here, but there is a plugin, mdformat-mkdocs. Not sure if it handles the extension you asked about, but you can ask for support in the plugin's issue tracker.

[ALizarazoTellez]

GitHub blocks highlights are escaped too: https://github.com/orgs/community/discussions/16925

Changes:

> [!NOTE]
> Test

With:

> \[!NOTE\] Test

And changes:

> **Note**
> This is a note

With:

> **Note** This is a note

EDIT by @hukkin:
This is off-topic, unrelated to Hugo, but there is mdformat_gfm_alerts plugin for this issue.

[PythonFZ]

Any updates on this?

[mmcdermott]

It would be really nice to have support for turning off this escape option. Is there any progress on this? I've also added an issue on snippets to support this via a plugin, but it'd be great to have more control in mdformat directly.

[hukkin]

Hey, thanks for the issue!

Mdformat is a CommonMark formatter. It doesn't support Hugo, mkdocs, GitHub etc. extensions out of the box. The syntax it does support is listed here https://spec.commonmark.org/0.29/

There is plugin API that can be used to extend the syntax. For mkdocs there is mdformat-mkdocs, and for GitHub the required plugins are listed in the README. For Hugo I'm not aware of an existing plugin yet. See the docs if you are willing to create a plugin.

It would be really nice to have support for turning off this escape option. Is there any progress on this? I've also added an issue on facelessuser/pymdown-extensions#2434 to support this via a plugin, but it'd be great to have more control in mdformat directly.

There is really no "escape option" to turn off. We need the escapes to make sure our formatting changes do not change Markdown "AST". There is an issue for reducing the occasions where escapes are necessary #112

[ecbaldwin]

For those looking for a work-around, I found one that works at least in my case. I don't love it but it enabled me to move on.

Surrounding the shortcode with an inline code block prevents mdformat from escaping anything inside it. The surprising thing to me is the precedence of the shortcode is apparently higher than the precedence of the code block. The side-effect is that there is an empty <code></code> rendered before the output of the shortcode. I was also surprised that the shortcode did not render inside the code block, and maybe this will break in some cases (or in the future).

 `{{< highlight-file "file.yaml" yaml "">}}`