adam/vscode-front-matter
1
0 Fork 0
mirror of https://github.com/estruyf/vscode-front-matter.git synced 2026-11-04 09:32:21 +01:00
Code Issues 77 Packages Projects Releases Wiki Activity

Enhancement: 😀 Support for YAML non-printing characters #426

New Issue
Closed
opened 2026-10-05 02:46:31 +02:00 by adam · 21 comments
adam commented 2026-10-05 02:46:31 +02:00
Owner
Copy link
Copy source

Related to the following Discord thread: https://discord.com/channels/992409023607476224/1085255612939640832

Very often, the YAML front matter includes text that will display on a page, like the following:

title: Hello World 🌎

However, the proper YAML would be:
title: Hello World \U0001F4CD

FM currently adds the non-printable character (like an emoji) to the text, which errors out on build since it is not properly formatted YAML

I can understand that FM would not want to modify an existing file, but there is an opportunity to catch non-printable characters when creating/modifying metadata in the Front Matter rail

Author: @BillRaymond

Related to the following Discord thread: https://discord.com/channels/992409023607476224/1085255612939640832 Very often, the YAML front matter includes text that will display on a page, like the following: title: Hello World 🌎 However, the proper YAML would be: title: Hello World \U0001F4CD FM currently adds the non-printable character (like an emoji) to the text, which errors out on build since it is not properly formatted YAML I can understand that FM would not want to modify an existing file, but there is an opportunity to catch non-printable characters when creating/modifying metadata in the Front Matter rail Author: @BillRaymond <!-- github-sync-id:1624980968 -->
adam closed this issue 2026-10-05 02:46:31 +02:00
adam commented 2026-10-05 02:46:32 +02:00
Author
Owner
Copy link
Copy source

Adding the encodeEmoji property to the string field. When enabling this property on the field, it encodes all emojis.

Example: Hello World 🌎 becomes Hello World \u1F30E.

Verified against: https://unicodeplus.com/U+1F30E

Adding the `encodeEmoji` property to the string field. When enabling this property on the field, it encodes all emojis. Example: `Hello World 🌎` becomes `Hello World \u1F30E`. Verified against: https://unicodeplus.com/U+1F30E <!-- github-comment-id:1476073560 -->
adam commented 2026-10-05 02:46:32 +02:00
Author
Owner
Copy link
Copy source

Isn't

title: "Hello World 🌏"

possible/valid too?

Isn't title: "Hello World :earth_asia:" possible/valid too? <!-- github-comment-id:1476129051 -->
adam commented 2026-10-05 02:46:33 +02:00
Author
Owner
Copy link
Copy source

Yes @davidsneighbour, but for @BillRaymond it isn't it seems. That is why an extra property is introduced. If your SSG/framework wouldn't support it, you can let FM encode it for you.

Yes @davidsneighbour, but for @BillRaymond it isn't it seems. That is why an extra property is introduced. If your SSG/framework wouldn't support it, you can let FM encode it for you. <!-- github-comment-id:1476134783 -->
adam commented 2026-10-05 02:46:34 +02:00
Author
Owner
Copy link
Copy source

By default, it will keep using Hello World 🌏, only if you need to encode the emoji, you will have to tell FM to do this.

By default, it will keep using `Hello World 🌏`, only if you need to encode the emoji, you will have to tell FM to do this. <!-- github-comment-id:1476136114 -->
adam commented 2026-10-05 02:46:34 +02:00
Author
Owner
Copy link
Copy source

Isn't

title: "Hello World 🌏"

possible/valid too?

While you can add emojis and other non-printable characters to YAML, they are technically not allowed. See here. Some YAML processors handle this for you, but others do not. In my specific situation, I use Jekyll (also the default for GitHub Pages), which errors out. That is why I requested that FM provide a conversion option.

> Isn't > > title: "Hello World 🌏" > > possible/valid too? While you can add emojis and other non-printable characters to YAML, they are technically not allowed. See [here](https://yaml.org/spec/1.2.2/#character-set). Some YAML processors handle this for you, but others do not. In my specific situation, I use Jekyll (also the default for GitHub Pages), which errors out. That is why I requested that FM provide a conversion option. <!-- github-comment-id:1476631600 -->
adam commented 2026-10-05 02:46:35 +02:00
Author
Owner
Copy link
Copy source

Thank you!

Thank you! <!-- github-comment-id:1476634748 -->
adam commented 2026-10-05 02:46:36 +02:00
Author
Owner
Copy link
Copy source

Since you are still working on this enhancement, I did some testing on my local machine using Front Matter. I noticed you only sometimes add quotations around strings. However, you need the quotes if you add escape codes to display non-printable characters.

For example:

title: \U0001F30E The title of my post

Outputs:

\U0001F30E The title of my post

But, if you put the title in quotes, you get the non-printable character. For example:

title: "\U0001F30E The title of my post"

Outputs:

🌎 The title of my post

I hope this will help you during your development process.

Since you are still working on this enhancement, I did some testing on my local machine using Front Matter. I noticed you only sometimes add quotations around strings. However, you need the quotes if you add escape codes to display non-printable characters. For example: ``` title: \U0001F30E The title of my post ``` Outputs: ``` \U0001F30E The title of my post ``` But, if you put the title in quotes, you get the non-printable character. For example: ``` title: "\U0001F30E The title of my post" ``` Outputs: ``` 🌎 The title of my post ``` I hope this will help you during your development process. <!-- github-comment-id:1476943767 -->
adam commented 2026-10-05 02:46:36 +02:00
Author
Owner
Copy link
Copy source

Thanks @BillRaymond, it is the YAML parser which decides to add quotes or not, will take a look why and when.

Thanks @BillRaymond, it is the YAML parser which decides to add quotes or not, will take a look why and when. <!-- github-comment-id:1477433327 -->
adam commented 2026-10-05 02:46:37 +02:00
Author
Owner
Copy link
Copy source

I have done some tests, and in my environment, when double quotes are added, it will use a double backslash, which is how it should be to correctly parse the string.

https://user-images.githubusercontent.com/2900833/226557345-c5af9121-9e85-44e9-8546-3da487597e7c.mov

I have done some tests, and in my environment, when double quotes are added, it will use a double backslash, which is how it should be to correctly parse the string. https://user-images.githubusercontent.com/2900833/226557345-c5af9121-9e85-44e9-8546-3da487597e7c.mov <!-- github-comment-id:1477466521 -->
adam commented 2026-10-05 02:46:38 +02:00
Author
Owner
Copy link
Copy source

In this example, I add the Unicode escape character into the title. FM does not automatically put the title in quotes.

If the YAML is not in quotes, the Unicode is processed as regular text and does not display the non-printable character.

However, if I manually put the title in quotes, the proper non-printable character is processed and displays on the website.

emoji-no-yaml-quotes

In this example, I add the Unicode escape character into the title. FM does not automatically put the title in quotes. If the YAML is not in quotes, the Unicode is processed as regular text and does not display the non-printable character. However, if I manually put the title in quotes, the proper non-printable character is processed and displays on the website. ![emoji-no-yaml-quotes](https://user-images.githubusercontent.com/15895828/226780598-d588b517-0ee2-45be-abbf-be1abf0fa908.gif) <!-- github-comment-id:1478810265 -->
adam commented 2026-10-05 02:46:38 +02:00
Author
Owner
Copy link
Copy source

Although it makes it work for your use case, it is not valid for a JSON object. We will have to think about how we can get this supported, as the double backslash is the right way to encode it.

Although it makes it work for your use case, it is not valid for a JSON object. We will have to think about how we can get this supported, as the double backslash is the right way to encode it. <!-- github-comment-id:1479089927 -->
adam commented 2026-10-05 02:46:39 +02:00
Author
Owner
Copy link
Copy source

Like I thought, it becomes invalid YAML for the parser:

image

Like I thought, it becomes invalid YAML for the parser: ![image](https://user-images.githubusercontent.com/2900833/226848383-f26b4f8a-640f-487e-bbe4-16f64f82b256.png) <!-- github-comment-id:1479119746 -->
adam commented 2026-10-05 02:46:40 +02:00
Author
Owner
Copy link
Copy source

I just did a test on Jekyll, and I can just use 🌎 in the title:

image

---
layout: post
title:  🌎 Welcome to Jekyll!
date:   2022-04-21 11:44:23 +0200
categories: jekyll update
---

Can it be related to your theme?

I just did a test on Jekyll, and I can just use `🌎` in the title: ![image](https://user-images.githubusercontent.com/2900833/226857767-9928f0d7-2ae4-471a-b545-97ca6e05de8f.png) ```markdown --- layout: post title: 🌎 Welcome to Jekyll! date: 2022-04-21 11:44:23 +0200 categories: jekyll update --- ``` Can it be related to your theme? <!-- github-comment-id:1479191896 -->
adam commented 2026-10-05 02:46:41 +02:00
Author
Owner
Copy link
Copy source

I do not have a theme (I started from the default Minima, but it does not use anything custom). Was the way I added the emoji a problem? I am unsure about that question, so I will run some tests and remove any plugins I may not use.

This still seems valid to look at since YAML does not support non-printable characters and could cause portability issues.

I do not have a theme (I started from the default Minima, but it does not use anything custom). Was the way I added the emoji a problem? I am unsure about that question, so I will run some tests and remove any plugins I may not use. This still seems valid to look at since YAML does not support non-printable characters and could cause portability issues. <!-- github-comment-id:1479908512 -->
adam commented 2026-10-05 02:46:41 +02:00
Author
Owner
Copy link
Copy source

It seems that backslashes and YAML is a big topic, most recommend to avoid using backslashes. That said, the only valid way to have a backslash in the JS and Python YAML parser is to use two backslashes when using quotes. This is what the parser will automatically add while parsing.

It seems that backslashes and YAML is a big topic, most recommend to avoid using backslashes. That said, the only valid way to have a backslash in the JS and Python YAML parser is to use two backslashes when using quotes. This is what the parser will automatically add while parsing. <!-- github-comment-id:1479921983 -->
adam commented 2026-10-05 02:46:42 +02:00
Author
Owner
Copy link
Copy source

Ah, so it still needs to be in quotes, but also use \\ is the proper approach?

Ah, so it still needs to be in quotes, but also use `\\` is the proper approach? <!-- github-comment-id:1479925982 -->
adam commented 2026-10-05 02:46:43 +02:00
Author
Owner
Copy link
Copy source

image

It is also in the YAML docs.

![image](https://user-images.githubusercontent.com/2900833/226979369-86b6ce7f-134f-4b88-8d1f-4d5db65d3eeb.png) It is also in the YAML docs. <!-- github-comment-id:1479926990 -->
adam commented 2026-10-05 02:46:43 +02:00
Author
Owner
Copy link
Copy source

Okay, I am sorry I led you astray with the single \. That is what I get for using SO and not digging properly into the YAML docs for that one.

Okay, I am sorry I led you astray with the single `\`. That is what I get for using SO and not digging properly into the YAML docs for that one. <!-- github-comment-id:1479938764 -->
adam commented 2026-10-05 02:46:44 +02:00
Author
Owner
Copy link
Copy source

No worries, I learned something from it as well. I might just keep the setting as is right now, in case anyone else might ever need it.

No worries, I learned something from it as well. I might just keep the setting as is right now, in case anyone else might ever need it. <!-- github-comment-id:1480052379 -->
adam commented 2026-10-05 02:46:45 +02:00
Author
Owner
Copy link
Copy source

Is there a feature in the beta I should be testing, or are you still in development?

Is there a feature in the beta I should be testing, or are you still in development? <!-- github-comment-id:1481375897 -->
adam commented 2026-10-05 02:46:45 +02:00
Author
Owner
Copy link
Copy source

All is fine, I keep it in progress, as I still need to document it.

All is fine, I keep it in progress, as I still need to document it. <!-- github-comment-id:1481681973 -->
Sign in to join this conversation.
No Branch/Tag Specified
Labels
No items
No labels
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
adam (Adam Wojcieszonek)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: adam/vscode-front-matter#426