Can GDScript JSON.parse handle non-BMP Unicode?

sarrowsmith

I suppose the answer is "no" because JSON doesn't handle non-BMP characters. But there is an established convention of using surrogate pairs, which JSON.parse doesn't appear to know about. Am I missing something? Is this a feature which could, or should, be implemented?

(Previously asked on the Q&A site, but unanswered.)

TwistedTwigleg

From what I can tell, Godot has limited support for non-BMP Unicode characters, but I have no idea how that support extends to JSON. I know that there are plans to improve the Unicode support in Godot with Godot 4, so maybe it will be included then? I wasn't able to find anything JSON specific though. If there is a use case for it and you think it would be a good addition, I would suggest opening a proposal on the Godot Proposal repository.

I don't know enough about non-BMP unicode to say whether the feature could/should be implemented, though if Godot's other text-based stuff (labels, GDScript, etc) support non-BMP unicode, then I think it is reasonable to have JSON support it too.

UnknownUser

@TwistedTwigleg said: If there is a use case for it and you think it would be a good addition, I would suggest opening a proposal on the Godot Proposal repository.

Hm, I think I'd have to come up with a slightly stronger use case than I currently have to do that (configuration mapping text to emoji, like this, the way :) gets turned into a smiley here).

I don't know enough about non-BMP unicode to say whether the feature could/should be implemented, though if Godot's other text-based stuff (labels, GDScript, etc) support non-BMP unicode, then I think it is reasonable to have JSON support it too.

GDScript correctly handles non-BMP UTF-8 characters read in from a File, and can display it in labels given a suitable font. The limitations are (a) no way of representing them in a string literal (ie an equivalent of Python's \UXXXXXXXX escape) and (b) not parsing them from JSON escape sequences representing a surrogate pair (as generated by Python's json module).

TwistedTwigleg

@sarrowsmith said: Victim of the Great Data Loss here: https://godotforums.org/discussion/comment/41718 was one of mine (I've only just come back to Godot after a couple of months away). @TwistedTwigleg (as you're the other participant :) ) can you do the merge thing?

Done! :smile: If you want, I can move the old opening post to the current opening post and delete OP by UnknownUser, just let me know.

sarrowsmith

@TwistedTwigleg said: If you want, I can move the old opening post to the current opening post and delete OP by UnknownUser, just let me know.

Yes please!

TwistedTwigleg

@sarrowsmith said:

@TwistedTwigleg said: If you want, I can move the old opening post to the current opening post and delete OP by UnknownUser, just let me know.

Yes please!

Done :smile: