create_file method (and others) struggle with unicode and emoji handling (create, search andreplace)

[rmoriz]

This happened to me when dealing with markdown based README files that contain emojis. I've also issues with comments in TOML files that contain German umlauts (ÄÖÜäöüß). 

It usually ends in empty files. The agent is smart enough to re-create the file and retries it without emojis and other ways but it takes a lot of cycles.

╭─ Response ────────────────────────────────────────────────────────────────────────────────────────────────╮
│ Now let me create a new English version of the config file by creating it from scratch: │
╰───────────────────────────────────────────────────────────────────────────────────────────────────────────╯
└── Calling create_file: {"file_path": "config.example.toml", "overwrite": true, "initial_content": "# 
Buntspecht C...

``` 
Error executing tool create_file: 'charmap' codec can't encode character '\U0001f916' in position 1021: 
character... 
``` 

[rmoriz]

I've added a longer log in a gist at https://gist.github.com/rmoriz/cade21e1ca201310d96c904769b93386

[Jovana Dunisijevic]

Hey @rmoriz 

Thank you for highlighting this issue with Unicode and emoji handling.

Rovo Dev CLI currently struggles with Unicode and emoji handling in file operations due to encoding limitations.

The CLI team is aware that robust Unicode support is important, especially for international users and documentation with emojis.

However, as a temporary workaround, you can try setting your environment’s default encoding to UTF-8 before running the CLI. In your terminal, you can set:

export PYTHONUTF8=1

Or ensure your locale is set to use UTF-8 (e.g., export LANG=en_US.UTF-8).

The best way to help prioritise a fix is to submit this as feedback directly from the CLI using the /feedback command, including your error message and a sample file that triggers the issue.

Keep your CLI updated, as the team regularly releases bug fixes and improvements.

I hope this helps!

Kindest regards,

Jov