Non-ASCII characters in testsets and report JSON files

[PierreMesure]

Issue Type

Bug

Source

source

Giskard Library Version

2.15.1

OS Platform and Distribution

Ubuntu 22.04

Python version

3.12

Installed python packages

No response

Current Behaviour?

Right now, the testsets are generated with ASCII characters in the texts. This is not optimal with content in Swedish, for instance.

Standalone code OR list down the steps to reproduce the issue

Generate a testset in Swedish and/or a report.

Relevant log output

No response

[PierreMesure]

My suggestion is to add force_ascii=False in these two calls to Pandas to_json() function: testset, report. I can send a PR if you agree.

[henchaves]

Hey @PierreMesure, thanks for reporting this issue.
Yes, of course, you can open a PR and request my review

[PierreMesure]

I did it, let me know if you want a test as well. It might take more time.

[henchaves]

Thanks @PierreMesure!
Yes, please. Could you add a simple test, for example saving and loading a testset in Swedish, to make sure that the content of the loaded one is similar to the original one?
Thanks a lot!

[PierreMesure]

I'll try! I also noticed that non ASCII-characters end up in other JSON files when generated with json.dumps so I'll fix it as well.

[henchaves]

Thanks a lot @PierreMesure

[PierreMesure]

Done now. I only added ensure_ascii=False in report.py so you might want to add it in other places where JSON files are saved. But I didn't want to create too much disruption.

I also added a test inspired by another one I found in the same test file but I couldn't get Pytest to work so I don't know if it works. 🤔

EDIT: The test checks that exporting and reimporting a testset doesn't change its content but I don't think that was the case before. The problem I'm trying to solve is the files being hard to read (when opened outside Giskard) because of Unicode characters.