Will the successor model to GPT-4 be able to correctly translate the following sentences into Dhivehi and back, while preserving the original meaning?
1) Please lock the door after 6PM, thanks
2) I am feeling really bad, I think I need to go to the hospital
3) Two hours of Thai massage is a bit too much
4) Is this granola gluten free?
Currently, GPT-4 fails if the English->Dhivehi translation and the Dhivehi -> English translation are carried out in separate chats.
The "On Dangers of Stochastic Parrots..." paper was raising concerns about this back in 2021.
If no GPT-5 or equivalent model is released by Dec. 31 2026, I will resolve NA.
1,000People are also trading
A round-trip test does not suffice for this market's goal. I would be surprised if it failed it in any language, assuming it's the same model. Because this task will effectively be: embed English, write something in X language that matches the embedding. Then read from X language to an embedding,then from English back. All that tests is that the same English phrase has a consistent embedding, which will usually be true.
At minimum, one should check a different model.
@brubsby I also replicated this and found it to be incorrect, but let's let the creator get a chance at resolving. If they still haven't resolved in a month or two, ping again!
