Translation: Humans vs Gemini Who Ends On Top
- Miguel Antunez Vega
- Reading time: 5 minutes
TLDR: We put a human translator up against Gemini to translate ad copy and sent the answers to an international client who acted as the judge. The results encompass a win for Gemini and massive opportunities for integration.
Last week we sat down with one of our international clients and pitched them a strange request. Due to their global reach, we’ve had to work quite often with translation copy.
Curiosity grew stronger as we wondered how much better are humans at translation over AI? Are they even better?
The opportunity to test this came when we were working on some ad copy. Apart from English, we needed the text to be in French and Russian. We thought why not? Mentioned the experiment to the client, who quickly got on board, and the Human vs Gemini battle began.
We tested a total of 94 ad copy texts, each of them translated from English to French and Russian by both a professional translator and gemini.
To keep things fair, we named each version of the text as A or B, whilst swapping the letters out sporadically to ensure the labelling was 100% random and could not affect the outcomes.
Voters could select 4 options, answering which ad copy sounded better:
Nothing short of interesting:
Why sort of? Well, if we place close attention back to the tables we can see that 17 of the 94 votes in French and 10 of the Russian votes were actually attributed to NEITHER. This gives a glimpse of the real way to optimise: the combination of human translation with Gemini, combining both strengths to try and capture the wide margin that is still left behind (18% and 11%).
At the end of the day, that is where we should be going, not towards a blind trust in Gemini but an integration of technological advancements with human expertise.
If you’re still relying on old-school translation or (even worse) blindly hitting “translate” on a bot, you’re leaving your global reputation to chance.
Has Gemini won? Yes, it mostly has, at the very least it has caught up.
Moving forward the focus needs to lie in using AI tools to help grow human efficiency, not replace, but most definitely adapt and evolve.
How The Battle Began
Last week we sat down with one of our international clients and pitched them a strange request. Due to their global reach, we’ve had to work quite often with translation copy.
Curiosity grew stronger as we wondered how much better are humans at translation over AI? Are they even better?
The opportunity to test this came when we were working on some ad copy. Apart from English, we needed the text to be in French and Russian. We thought why not? Mentioned the experiment to the client, who quickly got on board, and the Human vs Gemini battle began.
The Test
We tested a total of 94 ad copy texts, each of them translated from English to French and Russian by both a professional translator and gemini.
To keep things fair, we named each version of the text as A or B, whilst swapping the letters out sporadically to ensure the labelling was 100% random and could not affect the outcomes.
Voters could select 4 options, answering which ad copy sounded better:
- A (depending on the text could be either Human or Gemini)
- B (depending on the text could be either Human or Gemini)
- Both (meaning essentially it’s the same answer)
- Neither (the two options aren’t quite right)
The Results
Nothing short of interesting:
- In French (Table 1), we found that Gemini received a total of 67 votes (71%) whilst Humans only received 61 (65%).
- Likewise, in Russian (Table 2), we can see that Gemini received a total of 65 votes (69%) whilst Humans only got 53 (56%).
Why sort of? Well, if we place close attention back to the tables we can see that 17 of the 94 votes in French and 10 of the Russian votes were actually attributed to NEITHER. This gives a glimpse of the real way to optimise: the combination of human translation with Gemini, combining both strengths to try and capture the wide margin that is still left behind (18% and 11%).
Table 1 and 2: Test Results in French and Russian
Final Thought
At the end of the day, that is where we should be going, not towards a blind trust in Gemini but an integration of technological advancements with human expertise.
If you’re still relying on old-school translation or (even worse) blindly hitting “translate” on a bot, you’re leaving your global reputation to chance.
Has Gemini won? Yes, it mostly has, at the very least it has caught up.
Moving forward the focus needs to lie in using AI tools to help grow human efficiency, not replace, but most definitely adapt and evolve.
