Microsoft claims MT system “achieved human parity” on test data

This discussion belongs to Translation news » "Microsoft claims MT system “achieved human parity” on test data".
You can see the translation news page and participate in this discussion from there.

Michael Marcoux
Michael Marcoux  Identity Verified
United States
Local time: 12:48
Russian to English
+ ...
Don't speak Chinese, but... Mar 21, 2018

it actually looks pretty good. Welp, time to find a new career.

 
Lincoln Hui
Lincoln Hui  Identity Verified
Hong Kong
Local time: 00:48
Member
Chinese to English
+ ...
Strengths and weaknesses Mar 21, 2018

I tested a few strings and the system does have its strengths. The key weaknesses are as follows:

- It doesn't "know" that what it produces is correct. It's capable of producing several options (in this case, two), one of which is likely to be quite accurate, but it can't tell on its own which one to use.

- It looks like it's designed to parse text written in a specific way, i.e. by trained journalists. News reports have certain style conventions and it's not hard to im
... See more
I tested a few strings and the system does have its strengths. The key weaknesses are as follows:

- It doesn't "know" that what it produces is correct. It's capable of producing several options (in this case, two), one of which is likely to be quite accurate, but it can't tell on its own which one to use.

- It looks like it's designed to parse text written in a specific way, i.e. by trained journalists. News reports have certain style conventions and it's not hard to imagine that a machine with access to a vast amount of data can identify and pick up relevant information. Evidence to this is that the machine performed extremely poorly when I threw a couple of interview quotes at it. They were not in a style that the machine was trained in, and the confused machine was not even able to try and fake a fluent output. I also tried several legal text excerpts, relatively simple ones (terms of service) that pose no significant difficulty for a well-educated reader without formal legal training. The machine was able to produce a more-or-less acceptable output for the more verbose sample, but the other two threw it off completely.

The second weakness can be mitigated with more data, but the first has no solution in the foreseeable future. Accumulating more human selection data can increase the confidence of the choice, but only to a certain extent. It can't deliver total confidence without a human reviewer, and at that point you have a competent translator in the loop.

Translators are cheap, translation mistakes are expensive. The translation market could transform and it could shrink, but it has a long way to go before it becomes irrelevant.
Collapse


 


To report site rules violations or get help, contact a site moderator:

Moderator(s) of this forum
Jared Tabor[Call to this topic]

You can also contact site staff by submitting a support request »

Microsoft claims MT system “achieved human parity” on test data







CafeTran Espresso
You've never met a CAT tool this clever!

Translate faster & easier, using a sophisticated CAT tool built by a translator / developer. Accept jobs from clients who use Trados, MemoQ, Wordfast & major CAT tools. Download and start using CafeTran Espresso -- for free

Buy now! »
Trados Studio 2022 Freelance
The leading translation software used by over 270,000 translators.

Designed with your feedback in mind, Trados Studio 2022 delivers an unrivalled, powerful desktop and cloud solution, empowering you to work in the most efficient and cost-effective way.

More info »