The standard was developed with the aim of maintaining “a consistent standard against which the translation quality of automotive service information can be objectively measured:
The method is based on a points-based system. The less points a translation scores, the higher the quality of the translation.
Errors are divided into 7 categories and each category is outlined comprehensively:
Each category is weighted. Some error categories are considered to affect final quality more than others. For instance, a spelling error may score fewer points than errors found in the Wrong Term category.
Once an error is identified as belonging to a certain category, the human reviewer decides whether the error is either serious or minor. A serious error is weighted to score more points than a minor error.
The metric is both easy to follow and easy to implement. It is an excellent step towards creating an objective linguistic quality measurement. Furthermore, it is highly customisable; if you feel that in a particular application a spelling error can be more damaging to a translated version than the use of an incorrect term, you can easily change the weighting.
Finally, the results of the metrics can be used for benchmarking linguistic standards and serve as a basis for discussion with both clients and translation groups.
If your organisation is planning to use J2450, please remember that:
The metric is published by SAE and specifically with the automotive service information industry in mind. Consequently, it may not be the most suitable method for evaluating translations for which style and/or tone are of paramount importance.
Results for the reviews have to be collated manually and comparisons can be difficult if you are working directly from spreadsheets.
Guidance is needed for reviewers using the metric. It needs to be made very clear what constitutes a serious or a minor error. Evaluators need to be trained to ensure clear and common understanding.
Once the points have been allocated, there is no benchmark to determine what constitutes a good or bad mark. We carried out a test of the metric at Pangeanic by submitting translations of the same text into 4 different languages to our in-house and external proofreaders. After evaluation, we found that the language with the second least points was not approved by the proofreader as publishable. However, proofreaders for languages that had scored more points were generally happy with the translation.
The J2450 Translation Quality Metric can be purchased through the SAE internet site.
Flat8, 279 Church Road,
+44 203 5400 256
Av. Cortes Valencianas, 26-5,
46015 Valencia (Spain)
(+34) 96 333 63 33
(+34) 91 326 29 33
One Boston Place
Boston MA 02108
228 E 45TH St Rm 9E
10017-3337 New York, NY
21st Floor, CMA Building
64 Connaught Road Central
Toll Free: +852 2157 3950
Ogawa Building 3F
3-37 Kanda Sakuma-cho
Tomson Commercial Building,
710 Dong Fang Road
Pu Dong, Shanghai 200122, China