Machine Translation

Machine Translation Quality Estimation

Machine Translation Quality Estimation (MTQE) is an AI-powered feature that provides segment-level quality estimations for machine translation (MT) suggestions. It is similar to the quality estimations for translation memory (TM) matches and non-translatables (NT).

The instant MT quality scores help guide post-editing and can be used to enhance the default (pre-translation) analysis and to assess MT engine quality.

MTQE is available as a standalone feature for any MT engine supported in Memsource or through Memsource Translate.

Using MTQE

When enabled for an MT engine, MTQE is available in projects with this MT engine selected. If using Memsource Translate, MTQE is available in projects with Memsource Translate as the selected MT engine type.

MTQE does not support all language combinations. See Supported Language Pairs.

Analyses

With MTQE enabled, Default Analysis includes MT scores alongside the TM and NT scores. This can be turned off in Analysis options.

Pre-translation

In addition to the instant, segment-level, quality matches in the Memsource Editor, MTQE is used in pre-translation. This can be turned off in Pre-translate options.

See Pre-translate for more information.

Configure MTQE

MTQE can be enabled for specific MT engines supported by Memsource. It is also automatically available with any of the engines supported by the Memsource Translate solution.

To enable MTQE for a specific MT engine, follow these steps:

  1. From the Setup Setup_gear.png page, scroll down to the Integrations section.

  2. Click on Machine Translation Engines.

  3. Select the applicable MT engine and click Edit.

    The Edit Machine Translation Engine page opens.

  4. Select the MT Quality Estimation option.

  5. Click Save.

    A checkmark appears under Quality Estimation in the MT settings page.

Evaluating MTQE Results

Once MTQE has been enabled and employed as part of the Machine Translation process, MTQE scores for content and engine can be measured for accuracy. Direct comparison between segment-level MTQE scores and post-editing analysis is not available, but the following options provide ways to quantitatively and qualitatively evaluate MTQE scores.

Evaluating with Post-editing Analysis

Post-editing analysis indicates editing effort; how much text the Linguist or Proofreader had to edit. For post-editing analysis in projects with Machine Translation and MTQE, results are calculated as the difference between the machine translation suggestions and the final text after post-editing is finished.

In order to evaluate the results of the post-editing analysis, run a Default Analysis before the first step of the workflow to see how MT matches are categorized into MTQE bands.

When post-editing is complete, run a Post-editing Analysis with the Analyze MT option.

If the machine translation or non-translatable suggestion was accepted without any editing, the results will indicate 100%.

If the machine translation has been changed, the match rate is lower and the more the segment is changed, the lower the score will be. This is the same score-counting algorithm as the one used to calculate the score of translation memory fuzzy matches.

If the default analysis indicates a high number of quality MT matches (75% or above), the post-editing analysis reflects the correspondingly minimal to moderate amount of editing to the MT suggestions.

Evaluating the Segment Changes

To evaluate the substance of the changes made during post-editing, create a workflow that generates a report showing the changes on a segment-level.

To create this workflow, follow these steps:

  1. Create a project with two Workflow steps (e.g. pre-translation and post-editing).

  2. In the first Workflow step, pre-translate the job with only MT. This provides a snapshot of the matches to be used.

  3. In the second Workflow step, let the linguist post-edit normally.

  4. Once the workflow is completed, run the post-editing analysis to see the edit distance between the two steps (the number of changes).

  5. Select the relevant jobs, then go to Tools and select Export Workflow Changes.

    The different versions of the segments are presented.

Buy MTQE Characters

MTQE requires purchased characters. Segments estimated by MTQE (via Analysis, the CAT panel in the Memsource Editor, or pre-translation) consume these characters. Characters purchased for Memsource Translate are automatically available for MTQE. 

MTQE characters for third-party MT engines are vendor specific; for example, MTQE characters available for DeepL can only be used for the DeepL MT engine and characters available for MTQE CrossLang would only be usable by the CrossLang engine.

Purchase Characters

To purchase characters, follow these steps:

  1. From the Setup Setup_gear.png page, scroll down to the Integrations section.

  2. Click on Machine Translation Engines.

    The Machine Translation Engine page opens.

  3. In the Remaining Characters column, click Top Up.

    The character purchase page opens.

  4. Select currency, payment method, bundle and location options.

  5. Click Continue.

    The billing data page opens.

  6. Provide billing data and click Continue.

    The review page opens.

  7. Ensure billing data is correct and click Continue.

    Payment gateway opens.

  8. Complete payment process to finish purchase of characters.

    Purchased characters will be added to the Remaining Characters column showing the number of characters available for use with MTQE.

MTQE Supported Language Pairs

Source

Target

cs

en

cs

hu

cs

sk

cs

sl

da

de

da

en

da

no

da

sv

de

cs

de

en

de

es

de

fr

de

it

de

nl

de

sk

de

sv

el

en

en

ar

en

bg

en

ds

en

dy

en

da

en

de

en

el

en

es

en

et

en

fi

en

fr

en

he

en

hr

en

hu

en

id

en

it

en

ja

en

ko

en

lt

en

lv

en

nl

en

no

en

pl

en

pt

en

ro

en

ru

en

sk

en

sl

en

sr

en

sv

en

th

en

tr

en

uk

en

vi

en

zh

es

en

es

fr

es

pt

fri

en

fr

sv

fr

de

fr

en

fr

it

fr

nl

it

en

ja

en

ja

ko

ja

zh

ko

en

ko

ja

ko

zh

lt

lv

lt

ru

lv

en

nl

de

nl

en

nl

fr

no

en

no

sv

pl

en

pt

en

pt

es

ru

en

sk

en

sl

hr

sr

en

sv

da

sv

de

sv

en

sv

fi

sv

no

tr

en

zh

en

zh

ja

Was this article helpful?

Comments

0 comments

Article is closed for comments.