Evaluation metric using sentence representation
YiSi is a metric for automatic evaluation of machine translation that calculates the similarity between a machine translation output and a reference translation using sentence representation.
The term YiSi comes from the Cantonese word 意思, which means ‘meaning’.
YiSi is a unified semantic MT quality evaluation and estimation metric for languages with different levels of available resources.
YiSi offers different variants depending on the resources available for the evaluation.
|Can be deployed to any language pair||Requires an embedding model||Requires a crosslingual embedding model|
|Measures lexical similarity via longest common character substring||Aggregates the similarity in meaning of the embeddings||Optionally requires a semantic role labeler in both input and output language|
|Compares machine translation output and human reference||Can incorporate shallow semantic structures||Evaluates the crosslingual semantic similarity using bilingual embeddings|
|Can also estimate quality without any reference|