| 引用本文: | 梁华参,赵铁军,薛永增,孙加东.词汇相似度约束的短语抽取[J].哈尔滨工业大学学报,2010,42(5):775.DOI:10.11918/j.issn.0367-6234.2010.05.023 |
| LIANG Hua-shen,ZHAO Tie-jun,XUE Yong-zeng,SUN Jia-dong.Phrase extraction based on constraints of word similarities[J].Journal of Harbin Institute of Technology,2010,42(5):775.DOI:10.11918/j.issn.0367-6234.2010.05.023 |
|
| 摘要: |
| 为克服传统的短语抽取方法对词对齐信息的依赖性强,抗噪声能力差这一缺陷,提出基于词汇相似度约束的短语抽取策略;在此框架下,提出了3种基于词汇相似度的约束方法:Dice系数、Phi平方系数和对数似然比.在IWSLT2004语料上进行的实验表明,3种基于词汇相似度的约束方法的翻译系统的BLEU评分均优于传统的翻译系统;其中基于对数似然比方法得到的翻译模型比基线系统Pharaoh的BLEU-4评分提高了15.14%. |
| 关键词: 机器翻译 统计机器翻译 短语抽取 词汇相似度 |
| DOI：10.11918/j.issn.0367-6234.2010.05.023 |
| 分类号:TP391.2 |
| 基金项目:国家自然科学基金重点资助项目(60736014);国家高技术研究发展计划重点资助项目(2006AA010208) |
|
| Phrase extraction based on constraints of word similarities |
|
LIANG Hua-shen1, ZHAO Tie-jun1, XUE Yong-zeng2, SUN Jia-dong1
|
|
1.MOE-MS Key Lab of Natural Language Processing and Speech,Harbin Institute of Technology,Harbin 150001,China;2.Dept.of New Media and Art,Harbin Institute of Technology,Harbin 150001,China)
|
| Abstract: |
| Aimed at the problem that the traditional phrase extraction method is strictly dependent on word alignments,and is not pruned to alignment errors,a loose phrase extraction method,which does not strictly depend on word alignments.In this method,constraints are posed on alignment points to avoid ill-formed phrase pairs.Three constraint strategies are proposed based on word similarities:Dice coefficient,Phi-square coefficient and log-likelihood ratio.Experiments were carried out on the corpus of IWSLT 2004.Results show that the BLEU scores of the best results of loose phrase extraction can be improved by 15.14%,compared with the baseline system Pharaoh. |
| Key words: machine translation statistical machine translation phrase extraction word similarity |
