| 引用本文: | 林建方,牛成,李生,郑德权.Web数据反馈的搭配抽取方法[J].哈尔滨工业大学学报,2010,42(2):281.DOI:10.11918/j.issn.0367-6234.2010.02.023 |
| LIN Jian-fang,NIU Cheng,LI Sheng,ZHENG De-quan.Automatic collocation extraction using web feedback data[J].Journal of Harbin Institute of Technology,2010,42(2):281.DOI:10.11918/j.issn.0367-6234.2010.02.023 |
|
| 摘要: |
| 为了提高搭配(Collocation)抽取的精度,提出一种新的互联网数据的搭配抽取方法.传统的搭配抽取统计方法都是基于语料库的,常受到语料库规模的影响和制约,而在互联网数据中蕴含着丰富的知识和信息,基于Web的词汇相关性度量方法,充分利用搭配在谷歌中的页面数模拟其对应语料库的词频数,并分别选取共现频率、互信息、卡方检验3种经典统计关联度量方法.实验结果表明召回率、精确率均好于对应的基于语料库的方法,这说明互联网中大量数据应用于自然语言处理各种任务的可行性. |
| 关键词: 搭配 共现频率 互信息 卡方检验 语料库 Web |
| DOI：10.11918/j.issn.0367-6234.2010.02.023 |
| 分类号:TP391.1 |
| 基金项目:国家自然科学基金重点资助项目(60736044);国家科技发展计划探索类资助项目(2006AA01Z150) |
|
| Automatic collocation extraction using web feedback data |
|
LIN Jian-fang1, NIU Cheng2, LI Sheng1, ZHENG De-quan1
|
|
1.MOE-MS Key Laboratory of Natural Language Processing and Speech,Harbin Institute of Technology,Harbin 150001,China;2.Microsoft Research Asia,Beijing 100080,China)
|
| Abstract: |
| To improve the precison of collocation extraction,this paper proposes a new method based on Internet data.For the constraint by the corpus scale for traditional collocation extraction approach based on linguistic corpus,we acquire collocations from Web,which contains plenty of information and knowledge.Three classical association measures of co-occurrence frequency,mutual information and χ2-test are used to automatically extract the collocation.Based on the experimental results,the benchmarks show that the performance of this new Web-based approach is superior to that of traditional approach in both precision and recall.Thus the data from Internet may be applied in many NLP applications. |
| Key words: collocation co-occurrence frequency mutual information χ2-test corpora web |
