EMA 2008 - Statistical Machine Translation

9th Summer School of the European Masters in Language and Speech Technology

Dr. Alexander Fraser

Assignment 1 - Expectation Maximization and Model 1

Assignment 2 - Building a French or German to English Decoder

Lecture 1 - Introduction ppt2003

Lecture 2 - Word Alignment and Expectation Maximization ppt2007 ppt2003

Lecture 3 - Phrase-based SMT and Decoding ppt2007 ppt2003

Lecture 4 - Assignments and Advanced Topics ppt2007 ppt2003

My homepage

9th Summer School of the European Masters in Language and Speech Technology

Bibliography for SMT lectures

Measuring translation quality

Kishore A. Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. BLEU: a method for automatic evaluation of machine translation. Technical Report RC22176 (W0109-022), IBM Research Division, Thomas J. Watson Research Center, Yorktown Heights, NY, 2001.

Defines the BLEU metric and shows that it correlates well with human judgements

Chris Callison-Burch, Cameron Fordyce, Philipp Koehn, Christof Monz, and Josh Schroeder. (Meta-) evaluation of machine translation. In Proceedings of the ACL 2007 Second Workshop on Statistical Machine Translation, Prague, Czech Republic, 2007.

Compares automatic metrics

Measuring alignment quality

Alexander Fraser and Daniel Marcu. Measuring word alignment quality for statistical machine translation. Computational Linguistics, 33(3):293-303, 2007.

Shows that F-alpha predicts BLEU

Generative alignment models

Peter F. Brown, Stephen A. Della Pietra, Vincent J. Della Pietra, and R. L. Mercer. The mathematics of statistical machine translation: parameter estimation. Computational Linguistics, 19(2):263-311, 1993.

IBM models and basics of SMT

Kevin Knight. A statistical machine translation tutorial workbook, 1999.

Tutorial on basics of SMT, Model 1 and Model 3

Stephan Vogel, Hermann Ney, and Christoph Tillmann. HMM-based word alignment in statistical translation. In 16th International Conference on Computational Linguistics, pages 836-841, Copenhagen, Denmark, 1996.

HMM model (best model that can be trained using exact EM. See also several recent papers citing this paper)

Franz J. Och and Hermann Ney. A systematic comparison of various statistical alignment models. Computational Linguistics, 29(1):19-51, 2003.

Comparison of IBM models and HMM model

Discriminative word alignment models

Alexander Fraser and Daniel Marcu. Getting the structure right for word alignment: LEAF. In Conference on Empirical Methods in Natural Language Processing and Conference on Computational Natural Language Learning, pages 51-60, Prague, Czech Republic, 2007.

Presents LEAF and hybrid generative/discriminative model

Robert C. Moore, Wen-Tau Yih, and Andreas Bode. Improved discriminative bilingual word alignment. In Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics, pages 513-520, Sydney, Australia, 2006.

Pure discriminative model inspired from Model 4

Phrase-based modeling

Franz J. Och and Hermann Ney. The alignment template approach to statistical machine translation. Computational Linguistics, 30(1):417-449, 2004.

Alignment Templates (first phrase-based model)

Philipp Koehn, Franz J. Och, and Daniel Marcu. Statistical phrase-based translation. In Proceedings of the Human Language Technology and North American Association for Computational Linguistics Conference, pages 127-133, Edmonton, Canada, 2003.

Phrase-based SMT and log-linear formulation

Phrase-based decoding

Philipp Koehn. Pharaoh: A beam search decoder for phrase-based statistical machine translation models, user manual. 2004.

Pharaoh manual (Pharaoh was a precursor of Moses), contains extensive information on implementation of a decoder

Minimum Error Rate Training

Franz J. Och and Hermann Ney. Discriminative training and maximum entropy models for statistical machine translation. In Processings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 295-302, Philadelphia, PA, 2002.

Maximum Entropy training for SMT

Franz J. Och. Minimum error rate training in statistical machine translation. In Processings of the 41st Annual Meeting of the Association for Computational Linguistics, pages 160-167, Sapporo, Japan, 2003.

Minimum Error Rate training for SMT

Syntactic modeling

Michael Collins, Philipp Koehn, and Ivona Kucerová. Clause restructuring for statistical machine translation. In Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics, pages 531-540, Ann Arbor, MI, 2005.

Manually written rules for reordering German VP to have similar order to English VP

Kenji Yamada and Kevin Knight. A syntax-based statistical translation model. In Processings of the 39th Annual Meeting of the Association for Computational Linguistics, pages 523-530, Toulouse, France, 2001.

String-to-tree model, statistical system using parse trees

Michel Galley, Mark Hopkins, Kevin Knight, and Daniel Marcu. What's in a translation rule?. In Proceedings of the Human Language Technology and North American Association for Computational Linguistics Conference, pages 273-280, 2004.

String-to-tree, generalizes Yamada and Knight, see also papers citing this one

David Chiang. Hierarchical phrase-based translation. Computational Linguistics, 33(2):201-228, 2007.

Using formal grammars (without syntactic parses)

General text book

Philipp Koehn. Statistical Machine Translation. Cambridge University Press, To Appear, 2008.

SMT text book from which some of my slides were derived, will be out soon

Other

Watch www.statmt.org for shared tasks and participate, only need to follow steps in assignment 2 on all of the data
If you are in Stuttgart, participate in our reading group on Thursday mornings!