Deep Learning is an interesting new branch of machine learning where neural networks consisting of multiple layers have shown new generalization capabilities. The seminar will look at advances in both general deep learning approaches, and at the specific case of Neural Machine Translation (NMT). NMT is a new paradigm in data-driven machine translation. In Neural Machine Translation, the entire translation process is posed as an end-to-end supervised classification problem, where the training data is pairs of sentences and the full sequence to sequence task is handled in one model.
Here is a link to last semester's seminar.
There is a Munich interest group for Deep Learning, which has an associated mailing list (initially organized by David Kaumanns), the paper announcements are sent out on this list. See the link here.
Email Address: SubstituteLastName@cis.uni-muenchen.de
Thursdays 14:45 (s.t.), location is room 161 (until 18.04.19, then after that room 131)
Click here for directions to CIS.
New attendees are welcome. Read the paper and bring a paper or electronic copy with you, you will need to refer to it during the discussion.
If this page appears to be out of date, use the refresh button of your browser
| Date | Paper | Links | Discussion Leader |
| Thursday, October 11th (in room C007) | Guillaume Lample, Myle Ott, Alexis Conneau, Ludovic Denoyer, Marc'Aurelio Ranzato (2018). Phrase-Based & Neural Unsupervised Machine Translation. EMNLP 2018 (best paper award). We might also look briefly at: Mikel Artetxe, Gorka Labaka, Eneko Agirre (2018). Unsupervised Statistical Machine Translation. EMNLP 2018. | lample artetxe | Alex Fraser |
| Thursday, October 18th | Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova (2018). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. arXiv. | paper | Timo Schick |
| Thursday, October 25th | Viktor Hangya, Fabienne Braune, Yuliya Kalasouskaya, Alexander Fraser (2018). Unsupervised Parallel Sentence Extraction from Comparable Corpora. In Proceedings of the 15th International Workshop on Spoken Language Translation (IWSLT) | No paper (talk rehearsal) | Viktor Hangya |
| Thursday, November 15th | Trip Report EMNLP (send one slide to Mengjie if you would like to present) | Mengjie Zhao | |
| Thursday, November 22nd | Mia Xu Chen, Orhan Firat, et al. The Best of Both Worlds: Combining Recent Advances in Neural Machine Translation (2018). ACL 2018 | arxiv version | Helmut Schmid |
| Thursday, November 29th | Konstantinos Bousmalis, George Trigeorgis, Nathan Silberman, Dilip Krishnan, Dumitru Erhan (2016). Domain Separation Networks. NIPS 2016 | paper | Nina Pörner |
| Thursday, December 6th | Colin Cherry, George Foster, Ankur Bapna, Orhan Firat, Wolfgang Macherey (2018). Revisiting Character-Based Neural Machine Translation with Capacity and Compression. EMNLP 2018 | arxiv version | Alex Fraser |
| Thursday, December 13th | NeurIPS Review (send one slide to Nora about one paper if you would like to present) | Nora Kassner | |
| Thursday, December 20th | Sergey Edunov, Myle Ott, Michael Auli, David Grangier (2018). Understanding Back-Translation at Scale. EMNLP 2018 | arxiv version | Matthias Huck |
| Thursday, January 10th | Gabriela Csurka (2017). Domain Adaptation for Visual Applications: A Comprehensive Survey, pages 1 to 26. Book chapter to appear in "Domain Adaptation in Computer Vision Applications" | paper | Philipp Dufter |
| Thursday, January 24th | Ofir Press and Noah Smith (2018). You May Not Need Attention. arXiv. | paper | Masoud Jalili Sabet |
| Thursday, January 31st | Guillaume Lample, Alexis Conneau (2019). Cross-lingual Language Model Pretraining. arXiv. | paper | Dario Stojanovski |
| Thursday, February 7th | Emily Dinan, Stephen Roller, Kurt Shuster, Angela Fan, Michael Auli, Jason Weston (2018). Wizard of Wikipedia: Knowledge-Powered Conversational agents. arXiv. | paper | Alena Moiseeva |
| Thursday, February 14th | Mikel Artetxe, Holger Schwenk (2018). Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and Beyond. arXiv. | paper | Viktor Hangya |
| Thursday, February 21st | Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever (2019). Language Models are Unsupervised Multitask Learners. Preprint | paper | Dietrich Trautmann |
| Thursday, March 7th | Kevin Clark, Minh-Thang Luong, Christopher D. Manning, Quoc V. Le. Semi-Supervised Sequence Modeling with Cross-View Training. EMNLP 2018 | paper | Helmut Schmid |
| Thursday, March 14th | Qipeng Guo, Xipeng Qiu, Pengfei Liu, Yunfan Shao, Xiangyang Xue, Zheng Zhang (2019). Star-Transformer. NAACL 2019 | paper | Dietrich Trautmann |
| Thursday, March 28th | Zichao Yang, Zhiting Hu, Chris Dyer, Eric P. Xing, Taylor Berg-Kirkpatrick (2018). Unsupervised Text Style Transfer using Language Models as Discriminators. NeurIPS 2018 | paper | Simon Rieß |
Further literature:
Please click here for an (old) NMT reading list, but also see the more general RNN reading list here (scroll down). You can also go back through the previous semesters by clicking on the link near the top of the page.