Deep Learning is an interesting new branch of machine learning where neural networks consisting of multiple layers have shown new generalization capabilities. The seminar will look at advances in both general deep learning approaches, and at the specific case of Neural Machine Translation (NMT). NMT is a new paradigm in data-driven machine translation. In Neural Machine Translation, the entire translation process is posed as an end-to-end supervised classification problem, where the training data is pairs of sentences and the full sequence to sequence task is handled in one model.
Here is a link to last semester's seminar.
There is a Munich interest group for Deep Learning, which has an associated mailing list, the paper announcements are sent out on this list. See the link here.
Email Address: SubstituteLastName@cis.uni-muenchen.de
Thursdays 14:45 (s.t.), location is room 115
Click here for directions to CIS.
New attendees are welcome. Read the paper and bring a paper or electronic copy with you, you will need to refer to it during the discussion.
If this page appears to be out of date, use the refresh button of your browser
| Date | Paper | Links | Discussion Leader |
| Thursday, October 17th | Gonzalo Mena (2017). The Gumbel-Softmax Trick for Inference of Discrete Variables. Blog (Columbia). | blogpost | Nina Pörner |
| Thursday, October 24th *at 2:15pm* | BRIEF TALK AND DISCUSSION: NLP in Sri Lanka, Prof Gihan Dias (Moratuwa) and K. Sarveswaran (Konstanz) | ||
| Thursday, October 31st | Sachin Kumar, Yulia Tsvetkov (2019). Von Mises-Fisher Loss for Training Sequence to Sequence Models with Continuous Outputs. ICLR 2019 | paper | Benjamin Roth |
| Thursday, November 14th | R. Thomas McCoy, Ellie Pavlick, Tal Linzen (2019). Right for the wrong reasons: Diagnosing syntactic heuristics in natural language inference. ACL 2019 Also look at: Timothy Niven and Hung-Yu Kao (2019). Probing Neural Network Comprehension of Natural Language Arguments. ACL 2019 | mccoy niven | Alex Fraser |
| Thursday, November 21st | Urvashi Khandelwal, Omer Levy, Dan Jurafsky, Luke Zettlemoyer, Mike Lewis (2019). Generalization Through Memorization: Nearest Neighbor Language Models. arXiv 2019. | paper | Helmut Schmid |
| Thursday, November 28th | Anonymous. You CAN Teach an Old Dog New Tricks! On Training Knowledge Graph Embeddings. ICLR 2020 submission. | openreview | Marina Speranskaya |
| Thursday, December 12th | Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J. Liu (201). Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer. arXiv. | paper You can optionally skip 3.2 to 3.6 (or 3.1 to 3.6), not 3.7 | Kerem Şenel |
| Thursday, December 19th | Emma Strubell, Ananya Ganesh, Andrew McCallum (2019). Energy and Policy Considerations for Deep Learning in NLP. ACL 2019. | paper | Alex Fraser |
| Thursday, January 9th | Steven C.Y. Hung, Cheng-Hao Tu, Cheng-En Wu, Chien-Hung Chen, Yi-Ming Chan, Chu-Song Chen (2019). Compacting, Picking and Growing for Unforgetting Continual Learning. NeurIPS 2019. | paper | Mengjie Zhao |
| Thursday, January 16th | (Some) Research Happening at Google Translate, talk by Markus Freitag | ||
| Thursday, January 23rd | Nikita Kitaev, Łukasz Kaiser, Anselm Levskaya (2020). Reformer: The Efficient Transformer. ICLR 2020 | paper | Nina Pörner |
| Thursday, January 30th | Kaitao Song, Xu Tan, Tao Qin, Jianfeng Lu, Tie-Yan Liu (2019). MASS: Masked Sequence to Sequence Pre-training for Language Generation. ICML 2019 | paper | Alexandra Chronopoulou |
| Thursday, February 6th | Alexey Romanov, Maria De-Arteaga, Hanna Wallach, Jennifer Chayes, Christian Borgs, Alexandra Chouldechova, Sahin Geyik, Krishnaram Kenthapadi, Anna Rumshisky, Adam Tauman Kalai (2019). What's in a Name? Reducing Bias in Bios without Access to Protected Attributes. NAACL 2019 (best paper) | paper | Ben Roth |
| Thursday, February 13th | Eric Malmi, Sebastian Krause, Sascha Rothe, Daniil Mirylenka, Aliaksei Severyn (2019). Encode, Tag, Realize: High-Precision Text Editing. EMNLP 2019 | paper | Jindřich Libovický |
| Thursday, February 20th | Parker Riley, Daniel Gildea (2018, 2020). Orthographic Features for Bilingual Lexicon Induction. Unsupervised Bilingual Lexicon Induction Across Writing Systems. ACL and arXiv. | paper1 paper2 | Silvia Severini |
| Thursday, February 27th | Junxian He, Xinyi Wang, Graham Neubig, Taylor Berg-Kirkpatrick (2020). A Probabilistic Formulation of Unsupervised Text Style Transfer. ICLR 2020. | paper | Dario Stojanovski |
| Thursday, March 5th | Stephen Merity (2019). Single Headed Attention RNN: Stop Thinking With Your Head. arXiv. | paper | Martin Schmitt |
Further literature:
You can go back through the previous semesters by clicking on the link near the top of the page.