Deep Learning — Paper List

Generative Model

  1. Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A. and Bengio, Y., 2014. Generative adversarial nets. In Advances in neural information processing systems
  2. Kingma, D.P. and Welling, M., 2013. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114.
  3. Bengio, Y., Yao, L., Alain, G. and Vincent, P., 2013. Generalized denoising auto-encoders as generative models. In Advances in Neural Information Processing Systems (pp. 899-907).
  4. Vincent, P., Larochelle, H., Bengio, Y. and Manzagol, P.A., 2008, July. Extracting and composing robust features with denoising autoencoders. In Proceedings of the 25th international conference on Machine learning (pp. 1096-1103). ACM.

Generative Adversarial Networks

  1. Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A. and Bengio, Y. Generative adversarial nets, NIPS (2014).
  2. Goodfellow, Ian NIPS 2016 Tutorial: Generative Adversarial Networks, NIPS (2016).
  3. Radford, A., Metz, L. and Chintala, S., Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv preprint arXiv:1511.06434. (2015)
  4. Salimans, T., Goodfellow, I., Zaremba, W., Cheung, V., Radford, A., & Chen, X. Improved techniques for training gans. NIPS (2016).
  5. Chen, X., Duan, Y., Houthooft, R., Schulman, J., Sutskever, I., & Abbeel, P. InfoGAN: Interpretable Representation Learning by Information Maximization Generative Adversarial Nets, NIPS (2016).
  6. Zhao, Junbo, Michael Mathieu, and Yann LeCun. Energy-based generative adversarial network. arXiv preprint arXiv:1609.03126 (2016).
  7. Mirza, Mehdi, and Simon Osindero. Conditional generative adversarial nets. arXiv preprint arXiv:1411.1784 (2014).
  8. Isola, P., Zhu, J. Y., Zhou, T., & Efros, A. A. Image-to-image translation with conditional adversarial networks. arXiv preprint arXiv:1611.07004. (2016).
  9. Reed, S., Akata, Z., Yan, X., Logeswaran, L., Schiele, B., & Lee, H. Generative adversarial text to image synthesis. JMLR (2016).
  10. Antipov, G., Baccouche, M., & Dugelay, J. L. Face Aging With Conditional Generative Adversarial Networks. arXiv preprint arXiv:1702.01983. (2017).
  11. Liu, Ming-Yu, and Oncel Tuzel. Coupled generative adversarial networks. NIPS (2016).
  12. Denton, E.L., Chintala, S. and Fergus, R., 2015. Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks. NIPS (2015).
  13. Dumoulin, V., Belghazi, I., Poole, B., Lamb, A., Arjovsky, M., Mastropietro, O., & Courville, A. Adversarially learned inference. arXiv preprint arXiv:1606.00704 (2016).

Variational Autoencoders

  1. D. Kingma, M. Welling, Auto-Encoding Variational Bayes, ICLR, 2014
  2. Carl Doersch, Tutorial on Variational Autoencoders arXiv, 2016
  3. Xinchen Yan, Jimei Yang, Kihyuk Sohn, Honglak Lee, Attribute2Image: Conditional Image Generation from Visual Attributes, ECCV, 2016
  4. Jacob Walker, Carl Doersch, Abhinav Gupta, Martial Hebert, An Uncertain Future: Forecasting from Static Images using Variational Autoencoders, ECCV, 2016
  5. Anh Nguyen, Jason Yosinski, Yoshua Bengio, Alexey Dosovitskiy, Jeff Clune, Plug & Play Generative Networks: Conditional Iterative Generation of Images in Latent Space, arXiv, 2016
  6. Alireza Makhzani, Jonathon Shlens, Navdeep Jaitly, Ian Goodfellow, Brendan Frey, Adversarial Autoencoders, ICLR, 2016
  7. Anders Boesen Lindbo Larsen, Søren Kaae Sønderby, Hugo Larochelle, Ole Winther, Autoencoding beyond pixels using a learned similarity metric, ICML, 2016
  8. Aditya Deshpande, Jiajun Lu, Mao-Chuang Yeh, David Forsyth, Learning Diverse Image Colorization, arXiv, 2016
  9. Jiajun Lu, Aditya Deshpande, David Forsyth, CDVAE: Co-embedding Deep Variational Auto Encoder for Conditional Variational Generation, arXiv, 2016
  10. Diederik P. Kingma, Danilo J. Rezende, Shakir Mohamed, Max Welling, Semi-Supervised Learning with Deep Generative Models, NIPS, 2014
  11. Lars Maaløe, Casper Kaae Sønderby, Søren Kaae Sønderby, Ole Winther, Auxiliary Deep Generative Models arXiv, 2016
  12. Raymond Yeh, Ziwei Liu, Dan B Goldman, Aseem Agarwala, Semantic Facial Expression Editing using Autoencoded Flow arXiv, 2016

Model Compression

  1. Denton, Emily L., et al. “Exploiting linear structure within convolutional networks for efficient evaluation.” Advances in Neural Information Processing Systems. 2014.
  2. Jin, Jonghoon, Aysegul Dundar, and Eugenio Culurciello. “Flattened convolutional neural networks for feedforward acceleration.” arXiv preprint arXiv:1412.5474 (2014).
  3. Gong, Yunchao, et al. “Compressing deep convolutional networks using vector quantization.” arXiv preprint arXiv:1412.6115 (2014).
  4. Han, Song, et al. “Learning both weights and connections for efficient neural network.” Advances in Neural Information Processing Systems. 2015.
  5. Guo, Yiwen, Anbang Yao, and Yurong Chen. “Dynamic Network Surgery for Efficient DNNs.” Advances In Neural Information Processing Systems. 2016.
  6. Gupta, Suyog, et al. “Deep Learning with Limited Numerical Precision.” ICML. 2015.
  7. Courbariaux, Matthieu, Yoshua Bengio, and Jean-Pierre David. “Binaryconnect: Training deep neural networks with binary weights during propagations.” Advances in Neural Information Processing Systems. 2015.
  8. Courbariaux, Matthieu, et al. “Binarized neural networks: Training deep neural networks with weights and activations constrained to+ 1 or-1.” arXiv preprint arXiv:1602.02830 (2016).
  9. Han, Song, Huizi Mao, and William J. Dally. “Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding.” arXiv preprint arXiv:1510.00149 (2015).
  10. Iandola, Forrest N., et al. “SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and< 0.5 MB model size.” arXiv preprint arXiv:1602.07360 (2016).

RNN

  1. R. Pascanu, T. Mikolov, and Y. Bengio, On the difficulty of training recurrent neural networks, ICML 2013
  2. S. Hochreiter, and J. Schmidhuber, J., Long short-term memory, Neural computation, 1997 9(8), pp.1735-1780
  3. F.A. Gers, and J. Schmidhuber, J., Recurrent nets that time and count, IJCNN 2000
  4. K. Greff , R.K. Srivastava, J. Koutník, B.R. Steunebrink, and J. Schmidhuber, LSTM: A search space odyssey, IEEE transactions on neural networks and learning systems, 2016
  5. K. Cho, B. Van Merrienboer, C. Gulcehre, D. Bahdanau, F. Bougares, H. Schwenk, and Y. Bengio, Learning phrase representations using RNN encoder-decoder for statistical machine translation, ACL 2014
  6. R. Jozefowicz, W. Zaremba, and I. Sutskever, An empirical exploration of recurrent network architectures, JMLR 2015

Recurrent Architectures: LSTM, GRU, RNN

  1. Survey Papers
  2. Training
  3. Architectural Complexity Measures
  4. RNN Variants
  5. Visualization

Advanced CNN Architectures

  1. K. He, X. Zhang, S. Ren, and J. Sun, Deep Residual Learning for Image Recognition, CVPR 2016
  2. Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun, Identity Mappings in Deep Residual Networks, ECCV 2016
  3. Gao Huang, Zhuang Liu, Kilian Q. Weinberger, Laurens van der Maaten: Densely Connected Convolutional Networks
  4. Andreas Veit, Michael Wilber, Serge Belongie, Residual Networks Behave Like Ensembles of Relatively Shallow Networks, NIPS 2016
  5. Klaus Greff, Rupesh K. Srivastava & Jürgen Schmidhuber, Highway and Residual Networks Learn Unrolled Iterative Estimation

Advanced Training Techniques

  1. D. Kingma, and J. Ba, Adam: a method for stochastic optimization, ICLR 2015
  2. J. Dean et al., Large scale distributed deep networks, NIPS 2012
  3. N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov, Dropout: a simple way to prevent neural networks from overfitting, JMLR 2014
  4. S. Ioffe and C. Szegedy, Batch normalization: accelerating deep network training by reducing internal covariate shift, ICML 2015
  5. K. He, X. Zhang, S. Ren, and J. Sun, Delving deep into rectifiers: surpassing human-level performance on ImageNet classification, ICCV 2015

Object Detection

  1. Girshick, Ross and Donahue, Jeff and Darrell, Trevor and Malik, Jitendra, Rich feature hierarchies for accurate object detection and semantic segmentation, CVPR 2014
  2. He, Kaiming and Zhang, Xiangyu and Ren, Shaoqing and Sun, Jian, Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition, ECCV 2014
  3. Jifeng Dai, Yi Li, Kaiming He, Jian Sun R-FCN: Object Detection via Region-based Fully Convolutional Networks , NIPS 2016
  4. Girshick, Ross, Fast R-CNN, ICCV 2015
  5. Ren, Shaoqing and He, Kaiming and Girshick, Ross and Sun, Jian, Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks, CVPR 2015
  6. Erhan, Dumitru and Szegedy, Christian and Toshev, Alexander and Anguelov, Dragomir, Scalable Object Detection using Deep Neural Networks, CVPR 2014
  7. Bell, Sean and Lawrence Zitnick, C and Bala, Kavita and Girshick, Ross, Inside-Outside Net: Detecting Objects in Context with Skip Pooling and Recurrent Neural Networks, CVPR 2016
  8. Redmon, Joseph and Divvala, Santosh and Girshick, Ross and Farhadi, Ali, You Only Look Once: Unified, Real-Time Object Detection, CVPR 2016
  9. Liu, Wei and Anguelov, Dragomir and Erhan, Dumitru and Szegedy, Christian and Reed, Scott and Fu, Cheng-Yang and Berg, Alexander C, SSD: Single Shot MultiBox Detector, ECCV 2016
  10. Tsung-Yi Lin, Piotr Dollár, Ross Girshick, Kaiming He, Bharath Hariharan, Serge Belongie, Feature Pyramid Networks for Object Detection, arXiv 2016
  11. Huang, Jonathan and Rathod, Vivek and Sun, Chen and Zhu, Menglong and Korattikara, Anoop and Fathi, Alireza and Fischer, Ian and Wojna, Zbigniew and Song, Yang and Guadarrama, Sergio and others, Speed/accuracy trade-offs for modern convolutional object detectors, arXiv 2016

Adversarial Samples

  1. Matthew D. Zeiler and Rob Fergus, Visualizing and Understanding Convolutional Networks, ECCV 2014
  2. Karen Simonyan, Andrea Vedaldi, Andrew Zisserman, Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps, arXiv:1312.6034v2
  3. Alexey Dosovitskiy and Thomas Brox, Inverting Visual Representations with Convolutional Networks, CVPR 2016
  4. Anh Nguyen, Jason Yosinski, and Jeff Clune, Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Images, CVPR 2015
  5. Christian Szegedy, et al., Intriguing properties of neural networks, arXiv preprint arXiv:1312.6199v4
  6. Seyed-Mohsen Moosavi-Dezfooli, et al, Universal adversarial perturbations, arXiv preprint arXiv:1610.08401v2
  7. Ian J. Goodfellow, et al, Explaining and Harnessing Adversarial Examples, arXiv preprint arXiv:1412.6572
  8. A. Kurakin et al., Adversarial examples in the physical world, ICLR 2017
  9. D. Krotov and J. Hopfield, Dense Associative Memory is Robust to Adversarial Inputs, arXiv preprint arXiv:1701.00939