Recasting of Deep Neural Network

Sritha Zith Dey Babu, Er. Sandeep Kour

Volume 4, Issue 2 2020

Page: 1-10

Abstract

The rapid increase of information and accessibility in recent years has activated a paradigm shift in algorithm design for artificial intelligence. Recently, Deep Learning (a surrogate of Machine Learning) has won several contests in pattern recognition and machine learning. This review comprehensively summarizes relevant studies, much of it from prior state-ofthe-art techniques. This paper also discusses the motivations and principles regarding learning algorithms for deep architectures.

Back Download



References

  • D. Li and D. Yu, 'Deep Learning: Methods and Applications,' Foundations and Trends in Signal Processing, Now Publishers, 2014.
  • Y. Bengio, 'Learning deep architectures for AI,' Foundations and trends® in Machine Learning, vol. 2, pp. 1-127, 2009.
  • Y. Bengio, A. Courville, and P. Vincent, 'Representation learning: A review and new perspectives,' Pattern Analysis and Machine Intelligence, IEEE Transactions on, vol. 35, pp. 1798-1828, 2013.
  • J. Schmidhuber, 'Deep learning in neural networks: An overview,' Neural Networks, vol. 61, pp. 85-117, 2015.
  • Y. LeCun, Y. Bengio, and G. Hinton, 'Deep learning,' Nature, vol. 521, pp. 436-444, 2015.
  • I. Arel, D. C. Rose, and T. P. Karnowski, 'Deep machine learning-a new frontier in artificial intelligence research [research frontier],' Computational Intelligence Magazine, IEEE, vol. 5, pp. 13-18, 2010
  • P. O. Glauner, 'Deep Convolutional Neural Networks for Smile Recognition,' arXiv preprint arXiv:1508.06535, 2015.
  • H. A. Song and S.-Y. Lee, 'Hierarchical Representation Using NMF,' in Neural Information Processing, 2013, pp. 466-473.
  • M. Xiong, J. Chen, Z. Wang, C. Liang, Q. Zheng, Z. Han, et al., 'Deep Feature Representation via Multiple Stack Auto-Encoders,' in Advances in Multimedia Information Processing--PCM 2015, end: Springer, 2015, pp. 275-284.
  • B. A. Olshausen, 'Emergence of simple-cell receptive field properties by learning a sparse code for natural images,' Nature, vol. 381, pp. 607-609, 1996.
  • R. Collobert, 'Deep learning for efficient discriminative parsing,' in International Conference on Artificial Intelligence and Statistics, 2011.
  • L. Gomes, 'Machine-learning maestro michael jordan on the delusions of big data and other huge engineering efforts,' IEEE Spectrum, Oct, vol. 20, 2014.
  • G. E. Hinton, S. Osindero, and Y.-W. Teh, 'A fast learning algorithm for deep belief nets,' Neural computation, vol. 18, pp. 1527-1554, 2006.
  • R. Salakhutdinov and G. Hinton, 'An efficient learning procedure for deep Boltzmann machines,' Neural computation, vol. 24, pp. 1967-2006, 2012.
  • D. H. Staelin and C. H. Staelin, 'Models for Neural Spike Computation and Cognition,' ed: CreateSpace.[Links], 2011.
  • N. Kriegeskorte, 'Deep Neural Networks: A New Framework for Modeling Biological Vision and Brain Information Processing,' Annual Review of Vision Science, vol. 1, pp. 417-446, 2015.
  • S. Bengio, L. Deng, H. Larochelle, H. Lee, and R. Salakhutdinov, 'Guest Editors' Introduction: Special Section on Learning Deep Architectures,' Pattern Analysis and Machine Intelligence, IEEE Transactions on, vol. 35, pp. 1795- 1797, 2013.
  • A. Krizhevsky, I. Sutskever, and G. E. Hinton, 'Imagenet classification with deep convolutional neural networks,' in Advances in neural information processing systems, 2012, pp/ 1097-1105
  • E. Rublee, V. Rabaud, K. Konolige, and G. Bradski, 'ORB: an efficient alternative to SIFT or SURF,' in Computer Vision (ICCV), 2011 IEEE International Conference on, 2011, pp. 2564-2571.
  • B. C. Csáji, 'Approximation with artificial neural networks,' Faculty of Sciences, Etvs Lornd University, Hungary, vol. 24, 2001.
  • G. Cybenko, 'Approximation by superpositions of a sigmoidal function,' Mathematics of control, signals and systems, vol. 2, pp. 303-314, 1989.
  • K. Horne, 'Approximation capabilities of multilayer feedforward networks,' Neural networks, vol. 4, pp. 251-257, 1991.
  • M. H. Hassoun, Fundamentals of artificial neural networks: MIT press, 1995.
  • K. P. Murphy, Machine learning: a probabilistic perspective: MIT press, 2012.
  • G. E. Hinton, N. Srivastava, A. Krizhevsky, I. Sutskever, and R. R. Salakhutdinov, 'Improving neural networks by preventing co-adaptation of feature detectors,' arXiv preprint arXiv:1207.0580, 2012.
  • Y. Bengio, 'Deep learning of representations: Looking forward,' in Statistical language and speech processing, ed: Springer, 2013, pp. 1-37.
  • A. Testolin, I. Stoianov, M. De Filippo De Grazia, and M. Zorzi, 'Deep unsupervised learning on a desktop PC: a primer for cognitive scientists,' Front. Psychol, vol. 4, p. 10.3389, 2013.
  • A. Weapons, 'an Open Letter from AI & Robotics Researchers, 2015,' ed.
  • C. Szegedy, A. Toshev, and D. Erhan, 'Deep neural networks for object detection,' in Advances in Neural Information Processing Systems, 2013, pp. 2553-2561.
  • S. Kombrink, T. Mikolov, M. Karafiát, and L. Burget, 'Recurrent Neural Network Based Language Modeling in Meeting Recognition,' in INTERSPEECH, 2011, pp. 2877-2880.
  • Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, 'Gradient-based learning applied to document recognition,' Proceedings of the IEEE, vol. 86, pp. 2278-2324, 1998.
  • T. N. Sainath, A.-r. Mohamed, B. Kingsbury, and B. Ramabhadran, 'Deep convolutional neural networks for LVCSR,' in Acoustics, Speech and Signal Processing (ICASSP), 2013 IEEE International Conference on, 2013, pp. 8614-8618.
  • G. Hinton, L. Deng, D. Yu, G. E. Dahl, A.-r. Mohamed, N. Jaitly, et al., 'Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups,' Signal Processing Magazine, IEEE, vol. 29, pp. 82-97, 2012.
  • Y. Bengio, N. Boulanger-Lewandowski, and R. Pascanu, 'Advances in optimizing recurrent networks,' in Acoustics, Speech and Signal Processing (ICASSP), 2013 IEEE International Conference on, 2013, pp. 8624-8628.
  • G. E. Dahl, T. N. Sainath, and G. E. Hinton, 'Improving deep neural networks for LVCSR using rectified linear units and dropout,' in Acoustics, Speech and Signal Processing (ICASSP), 2013 IEEE International Conference on, 2013, pp. 8609-8613.
  • G. Hinton, 'A practical guide to training restricted Boltzmann machines,' Momentum, vol. 9, p. 926, 2010.
  • G.-B. Huang, Q.-Y. Zhu, and C.-K. Siew, 'Extreme learning machine: theory and applications,' Neurocomputing, vol. 70, pp. 489-501, 2006.
  • B. Widrow, A. Greenblatt, Y. Kim, and D. Park, 'The no-prop algorithm: A new learning algorithm for multilayer neural networks,' Neural Networks, vol. 37, pp. 182-188, 2013.
  • Y. Ollivier and G. Charpiat, 'Training recurrent networks online without backtracking,' arXiv preprint arXiv:1507.07680, 2015.
  • I. Aleksander, M. De Gregorio, F. M. G. França, P. M. V. Lima, and H. Morton, 'A brief introduction to Weightless Neural Systems,' in ESANN, 2009.
  • G. E. Hinton, 'Deep belief networks,' Scholarpedia, vol. 4, p. 5947, 2009.
  • H. Larochelle, D. Erhan, A. Courville, J. Bergstra, and Y. Bengio, 'An empirical evaluation of deep architectures on problems with many factors of variation,' in Proceedings of the 24th international conference on Machine learning, 2007, pp. 473-480.
  • G. E. Hinton, 'Training products of experts by minimizing contrastive divergence,' Neural computation, vol. 14, pp. 1771-1800, 2002.
  • A. Fischer and C. Igel, 'Training restricted Boltzmann machines: an introduction,' Pattern Recognition, vol. 47, pp. 25-39, 2014.
  • G. Barnich, H. A. Gonzalez, A. Maloney, and B. Oblak, 'One loop partition function of three-dimensional flat gravity,' Journal of High Energy Physics, vol. 2015, pp. 1-8, 2015.
  • T. Sercu, C. Puhrsch, B. Kingsbury, and Y. LeCun, 'Very deep multilingual convolutional neural networks for LVCSR,' arXiv preprint arXiv:1509.08967, 2015.
  • C. Yan, F. Coenen, and B. Zhang, 'Driving posture recognition by convolutional neural networks,' IET Computer Vision, 2015.
  • C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, et al., 'Going deeper with convolutions,' arXiv preprint arXiv:1409.4842, 2014.
  • A. Krizhevsky and G. Hinton, 'Convolutional deep belief networks on cifar-10,' Unpublished manuscript, 2010.
  • H. Lee, R. Grosse, R. Ranganath, and A. Y. Ng, 'Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations,' in Proceedings of the 26th Annual International Conference on Machine Learning, 2009, pp. 609-616.
  • Y. LeCun, 'The MNIST database of handwritten digits.{Online},' ed, 2007.
  • B. Kwolek, 'Face detection using convolutional neural networks and Gabor filters,' in Artificial Neural Networks: Biological Inspirations–ICANN 2005, ed: Springer, 2005, pp. 551-556.
  • M. Osadchy, Y. L. Cun, and M. L. Miller, 'Synergistic face detection and pose estimation with energy-based models,' The Journal of Machine Learning Research, vol. 8, pp. 1197-1215, 2007.
  • S. Sukittanon, A. C. Surendran, J. C. Platt, and C. J. Burges, 'Convolutional networks for speech detection,' in Interspeech, 2004.
  • F. J. Huang and Y. LeCun, 'Large-scale learning with svm and convolutional for generic object categorization,' in Computer Vision and Pattern Recognition, 2006 IEEE Computer Society Conference on, 2006, pp. 284-291.
  • H. Lee, P. Pham, Y. Largman, and A. Y. Ng, 'Unsupervised feature learning for audio classification using convolutional deep belief networks,' in Advances in neural information processing systems, 2009, pp. 1096-1104.
  • T. Khorshed, 'Research Problem Definition Part.'
  • I. Lenz, H. Lee, and A. Saxena, 'Deep learning for detecting robotic grasps,' The International Journal of Robotics Research, vol. 34, pp. 705-724, 2015.
  • K. Kavukcuoglu, M. A. Ranzato, R. Fergus, and Y. Le-Cun, 'Learning invariant features through topographic filter maps,' in Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE Conference on, 2009, pp. 1605-1612.
  • B. B. Benuwa, B. Ghansah, D. K. Wornyo, and S. A. Adabunu, 'A Comprehensive Review of Particle Swarm Optimization,' in International Journal of Engineering Research in Africa, 2016, pp. 141-161.
  • J. Weston, F. Ratle, H. Mobahi, and R. Collobert, 'Deep learning via semi-supervised embedding,' in Neural Networks: Tricks of the Trade, ed: Springer, 2012, pp. 639-655.
  • K. A. De Jong, 'Evolving intelligent agents: A 50 year quest,' IEEE Computational Intelligence Magazine, vol. 1, pp. 12-17, 2008.
  • M. M. Islam and X. Yao, 'Evolving artificial neural network ensembles,' in Computational intelligence: a compendium, ed: Springer, 2008, pp. 851-880.
  • S. V. Rice, F. R. Jenkins, and T. A. Nartker, The fifth annual test of OCR accuracy: Information Science Research Institute, 1996.
  • E. M. Newton and P. J. Phillips, 'Meta-analysis of third-party evaluations of iris recognition,' Systems, Man and Cybernetics, Part A: Systems and Humans, IEEE Transactions on, vol. 39, pp. 4-11, 2009.
  • A. Adler and M. E. Schuckers, 'Comparing human and automatic face recognition performance,' Systems, Man, and Cybernetics, Part B: Cybernetics, IEEE Transactions on, vol. 37, pp. 1248-1255, 2007.

Looking for Paper Publication??

Come to us.

Submit your Paper