Prerequisites for Deep Learning

1.Knowledge of calculus and linear algebra, Math 53/54 or equivalent.

2.Probability and Statistics, CS70 or Stat 134. CS70 is bare minimum preparation, a stat course is better.

3.Machine Learning, CS189.

4.Programming, CS61B or equivalent. Assignments will mostly use Python.

Deep Learning

A machine learning subfield of learning representations of data. Exceptional effective at learning patterns. Deep learning algorithms attempt to learn (multiple levels of) representation by using a hierarchy of multiple layers. If you provide the system tons of information, it begins to understand it and respond in useful ways.

deep_learning_1

Before going into deep learning, We will take a look at Types of Learning

1.Supervised: Learning with a labeled training set

Example: email classification with already labeled emails

2.Unsupervised: Discover patterns in unlabeled data

Example: cluster similar documents based on text

3.Reinforcement learning: learn to act based on feedback/reward

Example: learn to play Go, reward: win or lose

deep_learning_2

Anomaly Detection

Sequence labeling

Machine Learning  Verses  Deep Learning

Most machine learning methods work well because of human-designed representations and input features, ML becomes just optimizing weights to best make a final predictiondeep_learning_8

Why Deep Learning  is useful?

1.Manually designed features are often over-specified, incomplete and take a long time to design and validate

2.Learned Features are easy to adapt, fast to learn

3.Deep learning provides a very flexible, (almost?) universal, learnable framework for representing world, visual and linguistic information.

4.Can learn both unsupervised and supervised

5.Effective end-to-end joint system learning

6.Utilize large amounts of training data

In ~2010 DL started outperforming other ML techniques first in speech and vision, then NLPdeep_learning_3

State of Art in NLP

Several big improvements in recent years in NLP (Natural Language Processing)

* Machine Translation
* Sentiment Analysis
* Dialogue Agents
* Question Answering
* Text Classification

Leverage different levels of representation

* words & characters
* syntax & semantics
deep_learning_4

deep_learning_5

Neural Network Intro

deep_learning_6

Application Example: Relation Extraction from text

deep_learning_7

Useful for:

  • knowledge base completion
  • social media analysis
  • question answering

References

  • http://web.stanford.edu/class/cs224n
  • https://www.coursera.org/specializations/deep-learning
  • https://chrisalbon.com/#Deep-Learning
  • http://www.asimovinstitute.org/neural-network-zoo
  • https://www.google.co.in