Please enable JavaScript.

Coggle requires JavaScript to display documents.

Modern AI - Coggle Diagram

- - - - Linear regression -> MSE
      - Classification --> cross-entropy cost
    - - Gradient descent
        
        Compute weights W rto minimize costs
        
        Learning rate
        
        Learning rate decay
        
        Standard gradient descent:
        
        Mini-batch stochastic gradient descent
        
        Escapes local minima
        
        Optimal batch size is a hypter parameter we have to decide
        
        If too large; then might have technical difficulties like keeping all the data in memeory
        
        Iff too small; the gradient is fitted on small proportion of the data; then traning can take a long time; unused memory
        
        Local v minimum
      - Epochs
        
        1 data pass = i epoch
        
        Overfitting; too many epochs
        
        Underfitting; too few epochs
  - - - Weight initialization
        
        Truncated Normal
- - - - Binary input times weight add a bias activate
      - Parameters constitute of a neurons bias and weights
- - - - CNN
        
        Filter/Kernel
        
        Padding
        
        Adds white space with zeros around image to make sure the size of the output is a certain number
        
        Pooling
        
        Wants to shrink the feature size
        
        Max-pooling: take the largest number in the window filter
        
        reduces complexity
        
        Avoids overfitting
- - - - Clustering
      - Natural language processing
  - - - When an artificial netowrk is involved in learning actions to take when presented with a given state from the environments in order to have a high probability of obtaining a positive reward
- - - - unstructured
      - dirty
      - normalized aka lowercased
      - stemmed
      - removed stop words
      - Goal is to take a set of documents each of which is a relatively free-form sequence of words, and trurn it into a familiar feature-vector form each document is an instance
      - bag of words
        
        every document is a collection of individual words
        
        one hot encoding
      - normalized term frequency
        
        TFIDF
      - topic modeling
      - one-hot-encoding
      - word2vec and word embedding
        
        contextual representation
        
        skim-gram
        
        cbow
  - - - sequence to sequence modeling
- - - - recommends items based on preferences of similar users
        
        neighborhood-based
        
        uses similarities between used or items, recommend items for users for items based on similarity
        
        user-based: what do users similar to you like?
        
        item-based: what
        
        latent factor
        
        each user has underlying tastes, each item has underlying appeal for these different tastes, tastes and appeals are callefd latent factors
        
        uses factors to be considdered as consumers tastes
    - - each item is defined by content c ... think of content as a ferature vector
- - - - 1980 Fukushima proposes an alagous architecture for machine vision called Neocognition
        
        simple, complex cells
        
        Arranging artificial neurons on a hierarchical manner generally represent lin orientations in the cells of the layers closts to the raw visual image, while successive deep layers represent complex, abstract objects
        
        LeNet-5 build upon Hubel, Wiesel, and Hukoshima which retains the hierarchical architecture in the primary visual cortex. The simple edges represent the first layer while successive layers represent complex features. Like correcly classifying handwritten digits.
        
        The feature-free ideaology was perceived to have limited bredth of applicability and traditional ML like feature engineering appeared to hold more promise shifting funding away from deep learning
        
        2001 Viola and Jones detect faces reliably with ml
        
        2012 AlexNet Alex Krizhevsky and Ilya Sutskever at university of toronto submitted to the ILSVRC (which holds millions of images for classification) crushed traditinal ml models; hiearchical atchitecture is reminnicant of LeNet-5 with an increase in visual feature complexity but had more more training data and processing power
        
        Dramatically decreases the need for subject matter expertise; one no longer nees a specialist in visual attributes to create a face-recognition algo; one no longer need to know the structure of language in different semantics
  - - - Deep learning
        
        Layering simple algorithms call artificial neurons into networks several layers deep
        
        Resides in representation learning which include any techniques that learn features from data automatically
      - Natural language processing
        
        Sits at the intersection of linguistics ai, and cs and includes classifying documents, machine translation, search engines, speech recognition, and chatbots, and spell checkers with translation, automated question-answering, and chatbots being the most sophistocated
        
        Computational representations of language
        
        one-hot-encoding
        
        Cells are binary that capture information about word location
        
        Ignores word similiarity
        
        word2vec
        
        Captures information about word meaning and similarity as well as location; this additional information renders word2vec favorable. It is analagous to the visual features learned automatically by deep learning machine vision models. In other words, linguistic features are learned automatically; vector spaced embeddings improves model performance and accuracy
        
        Inherent bias in language
        
        N-dimensional space allows for infinit combinations to find vectors of words related to one another
        
        Language
        
        Morphoogy: the form of words
        
        Phonemes and morphemes combine to form words which are combined to form syntax
        
        Phonology: the way language sounds when spoken
        
        Semantics: is concerned with the meaning of sentences which is the most abstract element of natural language like words and phrases and inferring humor and sarcasm and context and cultural norms
        
        Google Duplex has mastered the natural flow of human conversation; Tacotron maps sequences of words to corresponding sequences of audio features, which capture subtleties of human speech such as pitch, speed, intonation, and even pronunciation