Please enable JavaScript.

Coggle requires JavaScript to display documents.

AWS Cert ML Specialty, Machine Learning Cycle, Generate Example Data,…

- - - - binary encoding of types
        
        not good for too many categories
    - - i.e. name, evil or not, affiliation
    - - order does not matter
    - - order does matter
  - - - Normalization
        
        0-1
      - Standardization
        
        0 avg Z-score
      - Quantile Binning
    - - MNIST data
    - - amplitude
      - time
    - - split by whitespace
    - - bigrams
  - - - Supervised learning
        
        prediccts values based on values of other features
      - Mean
        
        Avg value
      - Median
        
        middle value
      - Mode
        
        Most common value
      - Dropping rows
        
        removing missing values
  - - - CSV
      - JSON
      - Parquet
    - - recordIO-protobuf(creates tensor)
  - - - unsupervised learning
  - - - ETL
      - Apache Zeppelin
      - Jupyter
      - python or scala
      - output to S3
      - Most recommended
    - - Hadoop Eco-System
      - pySpark,Hive
    - - SQL on S3
    - - process between compute services
        
        custom processing
        
        RDS, EMR, Dynamo, RedShift
      - templates for pipelines
- - - - same as input data
        
        same as training/testing data
      - structured
        
        databases
        
        schema
        
        data warehouse
      - unstructured
        
        data lakes
      - semi-structured
        
        CSV,JSON,XML
      - text data (Corpus Data)
      - image data
        
        MNIST data
        
        hand writing
        
        image net
        
        classified/tagged
      - time series
        
        stock prices
      - Types
        
        Physical
        
        i.e. earth science, systems
        
        Biological
        
        Multi-variable
        
        csv,text, stockmarket data
    - - trusted data
      - Amazon SageMaker Ground Truth
        
        create labeled data
  - - - has attribute
      - ie spam not spam
      - tag associated
      - supervised training
    - - no attribute
      - audio streams
      - log files
      - unsupervised training
      - ie tweets
    - - qualitative
      - values associated with group
    - - quantitative
      - numerical value
  - - - go to place for machine learning data
    - - can use analytics tools
  - - - to S3
      - scheduled
      - on demand
      - assembly required
    - - manages resources for you
      - relational Db source
    - - full ETL service
    - - Hadoop cluster with all the fixins
    - - serverless SQL against S3
- - - - two values
    - - three values
- - - - for data consumers
      - real time analytics
      - storing data is optional
      - data retention
      - log file processing
      - shards
      - streaming to custom apps/other AWS services
      - massively scalable
    - - no shards
      - etl with lambda
      - to redshift or S3, a deliver service
      - S3 bucket events to Lambda
      - direct to S3 or other
      - use cases
        
        stream and store data into data lake
        
        Create ETL jobs on stream
    - - read and process in real-time
      - process from firehose
      - realtime complex SQL queries
      - store off to S3 after processing
      - metrics, dashboards, monitoring, notifications and alarms
      - responsive real-time analytics
      - Stream ETL jobs
      - think Apama event stream processing
    - - container holds data
      - unique partition keys
      - sequence of data records
        
        seq number
        
        partition key
        
        data blob
        
        transient TTL (24Hr to 7days)
    - - EC2
      - EMR
      - Lambda
      - Kinesis Data Analytics
    - - ingesting data
      - Java only
      - asynchronous send
      - slower
      - retry logic
    - - for consumer
    - - Any AWS SDKs
      - synchronous PutRecords
      - faster
      - low level API
- - - - train model
        
        evaluate model
        
        deploy to prod
        
        monitor&evaluate #