Please enable JavaScript.

Coggle requires JavaScript to display documents.

Hive - Distributed NoSql Database - Coggle Diagram

- - - - Flexible ETL (exact / transform / load)
      - Supporting computing engines, such as Mapreduce, Tez, and Spark
      - Direct Access to HDFS files and HBase
      - Easy to use and program
  - - - Daily weekly click count
        Traffic statistics
    - - Log analysis
        Text analysis
    - - Interest analysis
        user behaviour
        partition demonstration
    - - Data extraction
        Data loading
        Data transformation
  - - - Beeline
        JDBC
        Thrift
        Python
        ODBC
- - - - MetaStore
        
        provides metadata services and depends on DBService
      - WebHCat
        
        Provides HTTP services, such as metadata, Data Defined Language DDL for external users
        provides Rest interface for users to make the following operations through safe HTTPS
        
        architecture of WebHCat
        
        Hive DDL operations
        
        Running Hive HQL task
        
        Running MapReduce task
      - HiveServer
        
        recv requests from clients , parses and excutes HQL commands and return query results.
        even if one HiveServer is unavailable > the Hive Cluster is not entirely unavailable
        At the same time, one HiveSever is in the active state, and the other is in the standby state.
  - - - Partition > data table can be divided into partitions by using a field value (each partition is dir and can be partitioned and bucketed)
      - Bucket > data can be stored in diffrent buckets (each bucket is a file in which data is stored by the hash value if a field )
    - - managed tables
        
        managed tables are created by default by Hive , in which Hive migrates data to data wareHouses Directories
        Used when Hive preforms all operations
      - External tables
        
        when extTables created > Hive access data from locations outside data warehouse directories
        Used when Hive and other tools share the same data set for different processing
        The external keyword must be specified when an external table is created
        When an ext table is deleted, the metadata and data in the table are deleted.
      - Common tables > created by default
        When a common table is deleted, the metadata and data in the table are deleted.
  - - - Storing associated data or data on which associated operations are preformed on the same storage node
        5-Level collocation allows quick file access which avoids network consumption caused by data migration
      - Using Colocation
        
        Use HDFS interfaces to create groupid and locatorid
        
        Use the Hive Colocation function
    - - when creating a table, specify the columns to be encrypte dand the encryption algorithm
      - use an insert syntsx to import data to tables whose columns are encrypted
    - - Hive allows delection of a single record from an HBase.
        Hive can use specific syntax to delete one or more data records that meet criteria from HBase tables.
      - usage >
        HQL statement > remove table HBase_table where expression
    - - Set inputFormat When creating a table.
      - Specify the delimiter before a query