Please enable JavaScript.

Coggle requires JavaScript to display documents.

Chapter 7: Big Data Technologies, Hadoop V1 vs V2: - Coggle Diagram

- - - - Data storage and data processing file system
      - Function
        
        Stores and offers parallel, streaming access to massive data sets.
        
        Data is distributed across a cluster of nodes; large files can spread multiple nodes.
        
        Files are divided into blocks, typically 64MB in size, abstracting the underlying filesystem.
        
        Suited for storing large files and managing numerous files efficiently.
    - - Distributed data processing framework for large quantities of data, distibuted across cluster of nodes, in parallel
- - - - Managed by Hive framework
      - When delete, the table and metadata are deleted
    - - when delte only metadata deleted, dta is retained
      - The location referred to is external to the Hive framework but still resides within HDFS, as data for a Hive external table is stored in HDFS.
      - Hive only manage metadata, data stored in some external location
- - - - Coordinates HBase cluster activities.
    - - Manage table data and serve client requests.
    - - Manages DDL operations like table creation and deletion.
- - - - Acts as a passive data store where events received by the source are stored.
    - - Takes events from the channel and stores them externally, often in HDFS.
      - can also forward events to other Flume sources for processing by another Flume agent.
    - - Gathers events from external sources like log files or web servers.