Please enable JavaScript.

Coggle requires JavaScript to display documents.

HDFS - Hadoop Distributed File System - Coggle Diagram

- - - - High fault tolerance > resolves hardware unreliability problems
      - Hight ThroughPut > supports applications involved with a large amount of data
      - Large file Storage > supports TB < PB level data storage
      - dosent support random write and low delay read
    - - HDFS
      - HBase
- - - - used to store and generate Metadata of the file System, also know the DataNodes on Which are the blocks for a given file are located > HDFS has only one NameNode instance
      - functions of the standby NameNode
        
        Hot spare for the active NameNode
        
        The standby NameNode helps the active NameNode merge edit logs, reducing the startup time of the active NameNode.
    - - used to store actual data and periodically report back to the nameNode with data blocks under its management > also prefom ops > created , delete..
        Blocks on a DataNode must be different.
        Blocks stored on different DataNodes can be the same.
      - Rack is is group of DataNodes
      - Cluster is Group of NameNodes and DataNodes
    - - Uploading files from client
        
        The client divides files into multiple blocks and writes data blocks
        to each DataNode based on the DataNode address sequence.
        BlockSize > 128 MB
    - - Files are divided into blocks (1B>128Mb) of a certain size and each block is replicated 3 times throughout the DataNodes
        HDFS stores the first replica on the nearest node.
        HDFS is ideal for Write Once Read Many (WORM) tasks
        
        why ?? >> Bicuz lets say one of our dataNodes is down which was containing several block of data in it > soo we are underReplicated we only have 2 replications
        NameNode realize that and make new replication for those blocks
- - - - HBase > T1
      - Hive > T1 T3
        
        Bicuz Hive does the collection of Data it can take Data T1 or T3
      - Flume > T3
      - Spark T2