Please enable JavaScript.

Coggle requires JavaScript to display documents.

MapReduce/YARN > Technical Principles of MapReduce and YARN - Coggle…

- - - - Easy to Program
      - Outstanding scalibilty
        
        Cluster capabilities can be improved by addinng notes
      - High fault tolerance
        
        Cluster Availiability and fault tolerance are improved by policies such as computing or data migration
  - - - everything has to do with ressource management and scheduling..
- - - - mapReduce determines the the number of partitions based on that of Reduce tasks. the records with the same key value are sent to the same Reduce tasks for processing
    - - before jobs are submitted < files to be processed are split
    - - MapReduce submits requests to RessourceManager > Ressource manager creates jobs > each App map to one job
    - - after map taks is processed many split files are generated . These split files must be combined into split files MOFMapOutFile
        that is partitioned and sorted
    - - Shuffle > data transfer process between the map phase and Reduce phase involves obtaining MOF files from the Map tasks and sorting and merging MOF files
- - - - Flexibility
      - Priority
        
        correct sequence of the resource allocation initiated by the YARN scheduler
        Local resources > Same rack > Any host
      - Multi Leasing
        
        multple users can share a cluster and multple apps can run concurrently.
        Adms can add multple restrictions to prevent cluster ressources from being exclusivly pccupied by an application user orr queue
      - Capacity assurance
        
        Adms can set upper and lower limits for the ressource usage of each queue all specifications submitted to the queue share the ressources
      - Dynamic update of configuration files
        
        Adms can dynamically modify configuration params to manage clusters online
  - - - User Limitaiton
      - Task Limitation