YARN Architecture
The Apache Hadoop YARN (Yet Another Resource Negotiator) is a cluster management technology. It is one of the key features in the second-generation Hadoop 2.0 version. The Apache Software Foundation's open source distributed process framework The idea of YARN is to split resource management and job scheduling/monitoring into separate. The idea is to have a global Resource Manager (RM) and per-application Application Master (AM). An application is either a single job or a DAG of jobs.
Resource Manager
The Resource Manager and the Node Manager from the data computation framework. The Resource Manager arbitrates resources among all the applications in the system. The Node Manager framework handles containers, monitoring. Their resource usage (CPU, memory, disk, network) and reporting the same to the Resource Manager/Scheduler.
Application Master
The per-application Application Master is the effect. Resource Manager and working with the Node Manager(s) to execute and check the tasks. The Resource Manager has two main components: Scheduler and Applications Manager.
Scheduler
The Scheduler handles various running applications subject to familiar constraints of capacities, queues etc. That it performs no monitoring or tracking of status for the application. No guarantees about restarting failed tasks either due to application failure or hardware failures. The Scheduler performs its scheduling function based on the resource requirements of the applications. It does so based on the abstract notion of a resource Container. This incorporates elements such as memory, CPU, disk, network etc. The Scheduler handles partitioning the cluster resources among the various queues, applications etc. The current schedulers such as the capacity scheduler and fair scheduler.
Applications Manager
The Applications Manager handles accepting job-submissions. The first container for executing the application specific Application Master. It provides the service for restarting the Application Master container on failure. The per-application Application Master has the responsibility of resource containers. Monitoring for progress the Scheduler, tracking their status. Map Reduce in Hadoop-2.x maintains API compatibility with the previous stable release (Hadoop-1.x). This means that all Map Reduce jobs should still run unchanged on top of YARN with just a recompile.
Hadoop Training Institute in Hyderabad
Kosmik Provides Hadoop training in Hyderabad. We are providing lab facilities with complete real-time training. Training is based on complete advance concepts. So that you can get easily "hands-on experience". We will give 100% job assistance.
