【YARN】yarn 集羣管理之memory、cpu 相關配置

yarn 管理內存相關配置

properties	value	描述	description
yarn.nodemanager.resource.memory-mb	-1	可分配給containers的物理內存量（以MB爲單位）。 如果設置爲-1並且yarn.nodemanager.resource.detect-hardware-capabilities爲true，則將自動計算該值（在Windows和Linux中）。 在其他情況下，默認值爲8192MB。	Amount of physical memory, in MB, that can be allocated for containers. If set to -1 and yarn.nodemanager.resource.detect-hardware-capabilities is true, it is automatically calculated(in case of Windows and Linux). In other cases, the default is 8192MB.
yarn.nodemanager.resource.detect-hardware-capabilities	false	啓用對節點容量（例如內存和CPU）的自動檢測功能。	Enable auto-detection of node capabilities such as memory and CPU.
yarn.nodemanager.resource.system-reserved-memory-mb	-1	爲非YARN進程保留的物理內存量（以MB爲單位）。 僅當yarn.nodemanager.resource.detect-hardware-capabilities設置爲true且yarn.nodemanager.resource.memory-mb爲-1時，才使用此配置。 如果設置爲-1，則此值計算爲（系統內存-2 * HADOOP_HEAPSIZE）的20％	Amount of physical memory, in MB, that is reserved for non-YARN processes. This configuration is only used if yarn.nodemanager.resource.detect-hardware-capabilities is set to true and yarn.nodemanager.resource.memory-mb is -1. If set to -1, this amount is calculated as 20% of (system memory - 2*HADOOP_HEAPSIZE)
yarn.nodemanager.vmem-pmem-ratio	2.1	設置容器的內存限制時，虛擬內存與物理內存之間的比率。 容器分配以物理內存的形式表示，虛擬內存使用量可以超出此分配比例。	Ratio between virtual memory to physical memory when setting memory limits for containers. Container allocations are expressed in terms of physical memory, and virtual memory usage is allowed to exceed this allocation by this ratio
yarn.nodemanager.pmem-check-enabled	true	是否啓動一個線程監測每個任務正使用的物理內存量，如果任務超出分配值，則直接將其殺掉	Whether physical memory limits will be enforced for containers
yarn.nodemanager.vmem-check-enabled	true	是否啓動一個線程監測每個任務正使用的虛擬內存量，如果任務超出分配值，則直接將其殺掉	Whether virtual memory limits will be enforced for containers.
yarn.scheduler.maximum-allocation-mb	8192	RM爲每個container請求分配的最大內存（以MB爲單位）。 高於此值的內存請求將引發InvalidResourceRequestException。	The maximum allocation for every container request at the RM in MBs. Memory requests higher than this will throw an InvalidResourceRequestException.
yarn.scheduler.minimum-allocation-mb	1024	RM處每個container請求分配的最小內存（以MB爲單位）。 低於此值的內存請求將設置爲此屬性的值。 此外，資源管理器將關閉配置爲具有少於此值的內存的節點管理器。	The minimum allocation for every container request at the RM in MBs. Memory requests lower than this will be set to the value of this property. Additionally, a node manager that is configured to have less memory than this value will be shut down by the resource manager.

yarn 管理vcores相關配置

properties	value	描述	description
yarn.nodemanager.resource.cpu-vcores	-1	可以爲容器分配的vcore數。 RM調度程序在爲容器分配資源時使用它。 這不是用來限制YARN容器使用的CPU數量。 如果將其設置爲-1，並且yarn.nodemanager.resource.detect-hardware-capabilities爲true，則在Windows和Linux情況下會自動從硬件確定它。 在其他情況下，默認情況下，vcore的數量爲8。	Number of vcores that can be allocated for containers. This is used by the RM scheduler when allocating resources for containers. This is not used to limit the number of CPUs used by YARN containers. If it is set to -1 and yarn.nodemanager.resource.detect-hardware-capabilities is true, it is automatically determined from the hardware in case of Windows and Linux. In other cases, number of vcores is 8 by default.
yarn.nodemanager.resource.pcores-vcores-multiplier	-1	確定如何將physcal cores轉換爲vcore的乘數。 如果將yarn.nodemanager.resource.cpu-vcores設置爲-1（這意味着自動計算vcore），並且yarn.nodemanager.resource.detect-hardware-capabilities設置爲true，則使用此值。 vcore的數量將計算爲CPUs * multiplier.	Multiplier to determine how to convert phyiscal cores to vcores. This value is used if yarn.nodemanager.resource.cpu-vcores is set to -1(which implies auto-calculate vcores) and yarn.nodemanager.resource.detect-hardware-capabilities is set to true. The number of vcores will be calculated as number of CPUs * multiplier.
yarn.scheduler.minimum-allocation-vcores	1	就vcores而言，RM處每個container請求分配的最小vcores數量。 低於此值的請求將設置爲此屬性的值。 此外，資源管理器將關閉配置爲具有比該值更少的虛擬核的節點管理器。	The minimum allocation for every container request at the RM in terms of virtual CPU cores. Requests lower than this will be set to the value of this property. Additionally, a node manager that is configured to have fewer virtual cores than this value will be shut down by the resource manager.
yarn.scheduler.maximum-allocation-vcores	4	就vcores而言，RM上每個container請求分配的最大vcores數量。 高於此值的請求將引發InvalidResourceRequestException	The maximum allocation for every container request at the RM in terms of virtual CPU cores. Requests higher than this will throw an InvalidResourceRequestException.

相關鏈接：
yarn-site官方鏈接：
https://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-common/yarn-default.xml