linux 內存管理(13) - memblock

瞭解memblock機制。

1.概述

在引導內核的過程中,需要使用內存, 而這個時候內核的內存管理並沒有被創建, 因此也就需要一種精簡的內存管理系統先接受這個工作, 而在初始化完成後, 再將舊的接口廢棄, 轉而使用強大的buddy系統來進行內存管理.

早期的Linux內核在引導階段都是通過bootmem來完成初期的內存管理的, 但是後來的版本開始把bootmem棄用了,使用memblock機制.[refer to: Use memblock interface instead of bootmem]

拓展：bootmem
在啓動過程期間，儘管內存管理尚未初始化，但內核仍然需要分配內存以創建各種數據結構。bootmem分配器用於在啓動階段早期分配內存。
顯然，對該分配器的需求集中於簡單性方面，而不是性能和通用性。因此內核開發者決定實現一個最先適配(first-fit)分配器用於在啓動階段管理內存。這是可能想到的最簡單的方式。
該分配器使用一個位圖來管理頁，位圖比特位的數目與系統中物理內存頁的數目相同。比特位位1，表示頁以用；比特位位0，表示頁空閒。
在需要分配內存時，分配器逐位掃描位圖，知道找到一個能提供足夠連續頁的位置，即所謂的最先最佳(first-best)或最先適配位置。
該過程不是很高效，因爲每次分配都必須從頭掃描比特鏈。因此在內核完全初始化之後，不能將該適配器用於內存管理。夥伴系統(連同slab、slub、或slob分配器)是一個好得多的備選方案。

memblock算法是linux內核初始化階段的一個內存分配器,本質上是取代了原來的bootmem算法. memblock實現比較簡單,而它的作用就是在page allocator初始化之前來管理內存,完成分配和釋放請求.

Memblock is a method of managing memory regions during the early boot period when the usual kernel memory allocators are not up and running.

Memblock views the system memory as collections of contiguous regions. There are several types of these collections:

memory
describes the physical memory available to the kernel; this may differ from the actual physical memory installed in the system, for instance when the memory is restricted with mem= command line parameter
reserved
describes the regions that were allocated
physmap
describes the actual physical memory regardless of the possible restrictions; the physmap type is only available on some architectures.

2.memblock的數據結構

2.1.struct memblock

include/linux/memblock.h
struct memblock {
    bool bottom_up;  /* is bottom up direction? 
    如果true, 則允許由下而上地分配內存*/
    phys_addr_t current_limit; /*指出了內存塊的大小限制*/  
    /*  接下來的三個域描述了內存塊的類型，即預留型，內存型和物理內存*/
    struct memblock_type memory;
    struct memblock_type reserved;
#ifdef CONFIG_HAVE_MEMBLOCK_PHYS_MAP
    struct memblock_type physmem;
#endif
};

2.2.struct memblock_type

struct memblock_type
{
    unsigned long cnt;      /* number of regions */
    unsigned long max;      /* size of the allocated array */
    phys_addr_t total_size; /* size of all regions */
    struct memblock_region *regions;
};

2.3.struct memblock_region 維護一塊內存區塊，其定義爲：

struct memblock_region
{
    phys_addr_t base;
    phys_addr_t size;
    unsigned long flags;
#ifdef CONFIG_HAVE_MEMBLOCK_NODE_MAP
    int nid;
#endif
};

內存區塊被維護在不同的 struct memblock_type 的 regions 鏈表上，這是一個由數組構成的鏈表，鏈表通過每個區塊的基地址的大小，從小到大的排列。每個內存區塊代表的內存區不能與本鏈表中的其他內存區塊相互重疊，可以相連。內核初始定義了兩個內存區塊數組，如下：

static struct memblock_region memblock_memory_init_regions[INIT_MEMBLOCK_REGIONS] __initdata_memblock;
static struct memblock_region memblock_reserved_init_regions[INIT_MEMBLOCK_RESERVED_REGIONS] __initdata_memblok;

2.4.memblock_region flags字段

/* Definition of memblock flags. */
enum {
    MEMBLOCK_NONE       = 0x0,  /* No special request */
    MEMBLOCK_HOTPLUG    = 0x1,  /* hotpluggable region */
    MEMBLOCK_MIRROR     = 0x2,  /* mirrored region */
    MEMBLOCK_NOMAP      = 0x4,  /* don't add to kernel direct mapping */
};

MEMBLOCK 內存分配器基礎框架如下：

MEMBLOCK 分配器使用一個 struct memblock 結構維護着兩種內存，其中成員 memory 維護着可用物理內存區域；成員 reserved 維護着操作系統預留的內存區域。每個區域使用數據結構 struct memblock_type 進行管理，其成員 regions 負責維護該類型內存的所有內存區，每個內存區使用數據結構 struct memblock_region 進行維護。

2.5.struct memblock 實例化

內核實例化了一個 struct memblock 結構，以供 MEMBLOCK 進行內存的管理，其定義如下：

mm/memblock.c:
struct memblock memblock __initdata_memblock = {
        .memory.regions         = memblock_memory_init_regions,
        .memory.cnt             = 1,    /* empty dummy entry */
        .memory.max             = INIT_MEMBLOCK_REGIONS,
        .memory.name            = "memory",

        .reserved.regions       = memblock_reserved_init_regions,
        .reserved.cnt           = 1,    /* empty dummy entry */
        .reserved.max           = INIT_MEMBLOCK_RESERVED_REGIONS,
        .reserved.name          = "reserved",

#ifdef CONFIG_HAVE_MEMBLOCK_PHYS_MAP
        .physmem.regions        = memblock_physmem_init_regions,
        .physmem.cnt            = 1,    /* empty dummy entry */
        .physmem.max            = INIT_PHYSMEM_REGIONS,
        .physmem.name           = "physmem",
#endif

        .bottom_up              = false,
        .current_limit          = MEMBLOCK_ALLOC_ANYWHERE,
};

從上面的數據中可以看出，對於可用物理內存，其名字設定爲 “memory”，初始狀態系統下，可用物理物理的所有內存區塊都維護在 memblock_memory_init_regions 上，當前情況下，可用物理內存區只包含一個內存區塊，然而可用物理內存可管理 INIT_MEMBLOCK_REGIONS 個內存區塊；同理，對於預留內存，其名字設定爲 “reserved”，初始狀態下，預留物理內存的所有區塊都維護在 memblock_reserved_init_regions 上，當前情況下，預留物理內存區只包含一個內存區塊，然而預留內存區可以維護管理 INIT_MEMBLOCK_RESERVED_REGIONS 個內存區塊。

3.APIs

memblock_phys_mem_size

memblock_phys_mem_size
phys_addr_t __init_memblock memblock_phys_mem_size(void)
{
	return memblock.memory.total_size;
}

函數的作用是獲得可用物理內存的總體積。函數直接返回 memblock.memory 的 total_size， total_size 成員存儲該內存區的體積大小。

memblock_reserved_size

memblock_reserved_size
phys_addr_t __init_memblock memblock_reserved_size(void)
{
	return memblock.reserved.total_size;
}

函數的作用是獲得預留區內存的總體積。函數直接返回 memblock.reserved 的 total_size， total_size 成員存儲該內存區的體積大小。

memblock_start_of_DRAM

/* lowest address */
phys_addr_t __init_memblock memblock_start_of_DRAM(void)
{
	return memblock.memory.regions[0].base;
}

函數的作用是獲得 DRAM 的起始地址。DRAM 的起始地址就是 memblock.memory 內存區第一個內存區塊的起始物理地址。函數直接返回 memblock.memory 的 regions[0].base， regions[0].base 成員存儲 DRAM 的起始物理地址。

memblock_end_of_DRAM

phys_addr_t __init_memblock memblock_end_of_DRAM(void)
{
	int idx = memblock.memory.cnt - 1;
	return (memblock.memory.regions[idx].base + memblock.memory.regions[idx].size);
}

函數的作用是獲得 DRAM 的終止地址。DRAM 的終止地址就是 memblock.memory 內存區最後一個內存區塊的終止物理地址。最後一個內存區的索引是 memblock.memory.cnt - 1，所以這個索引對應的內存區的終止地址就是 DRAM 的結束地址。

memblock_is_reserved

memblock_is_reserved
bool __init_memblock memblock_is_reserved(phys_addr_t addr)
{
	return memblock_search(&memblock.reserved, addr) != -1;
}

函數的作用是檢查某個物理地址是否屬於預留區。參數 addr 指向要檢查的地址。函數調用 memblock_search() 函數進行地址檢查。

memblock_is_memory

memblock_is_memory
bool __init_memblock memblock_is_memory(phys_addr_t addr)
{
	return memblock_search(&memblock.memory, addr) != -1;
}

函數的作用是檢查某個物理地址是否屬於可用內存區。參數 addr 指向要檢查的地址。函數調用 memblock_search() 函數進行地址檢查。

memblock_is_region_memory

bool __init_memblock memblock_is_region_memory(phys_addr_t base, phys_addr_t size)
{
	int idx = memblock_search(&memblock.memory, base);
	phys_addr_t end = base + memblock_cap_size(base, &size);

	if (idx == -1)
		return false;
	return (memblock.memory.regions[idx].base +
		 memblock.memory.regions[idx].size) >= end;
}

函數的作用是檢查某段內存區是否屬於可用內存區。參數 base 需要檢查的區段的起始物理地址。參數 size 需要檢查區段的長度。函數調用 memblock_search() 函數對 base 參數進行檢查，memblock_search() 函數返回 base 地址在可用內存區的索引，接着計算出需要檢查區段的終止地址，最後通過檢查終止地址是否在 idx 對應的內存區段之內，如果在表示這段內存區塊在可用物理內存區內；如果不在，則表示這段內存區塊不屬於可用物理內存區段。

memblock_is_region_reserved

bool __init_memblock memblock_is_region_reserved(phys_addr_t base, phys_addr_t size)
{
	memblock_cap_size(base, &size);
	return memblock_overlaps_region(&memblock.reserved, base, size);
}

函數的作用是檢查某段內存區是否屬於預留區。參數 base 需要檢查的區段的起始物理地址。參數 size 需要檢查區段的長度。函數調用 memblock_cap_size() 函數對 size 參數進行處理之後，傳遞給 memblock_overlaps_region() 函數檢查 base 和 size 對應的內存區是否屬於預留區。

memblock_get_current_limit

phys_addr_t __init_memblock memblock_get_current_limit(void)
{
	return memblock.current_limit;
}

函數的作用就是返回 MEMBLOCK 的當前 limit。

memblock_set_current_limit

void __init_memblock memblock_set_current_limit(phys_addr_t limit)
{
	memblock.current_limit = limit;
}

函數的作用就是設置 MEMBLOCK 的當前 limit。

具體函數使用參考案例：

https://github.com/BiscuitOS/HardStack/tree/master/Memory-Allocator/Memblock-allocator/API

4.案例分析

1>.驅動的源碼如下：

#include <linux/init.h>
#include <linux/kernel.h>
#include <linux/memblock.h>

int bs_debug = 0;

#ifdef CONFIG_DEBUG_MEMBLOCK_HELPER
int __init debug_memblock_helper(void)
{
	struct memblock_region *reg;
	phys_addr_t size;
	phys_addr_t addr;
	phys_addr_t limit;
	bool state;
	int nid;

	/*
	 * Emulate memory
	 *
	 *                      memblock.memory
	 * 0     | <----------------------------------------> |
	 * +-----+---------+-------+----------+-------+-------+----+
	 * |     |         |       |          |       |       |    |
	 * |     |         |       |          |       |       |    |
	 * |     |         |       |          |       |       |    |
	 * +-----+---------+-------+----------+-------+-------+----+
	 *                 | <---> |          | <---> |
	 *                 Reserved 0         Reserved 1
	 *
	 * Memroy Region:   [0x60000000, 0xa0000000]
	 * Reserved Region: [0x80000000, 0x8d000000]
	 * Reserved Region: [0x90000000, 0x92000000]
	 */
	memblock_reserve(0x80000000, 0xd000000);
	memblock_reserve(0x90000000, 0x2000000);
	pr_info("Memory Regions:\n");
	for_each_memblock(memory, reg)
		pr_info("Region: %#x - %#x\n", reg->base,
					reg->base + reg->size);
	pr_info("Reserved Regions:\n");
	for_each_memblock(reserved, reg)
		pr_info("Region: %#x - %#x\n", reg->base,
					reg->base + reg->size);

	/* Obtain memblock.memory total size */
	size = memblock_phys_mem_size();
	pr_info("Phyiscal Memory total size: %#x\n", size);

	/* Obtain memblock.reserved total size */
	size = memblock_reserved_size();
	pr_info("Reserved Memory total size: %#x\n", size);

	/* Obtain Start physical address of DRAM */
	addr = memblock_start_of_DRAM();
	pr_info("Start address of DRAM:      %#x\n", addr);

	/* Obtain End of physical address of DRAM */
	addr = memblock_end_of_DRAM();
	pr_info("End address of DRAM:        %#x\n", addr);

	/* Check address is memblock.reserved */
	addr = 0x81000000; /* Assume address in memblock.reserved */
	state = memblock_is_reserved(addr);
	if (state)
		pr_info("Address: %#x in reserved.\n", addr);

	/* Check address in memblock.memory */
	addr = 0x62000000; /* Assume address in memblock.memory */
	state = memblock_is_memory(addr);
	if (state)
		pr_info("Address: %#x in memory.\n", addr);

	/* Check region in memblock.memory */
	addr = 0x62000000;
	size = 0x100000; /* Assume [0x62000000, 0x62100000] in memory */
	state = memblock_is_region_memory(addr, size);
	if (state)
		pr_info("Region: [%#x - %#x] in memblock.memory.\n",
				addr, addr + size);

	/* Check region in memblock.reserved */
	addr = 0x80000000;
	size = 0x100000; /* Assume [0x80000000, 0x80100000] in reserved */
	state = memblock_is_region_reserved(addr, size);
	if (state)
		pr_info("Region: [%#x - %#x] in memblock.reserved.\n",
				addr, addr + size);

	/* Obtain current limit for memblock */
	limit = memblock_get_current_limit();
	pr_info("MEMBLOCK current_limit: %#x\n", limit);

	/* Set up current_limit for MEMBLOCK */
	memblock_set_current_limit(limit);

	/* Check memblock regions is hotpluggable */
	state = memblock_is_hotpluggable(&memblock.memory.regions[0]);
	if (state)
		pr_info("MEMBLOCK memory.regions[0] is hotpluggable.\n");
	else
		pr_info("MEMBLOCK memory.regions[0] is not hotpluggable.\n");

	/* Check memblock regions is mirror */
	state = memblock_is_mirror(&memblock.memory.regions[0]);
	if (state)
		pr_info("MEMBLOCK memory.regions[0] is mirror.\n");
	else
		pr_info("MEMBLOCK memory.regions[0] is not mirror.\n");

	/* Check memblock regions is nomap */
	state = memblock_is_nomap(&memblock.memory.regions[0]);
	if (state)
		pr_info("MEMBLOCK memory.regions[0] is nomap.\n");
	else
		pr_info("MEMBLOCK memory.regions[0] is not nomap.\n");

	/* Check region nid information */
	nid = memblock_get_region_node(&memblock.memory.regions[0]);
	pr_info("MEMBLOCK memory.regions[0] nid: %#x\n", nid);
	/* Set up region nid */
	memblock_set_region_node(&memblock.memory.regions[0], nid);

	/* Obtian MEMBLOCK allocator direction */
	state = memblock_bottom_up();
	pr_info("MEMBLOCK direction: %s", state ? "bottom-up" : "top-down");
	/* Set up MEMBLOCK allocate direction */
	memblock_set_bottom_up(state);

	return 0;
}
#endif

驅動直接編譯進內核，將驅動放到 drivers/BiscuitOS/ 目錄下，命名爲 memblock.c。

2>.添加drivers/BiscuitOS/Kconfig：

menu "Biscuitos support"

config BISCUITOS
    bool "BiscuitOS driver"

if BISCUITOS

config BISCUITOS_DRV 
    bool "BiscuitOS driver"

config DEBUG_MEMBLOCK_HELPER
    bool "memblock helper"

config MEMBLOCK_ALLOCATOR
    bool "MEMBLOCK allocator"

if MEMBLOCK_ALLOCATOR
                                                                                                             
config DEBUG_MEMBLOCK_PHYS_ALLOC_TRY_NID
    bool "memblock_phys_alloc_try_nid()"

endif # MEMBLOCK_ALLOCATOR

endif # BISCUITOS_DRV

endmenu

3>.添加drivers/BiscuitOS/Makefile:

obj-$(CONFIG_MEMBLOCK_ALLOCATOR) += memblock.o

4>.添加iTop-4412_scp_defconfig:

diff --git a/arch/arm/configs/iTop-4412_scp_defconfig b/arch/arm/configs/iTop-4412_scp_defconfig
index 828edc8f..58022467 100644
--- a/arch/arm/configs/iTop-4412_scp_defconfig
+++ b/arch/arm/configs/iTop-4412_scp_defconfig
 # CONFIG_STRING_SELFTEST is not set
 # CONFIG_VIRTUALIZATION is not set
+CONFIG_BISCUITOS=y
+CONFIG_BISCUITOS_DRV=y
+CONFIG_MEMBLOCK_ALLOCATOR=y
+CONFIG_DEBUG_MEMBLOCK_HELPER=y

5>.增加調試點

驅動運行還需要在內核的指定位置添加調試點，由於該驅動需要在內核啓動階段就使用，參考下面 patch 將源碼指定位置添加調試代碼：

diff --git a/arch/arm/kernel/setup.c b/arch/arm/kernel/setup.c
index 375b13f7e..fec6919a9 100644
--- a/arch/arm/kernel/setup.c
+++ b/arch/arm/kernel/setup.c
@@ -1073,6 +1073,10 @@ void __init hyp_mode_check(void)
 void __init setup_arch(char **cmdline_p)
 {
 	const struct machine_desc *mdesc;
+#ifdef CONFIG_DEBUG_MEMBLOCK_HELPER
+	extern int bs_debug;
+	extern int debug_memblock_helper(void);
+#endif

 	setup_processor();
 	mdesc = setup_machine_fdt(__atags_pointer);
@@ -1104,6 +1108,10 @@ void __init setup_arch(char **cmdline_p)
 	strlcpy(cmd_line, boot_command_line, COMMAND_LINE_SIZE);
 	*cmdline_p = cmd_line;

ck_add
+#ifdef CONFIG_DEBUG_MEMBLOCK_HELPER
+	debug_memblock_helper();
+#endif
+
 	early_fixmap_init();
 	early_ioremap_init();

Note:
詳細分析：- https://biscuitos.github.io/blog/MMU-ARM32-MEMBLOCK-memblock_information/

Arm64架構下, 內核在start_kernel()->setup_arch()中通過arm64_memblock_init( )完成了memblock的初始化之後, 接着通過setup_arch()->paging_init()開始初始化分頁機制。

refer to

Documentation/core-api/boot-time-mm.rst
https://www.lagou.com/lgeduarticle/23340.html
https://github.com/BiscuitOS/HardStack/tree/master/Memory-Allocator/Memblock-allocator/API

linux 內存管理(13) - memblock

使用neovim打造go ide(支持代碼跳轉, 代碼補全, 實時語法檢查)

挑戰程序設計競賽 2.3章習題 poj 3046 Ant Counting

Shell/Python中的用戶名獲取

linux 鎖機制 - spinlock

Linux音頻子系統(3) - ALSA ASoC

Linux 內核調試(2) - Kdump

linux block framework(4) - kernel devices.txt

linux 內存管理(13) - memblock

https://yachay.unat.edu.pe/blog/index.php?comment_area=format_blog&comment_component=blog&comment_co

linux以太網驅動總結