发表更新数据库 / PostgreSQL21 分钟读完 (大约3181个字)

PG内存管理篇二

更详细地讲述PG的内存管理，演示代码为OpenGauss。省流的话直接看总结。

引言

上一篇我们谈论了内存上下文的接呢结构，内存申请的函数palloc和释放函数pfree。现在我们讨论一下内存自动管理的原理。

PG给了我们内存申请的接口palloc，难道还要我们自己手动调用pfree吗？答案是当然不用。有了内存上下文之后，我们只需要切换到对应的内存上下文，直接调用palloc，申请的内存将在对应内存上下文MemContext删除时一并释放。

创建内存上下文

1
2
3
4
5
6
7
8
9
MemoryContext AllocSetContextCreate(
  _in_ MemoryContext parent, 
  _in_ const char* name, 
  _in_ Size minContextSize,
  _in_ Size initBlockSize, 
  _in_ Size maxBlockSize, 
  _in_ MemoryContextType contextType, 
  _in_ Size maxSize,
  _in_ bool isSession);

创建内存上下有一个公用的接口，该接口会根据内存上下文的类型选择合适的创建函数，这里以标准内存上下文的创建为说明。

注意这三个参数的作用：

1
2
3
* minContextSize: minimum context size
* initBlockSize: initial allocation block size
* maxBlockSize: maximum allocation block size
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
MemoryContext GenericMemoryAllocator::AllocSetContextCreate(MemoryContext parent, const char* name, Size minContextSize,
    Size initBlockSize, Size maxBlockSize, Size maxSize, bool isShared, bool isSession)
{
    AllocSet context;
    NodeTag type = isShared ? T_SharedAllocSetContext : T_AllocSetContext;
    bool isTracked = false;
    unsigned long value = isShared ? IS_SHARED : 0;
    MemoryProtectFuncDef* func = NULL;

    if (isShared)
        func = &SharedFunctions;
    else if (!isSession && (parent == NULL || parent->session_id == 0))
        func = &GenericFunctions;
    else
        func = &SessionFunctions;

    /* we want to be sure ErrorContext still has some memory even if we've run out elsewhere!
     * Don't limit the memory allocation for ErrorContext. And skip memory tracking memory allocation.
     */
    if ((0 != strcmp(name, "ErrorContext")) && (0 != strcmp(name, "MemoryTrackMemoryContext")) &&
        (strcmp(name, "Track MemoryInfo hash") != 0) && (0 != strcmp(name, "DolphinErrorData")))
        value |= IS_PROTECT;

    /* only track the unshared context after t_thrd.mem_cxt.mem_track_mem_cxt is created */
    if (func == &GenericFunctions && parent && (MEMORY_TRACKING_MODE > MEMORY_TRACKING_PEAKMEMORY) && t_thrd.mem_cxt.mem_track_mem_cxt &&
        (t_thrd.utils_cxt.ExecutorMemoryTrack == NULL || ((AllocSet)parent)->track)) {
        isTracked = true;
        value |= IS_TRACKED;
    }

    /* Do the type-independent part of context creation */
    context = (AllocSet)MemoryContextCreate(type, sizeof(AllocSetContext), parent, name, __FILE__, __LINE__);

    if (isShared && maxSize == DEFAULT_MEMORY_CONTEXT_MAX_SIZE) {
        // default maxSize of shared memory context.
        context->maxSpaceSize = SHARED_MEMORY_CONTEXT_MAX_SIZE;
    } else {
        // set by user (shared or generic),default generic max size(eg. 10M),
        // these three types run following scope:
        context->maxSpaceSize = maxSize + SELF_GENRIC_MEMCTX_LIMITATION;
    }
    /*
     * If MemoryContext's name is in white list(GUC parameter,see @memory_context_limited_white_list),
     * then set maxSize as infinite,that is unlimited.
     */
#ifdef MEMORY_CONTEXT_CHECKING
    MemoryContextControlSet(context, name);
#endif

    /* assign the method function with specified templated to the context */
    AllocSetContextSetMethods(value, ((MemoryContext)context)->methods);

    /*
     * Make sure alloc parameters are reasonable, and save them.
     *
     * We somewhat arbitrarily enforce a minimum 1K block size.
     */
    initBlockSize = MAXALIGN(initBlockSize);
    if (initBlockSize < 1024)
        initBlockSize = 1024;
    maxBlockSize = MAXALIGN(maxBlockSize);
    if (maxBlockSize < initBlockSize)
        maxBlockSize = initBlockSize;
    context->initBlockSize = initBlockSize;
    context->maxBlockSize = maxBlockSize;
    context->nextBlockSize = initBlockSize;

    /* initialize statistic */
    context->totalSpace = 0;
    context->freeSpace = 0;

    /* create the memory track structure */
    if (isTracked)
        MemoryTrackingCreate((MemoryContext)context, parent);

    /*
     * Compute the allocation chunk size limit for this context.  It can't be
     * more than ALLOC_CHUNK_LIMIT because of the fixed number of freelists.
     * If maxBlockSize is small then requests exceeding the maxBlockSize, or
     * even a significant fraction of it, should be treated as large chunks
     * too.  For the typical case of maxBlockSize a power of 2, the chunk size
     * limit will be at most 1/8th maxBlockSize, so that given a stream of
     * requests that are all the maximum chunk size we will waste at most
     * 1/8th of the allocated space.
     *
     * We have to have allocChunkLimit a power of two, because the requested
     * and actually-allocated sizes of any chunk must be on the same side of
     * the limit, else we get confused about whether the chunk is "big".
     */
    context->allocChunkLimit = ALLOC_CHUNK_LIMIT;
    while ((Size)(context->allocChunkLimit + ALLOC_CHUNKHDRSZ) >
           (Size)((maxBlockSize - ALLOC_BLOCKHDRSZ) / ALLOC_CHUNK_FRACTION))
        context->allocChunkLimit >>= 1;

    /*
     * Grab always-allocated space, if requested
     */
    if (minContextSize > ALLOC_BLOCKHDRSZ + ALLOC_CHUNKHDRSZ) {
        Size blksize = MAXALIGN(minContextSize);
        AllocBlock block;

        if (GS_MP_INITED)
            block = (AllocBlock)(*func->malloc)(blksize, (value & IS_PROTECT) == 1 ? true : false);
        else
            gs_malloc(blksize, block, AllocBlock);

        if (block == NULL) {
            ereport(ERROR,
                (errcode(ERRCODE_OUT_OF_LOGICAL_MEMORY),
                    errmsg("memory is temporarily unavailable"),
                    errdetail("Failed while creating memory context \"%s\".", name)));
        }
        block->aset = context;
        block->freeptr = ((char*)block) + ALLOC_BLOCKHDRSZ;
        block->endptr = ((char*)block) + blksize;
        block->allocSize = blksize;
#ifdef MEMORY_CONTEXT_CHECKING
        block->magicNum = BlkMagicNum;
#endif

        context->totalSpace += blksize;
        context->freeSpace += block->endptr - block->freeptr;

        /* update the memory tracking information when allocating memory */
        if (isTracked)
            MemoryTrackingAllocInfo((MemoryContext)context, blksize);

        block->prev = NULL;
        block->next = NULL;
        /* Remember block as part of block list */
        context->blocks = block;
        /* Mark block as not to be released at reset time */
        context->keeper = block;
    }

    context->header.is_shared = isShared;
    if (isShared)
        (void)pthread_rwlock_init(&(context->header.lock), NULL);

    return (MemoryContext)context;
}

这个函数分为几个部分：

  • 选择合适的内存申请函数 func
  • 设置value（用于methods字段，选取合适虚表）
  • 创建context
  • 设置context的methods字段
  • 设置context的allocChunkLimit
  • 若合适，申请context的第一个block

其中func、value、methods关联度比较大，涉及内存保护机制。

cllockChunkLimit适合仔细揣摩：maxBlockSize - ALLOC_BLOCKHDRSZ就是实际能够分配给chunk的空间，ALLOC_CHUNK_FRACTION为4，也就是说至少要能平均分配4个大小的chunk。context->allocChunkLimit + ALLOC_CHUNKHDRSZ就是申请一个段内存要加上header的大小。

1
2
3
4
context->allocChunkLimit = ALLOC_CHUNK_LIMIT;
while ((Size)(context->allocChunkLimit + ALLOC_CHUNKHDRSZ) >
       (Size)((maxBlockSize - ALLOC_BLOCKHDRSZ) / ALLOC_CHUNK_FRACTION))
    context->allocChunkLimit >>= 1;

合适时会申请aset的第一个block，这个时机依据的是minSize。

1
if (minContextSize > ALLOC_BLOCKHDRSZ + ALLOC_CHUNKHDRSZ)

下面看context的创建函数：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
MemoryContext MemoryContextCreate(
    NodeTag tag, Size size, MemoryContext parent, const char* name, const char* file, int line)
{
    MemoryContext node;
    Size needed = size + sizeof(MemoryContextMethods) + strlen(name) + 1;
    MemoryContext root = ChooseRootContext(tag, parent);
    errno_t rc = EOK;

    /* Get space for node and name */
    if (root != NULL) {
        /* Normal case: allocate the node in Root MemoryContext */
        node = (MemoryContext)MemoryAllocFromContext(root, needed, file, line);
    } else {
        /* Special case for startup: use good ol' malloc */
        node = (MemoryContext)malloc(needed);
        if (node == NULL)
            ereport(ERROR,
                (errcode(ERRCODE_OUT_OF_MEMORY),
                    errmsg("out of memory"),
                    errdetail("Failed on request of size %lu in %s:%d.", (unsigned long)needed, file, line)));
    }

		...
    return node;
}

其中最主要的就是：选取root，然后从root上下文创建。

1
2
MemoryContext root = ChooseRootContext(tag, parent);
node = (MemoryContext)MemoryAllocFromContext(root, needed, file, line);

这个tag是 NodeTag type = isShared ? T_SharedAllocSetContext : T_AllocSetContext;

选取root的函数，可以看到就三种值：

  • g_instance.instance_context;
  • u_sess->top_mem_cxt;
  • t_thrd.top_mem_cxt;
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
static MemoryContext ChooseRootContext(NodeTag tag, MemoryContext parent)
{
    MemoryContext root = NULL;

    if (tag == T_SharedAllocSetContext || tag == T_MemalignSharedAllocSetContext) {
        root = g_instance.instance_context;
    } else if (parent) {
        if (parent->type == T_SharedAllocSetContext || parent->type == T_MemalignSharedAllocSetContext) {
            root = g_instance.instance_context;
        } else if (parent->session_id > 0) {
            Assert(u_sess->session_id == parent->session_id);
            root = u_sess->top_mem_cxt;
        } else {
            root = t_thrd.top_mem_cxt;
        }
    } else {
        root = t_thrd.top_mem_cxt;
    }

    return root;
}

因此我们可以得出这样一个结论：不管传入公共接口AllocSetContextCreate的parent上下文是什么，最终总是会选取对应的根上下文进行内存分配。

但是context的树形关系并未被破坏，node的parent会被正确设置。

1
2
3
4
5
6
7
8
9
10
if (parent) {
    node->parent = parent;
    node->nextchild = parent->firstchild;
    if (parent->firstchild != NULL)
        parent->firstchild->prevchild = node;
    parent->firstchild = node;

    node->level = parent->level + 1;
} else
    node->level = 0;

总结

在内存上下文创建的公共接口中，parent只是用来正确设置树形关系，MemoryContext的创建内存实际由对应的根上下文分配。minContextSize决定了是否给创建的上下文申请第一个Block，initContextSize和maxContextSize没有多大用处。

删除内存上下文

在进程退出时gs_thread_exit(code); 看看进行了什么，标准的内存上下文删除函数，调用的是mcxt_methods->mcxt_destroy。

1
2
3
4
MemoryContextDestroyAtThreadExit(t_thrd.top_mem_cxt);
  t_thrd.top_mem_cxt = NULL;
  TopMemoryContext = NULL;
  u_sess = NULL;
1
2
#define MemoryContextDestroyAtThreadExit(context) \
    (((MemoryContext)context)->mcxt_methods->mcxt_destroy(context))
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
static McxtOperationMethods StdMcxtOpMtd = {
    std_MemoryContextReset,
    std_MemoryContextDelete,
    std_MemoryContextDeleteChildren,
    std_MemoryContextDestroyAtThreadExit,
    std_MemoryContextResetAndDeleteChildren,
    std_MemoryContextSetParent
#ifdef MEMORY_CONTEXT_CHECKING
    ,
    std_MemoryContextCheck
#endif
};

typedef struct McxtOperationMethods {
    void (*mcxt_reset)(MemoryContext context);
    void (*mcxt_delete)(MemoryContext context);
    void (*mcxt_delete_children)(MemoryContext context, List* context_list);
    void (*mcxt_destroy)(MemoryContext context);
    void (*mcxt_reset_and_delete_children)(MemoryContext context);
    void (*mcxt_set_parent)(MemoryContext context, MemoryContext new_parent);
#ifdef MEMORY_CONTEXT_CHECKING
    void (*mcxt_check)(MemoryContext context, bool own_by_session);
#endif
} McxtOperationMethods;

std_MemoryContextDestroyAtThreadExit

std_MemoryContextDestroyAtThreadExit主要做两件事情：

  1. MemoryContextDeleteChildren(pContext, NULL); 删除分配个子上下文的内存
  2. (*pContext->methods->delete_context)(pContext); 删除分配给上下文本身的内存

​ free(pContext); 释放上下文本身

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
void std_MemoryContextDestroyAtThreadExit(MemoryContext context)
{
    MemoryContext pContext = context;
    if (!IsTopMemCxt(context)) {
        PreventActionOnSealedContext(context);
    } else {
#ifdef MEMORY_CONTEXT_CHECKING
        /* before delete top memcxt,  you should close lsc */
        if (EnableGlobalSysCache() && context == t_thrd.top_mem_cxt && t_thrd.lsc_cxt.lsc != NULL) {
            Assert(t_thrd.lsc_cxt.lsc->is_closed);
        }
#endif
    }

    if (pContext != NULL) {
        /* To avoid delete current context */
        MemoryContextSwitchTo(pContext);

        /* Delete all its decendents */
        Assert(!pContext->parent);
        MemoryContextDeleteChildren(pContext, NULL); // 第一件事

        /* Delete the top context itself */
        RemoveMemoryContextInfo(pContext);
        (*pContext->methods->delete_context)(pContext); //第二件事

        if (pContext != t_thrd.top_mem_cxt)
            return;

        /*
         *	Lock this thread's delete MemoryContext.
         *	Because pv_session_memory_detail view will recursive MemoryContext tree.
         *	Relate to pgstatfuncs.c:pvSessionMemoryDetail().
         */
        if (t_thrd.proc != NULL && t_thrd.proc->topmcxt != NULL) {
            (void)syscalllockAcquire(&t_thrd.proc->deleMemContextMutex);
            free(pContext);
            (void)syscalllockRelease(&t_thrd.proc->deleMemContextMutex);
        } else {
            free(pContext);
        }
    }
}

MemoryContextDeleteChildren

std_MemoryContextDeleteChildren 会调用 MemoryContextDeleteInternal，而 MemoryContextDeleteInternal还会调用std_MemoryContextDeleteChildren。

总之就是递归遍历context->firstchild，在依次调用methods->delete_context来收回分配给它的内存，注意context本身没有被释放。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
/*
 * std_MemoryContextDeleteChildren
 *		Delete all the descendants of the named context and release all
 *		space allocated therein.  The named context itself is not touched.
 */
void std_MemoryContextDeleteChildren(MemoryContext context, List* context_list)
{
    AssertArg(MemoryContextIsValid(context));
    List res_list = {T_List, 0, NULL, NULL};
    if (context_list == NULL) {
        context_list = &res_list;
    }

    if (MemoryContextIsShared(context))
        MemoryContextLock(context);

    /*
     * MemoryContextDelete will delink the child from me, so just iterate as
     * long as there is a child.
     */
    while (context->firstchild != NULL)
        MemoryContextDeleteInternal(context->firstchild, true, context_list);

    if (MemoryContextIsShared(context))
        MemoryContextUnlock(context);

    if (context_list == &res_list) {
        FreeMemoryContextList(&res_list);
    }
}

static void MemoryContextDeleteInternal(MemoryContext context, bool parent_locked,
    List* context_list)
{
    AssertArg(MemoryContextIsValid(context));
    /* We had better not be deleting t_thrd.top_mem_cxt ... */
    Assert(context != t_thrd.top_mem_cxt);
    /* And not CurrentMemoryContext, either */
    Assert(context != CurrentMemoryContext);

    MemoryContextDeleteChildren(context, context_list);

#ifdef MEMORY_CONTEXT_CHECKING
    /* Memory Context Checking */
    MemoryContextCheck(context, context->session_id > 0);
#endif

    MemoryContext parent = context->parent;


    PG_TRY();
    {
        HOLD_INTERRUPTS();
        if (context->session_id > 0) {
            (void)syscalllockAcquire(&u_sess->utils_cxt.deleMemContextMutex);
        } else if (t_thrd.proc != NULL && t_thrd.proc->topmcxt != NULL) {
            (void)syscalllockAcquire(&t_thrd.proc->deleMemContextMutex);
        }

        /*
         * If the parent context is shared and is already locked by the caller,
         * no need to relock again. In fact, that's not the right thing to do
         * since it will lead to a self-deadlock
         */
        if (parent && MemoryContextIsShared(parent) && (!parent_locked))
            MemoryContextLock(parent);
        /*
         * We delink the context from its parent before deleting it, so that if
         * there's an error we won't have deleted/busted contexts still attached
         * to the context tree.  Better a leak than a crash.
         */
        TopMemCxtUnSeal();
        MemoryContextSetParent(context, NULL);
        if (parent != NULL && MemoryContextIsShared(parent) && (parent_locked == false))
            MemoryContextUnlock(parent);

        RemoveMemoryContextInfo(context);
        (*context->methods->delete_context)(context);
        (void)lappend2(context_list, &context->cell);

        if (context->session_id > 0) {
            (void)syscalllockRelease(&u_sess->utils_cxt.deleMemContextMutex);
        } else if (t_thrd.proc != NULL && t_thrd.proc->topmcxt != NULL) {
            (void)syscalllockRelease(&t_thrd.proc->deleMemContextMutex);
        }
        TopMemCxtSeal();
        RESUME_INTERRUPTS();
    }
    PG_CATCH();
    {
        if (context->session_id > 0) {
            (void)syscalllockRelease(&u_sess->utils_cxt.deleMemContextMutex);
        } else if (t_thrd.proc != NULL && t_thrd.proc->topmcxt != NULL) {
            (void)syscalllockRelease(&t_thrd.proc->deleMemContextMutex);
        }
        TopMemCxtSeal();
        PG_RE_THROW();
    }
    PG_END_TRY();
}

AllocSetDelete

我们看AllocSetDelete这个函数，它的主要作用就是清空分配给context的blocks，从而达到释放context的内存目的，但是context本身并没有被释放。因此，在std_MemoryContextDestroyAtThreadExit中还需再次调用free来释放本身。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
/*
 * AllocSetDelete
 *		Frees all memory which is allocated in the given set,
 *		in preparation for deletion of the set.
 *
 * Unlike AllocSetReset, this *must* free all resources of the set.
 * But note we are not responsible for deleting the context node itself.
 */
template <bool enable_memoryprotect, bool is_shared, bool is_tracked>
void GenericMemoryAllocator::AllocSetDelete(MemoryContext context)
{
    AllocSet set = (AllocSet)context;
    AssertArg(AllocSetIsValid(set));

    AllocBlock block = set->blocks;
    MemoryProtectFuncDef* func = NULL;

    if (set->blocks == NULL) {
        return;
    }

    if (is_shared) {
        MemoryContextLock(context);
        func = &SharedFunctions;
    } else {
        CHECK_CONTEXT_OWNER(context);
        if (context->session_id > 0)
            func = &SessionFunctions;
        else
            func = &GenericFunctions;
    }

#ifdef MEMORY_CONTEXT_CHECKING
    /* Check for corruption and leaks before freeing */
    AllocSetCheck(context);
#endif

    /* Make it look empty, just in case... */
    MemSetAligned(set->freelist, 0, sizeof(set->freelist));
    set->blocks = NULL;
    set->keeper = NULL;

    while (block != NULL) {
        AllocBlock next = block->next;
        Size tempSize = block->allocSize;

        if (is_tracked)
            MemoryTrackingFreeInfo(context, tempSize);

        if (GS_MP_INITED)
            (*func->free)(block, tempSize);
        else
            gs_free(block, tempSize);
        block = next;
    }

    /* reset to 0 after deletion. */
    set->freeSpace = 0;
    set->totalSpace = 0;

    if (is_shared) {
        MemoryContextUnlock(context);
    }
}

总结

std_MemoryContextDestroyAtThreadExit 通过两步走的方式释放一个根内存上下文。

第一步递归释放孩子结点的内存，但是孩子结点并没有释放。

第二步释放根结点的内存（注意孩子结点是从根结点的内存申请的，因此孩子结点被释放了），然后在进一步释放根结点本身。

内存保护机制

OG还拓展了内存保护机制，在创建上下文确定methods时，调用了这么一个函数：

1
2
/* assign the method function with specified templated to the context */
AllocSetContextSetMethods(value, ((MemoryContext)context)->methods);
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
void GenericMemoryAllocator::AllocSetContextSetMethods(unsigned long value, MemoryContextMethods* method)
{
    bool isProt = (value & IS_PROTECT) ? true : false;
    bool isShared = (value & IS_SHARED) ? true : false;
    bool isTracked = (value & IS_TRACKED) ? true : false;

    if (isProt) {
        if (isShared)
            AllocSetMethodDefinition<true, true, false>(method);
        else {
            if (isTracked)
                AllocSetMethodDefinition<true, false, true>(method);
            else
                AllocSetMethodDefinition<true, false, false>(method);
        }
    } else {
        if (isShared)
            AllocSetMethodDefinition<false, true, false>(method);
        else {
            if (isTracked)
                AllocSetMethodDefinition<false, false, true>(method);
            else
                AllocSetMethodDefinition<false, false, false>(method);
        }
    }
}

template <bool enable_memoryprotect, bool is_shared, bool is_tracked>
void GenericMemoryAllocator::AllocSetMethodDefinition(MemoryContextMethods* method)
{
    method->alloc = &GenericMemoryAllocator::AllocSetAlloc<enable_memoryprotect, is_shared, is_tracked>;
    method->free_p = &GenericMemoryAllocator::AllocSetFree<enable_memoryprotect, is_shared, is_tracked>;
    method->realloc = &GenericMemoryAllocator::AllocSetRealloc<enable_memoryprotect, is_shared, is_tracked>;
    method->init = &GenericMemoryAllocator::AllocSetInit;
    method->reset = &GenericMemoryAllocator::AllocSetReset<enable_memoryprotect, is_shared, is_tracked>;
    method->delete_context = &GenericMemoryAllocator::AllocSetDelete<enable_memoryprotect, is_shared, is_tracked>;
    method->get_chunk_space = &GenericMemoryAllocator::AllocSetGetChunkSpace;
    method->is_empty = &GenericMemoryAllocator::AllocSetIsEmpty;
    method->stats = &GenericMemoryAllocator::AllocSetStats;
#ifdef MEMORY_CONTEXT_CHECKING
    method->check = &GenericMemoryAllocator::AllocSetCheck;
#endif
}

 * This is the virtual function table for Memory Functions
 */
MemoryProtectFuncDef GenericFunctions = {MemoryProtectFunctions::gs_memprot_malloc<MEM_THRD>,
    MemoryProtectFunctions::gs_memprot_free<MEM_THRD>,
    MemoryProtectFunctions::gs_memprot_realloc<MEM_THRD>,
    MemoryProtectFunctions::gs_posix_memalign<MEM_THRD>};

MemoryProtectFuncDef SessionFunctions = {MemoryProtectFunctions::gs_memprot_malloc<MEM_SESS>,
    MemoryProtectFunctions::gs_memprot_free<MEM_SESS>,
    MemoryProtectFunctions::gs_memprot_realloc<MEM_SESS>,
    MemoryProtectFunctions::gs_posix_memalign<MEM_SESS>};

MemoryProtectFuncDef SharedFunctions = {MemoryProtectFunctions::gs_memprot_malloc<MEM_SHRD>,
    MemoryProtectFunctions::gs_memprot_free<MEM_SHRD>,
    MemoryProtectFunctions::gs_memprot_realloc<MEM_SHRD>,
    MemoryProtectFunctions::gs_posix_memalign<MEM_SHRD>};

最核心的片段就是根据value决定模版函数的is_shared和 is_tracked，而这些参数也会影响到palloc时对应的alloc函数的选择。

1
2
3
4
5
6
7
8
9
10
if (is_shared) {
    MemoryContextLock(context);
    func = &SharedFunctions;
} else {
    if (context->session_id > 0)
        func = &SessionFunctions;
    else
        func = &GenericFunctions;
}
/*

gs_memprot_malloc

逻辑比较复杂，涉及到内存追踪、内存预留等，这里不展开讲。内存保护的基本思想是：

  • 设定系统最大内存，追踪每次分配的内存字节
  • 调用malloc前先进行内存充足检查（已经分配的字节小于系统最大内存）
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
/*
 * Memory allocation for sz bytes. If memory quota is enabled, it uses gs_malloc_internal to
 * reserve the chunk and allocate memory.
 */
template <MemType mem_type>
void* MemoryProtectFunctions::gs_memprot_malloc(Size sz, bool needProtect)
{
    if (!t_thrd.utils_cxt.gs_mp_inited)
        return malloc(sz);

    void* ptr = NULL;
    bool status = memTracker_ReserveMem<mem_type>(sz, needProtect);

    if (status == true) {
        ptr = malloc(sz);  //这里分配内存
        if (ptr == NULL) {
            memTracker_ReleaseMem<mem_type>(sz);
            gs_memprot_failed<false>(sz, mem_type);

            return NULL;
        }

        return ptr;
    }

    gs_memprot_failed<true>(sz, mem_type);

    return NULL;
}

PG内存管理篇二

https://xyz.desirer233.fun/2024/09/01/数据库/PostgreSQL/PG内存管理篇二/

作者

Desirer

发布于

2024-09-01

更新于

2024-11-15

许可协议

分类

×