This paper focuses on the virtual memory consumed by the stack of the threads inside a Linux process even when threads are terminated.
#include <stdio.h>
#include <pthread.h>
#define NB_MAX_THD 50
static pthread_t tid[NB_MAX_THD];
static unsigned int idx_create;
static unsigned int idx_terminate;
static void *thd(void *p)
{
pause();
return NULL;
} // thd
static void create_thd(void)
{
int rc;
if (0 == tid[idx_create]) {
rc = pthread_create(&(tid[idx_create]), NULL,
thd, NULL);
if (0 == rc) {
printf("Created thread %p\n", (void
*)(tid[idx_create]));
idx_create = (idx_create + 1) %
NB_MAX_THD;
}
} else {
fprintf(stderr, "Too many threads\n");
}
} // create_thd
static void terminate_thd(void)
{
int rc;
if (tid[idx_terminate]) {
rc = pthread_cancel(tid[idx_terminate]);
if (0 == rc) {
rc = pthread_join(tid[idx_terminate],
NULL);
if (0 == rc) {
printf("Terminated thread
%p\n", (void *)(tid[idx_terminate]));
tid[idx_terminate] = 0;
idx_terminate =
(idx_terminate + 1) % NB_MAX_THD;
} else {
fprintf(stderr, "Unable to
join thread %p\n", (void *)(tid[idx_terminate]));
}
} else {
fprintf(stderr, "Unable to cancel thread
%p\n", (void *)(tid[idx_terminate]));
}
} else {
fprintf(stderr, "No running threads\n");
}
} // terminate_thd
int main(void)
{
int c;
while(1) {
c = fgetc(stdin);
switch(c) {
case 'c' : {
create_thd();
}
break;
case 't' : {
terminate_thd();
}
break;
}
}
return 0;
} // main
|
Here is an example of compilation and execution of the preceding program:
$ gcc thd.c -lpthread $ ./a.out c Created thread 0x7f5e45b72700 c Created thread 0x7f5e45371700 c Created thread 0x7f5e44b70700 t Terminated thread 0x7f5e45b72700 t Terminated thread 0x7f5e45371700 c Created thread 0x7f5e45371700 |
Between each thread creation and termination, we display the memory map of the process (through "cat /proc/<pid>/maps).
Startup [...] 7fff7afcf000-7fff7aff0000 rw-p 00000000 00:000 [stack] [...] After creation of first thread: 0x7f5e45b72700 [...] 7f5e45372000-7f5e45373000 ---p 00000000 00:00 0 7f5e45373000-7f5e45b73000 rw-p 00000000 00:00 0 [stack:3649] [...] After creation of second thread: 0x7f5e45371700 [...] 7f5e44b71000-7f5e44b72000 ---p 00000000 00:00 0 7f5e44b72000-7f5e45372000 rw-p 00000000 00:00 0 [stack:3651] 7f5e45372000-7f5e45373000 ---p 00000000 00:00 0 7f5e45373000-7f5e45b73000 rw-p 00000000 00:00 0 [stack:3649] [...] After creation of third thread: 0x7f5e44b70700 [...] 7f5e44370000-7f5e44371000 ---p 00000000 00:00 0 7f5e44371000-7f5e44b71000 rw-p 00000000 00:00 0 [stack:3653] 7f5e44b71000-7f5e44b72000 ---p 00000000 00:00 0 7f5e44b72000-7f5e45372000 rw-p 00000000 00:00 0 [stack:3651] 7f5e45372000-7f5e45373000 ---p 00000000 00:00 0 7f5e45373000-7f5e45b73000 rw-p 00000000 00:00 0 [stack:3649] [...] After termination of first thread: 0x7f5e45b72700 [...] 7f5e44370000-7f5e44371000 ---p 00000000 00:00 0 7f5e44371000-7f5e44b71000 rw-p 00000000 00:00 0 [stack:3653] 7f5e44b71000-7f5e44b72000 ---p 00000000 00:00 0 7f5e44b72000-7f5e45372000 rw-p 00000000 00:00 0 [stack:3651] 7f5e45372000-7f5e45373000 ---p 00000000 00:00 0 7f5e45373000-7f5e45b73000 rw-p 00000000 00:00 0 [...] After termination of second thread: 0x7f5e45371700 [...] 7f5e44370000-7f5e44371000 ---p 00000000 00:00 0 7f5e44371000-7f5e44b71000 rw-p 00000000 00:00 0 [stack:3653] 7f5e44b71000-7f5e44b72000 ---p 00000000 00:00 0 7f5e44b72000-7f5e45372000 rw-p 00000000 00:00 0 7f5e45372000-7f5e45373000 ---p 00000000 00:00 0 7f5e45373000-7f5e45b73000 rw-p 00000000 00:00 0 [...] After creation of fourth thread: 0x7f5e45371700 [...] 7f5e44370000-7f5e44371000 ---p 00000000 00:00 0 7f5e44371000-7f5e44b71000 rw-p 00000000 00:00 0 [stack:3653] 7f5e44b71000-7f5e44b72000 ---p 00000000 00:00 0 7f5e44b72000-7f5e45372000 rw-p 00000000 00:00 0 [stack:3720] 7f5e45372000-7f5e45373000 ---p 00000000 00:00 0 7f5e45373000-7f5e45b73000 rw-p 00000000 00:00 0 [...] |
For each created thread, two lines appear (memory zones): one for the stack and TCB space and the other one for the red zone on top of the stack used to detect the stack overflows. For example, the first thread has the following allocated memory spaces:
The memory map also points out the fact that the termination of a thread does not free the virtual memory space belonging to it: the stack as well as the red zone are still present after the end of the thread.
Moreover, when threads are created after some terminated threads, the new threads get the memory spaces of the previously terminated thread if any. For example, the fourth thread with TCB 0x7f5e45371700 got the memory spaces of the terminated second thread (hence the same TCB value!).
When we look at the sources of the pthread library in the GLIBC, we can see that a 40 MB stack cache is kept. At process startup, this cache is empty. Then it is filled with the stacks of the terminated threads during the process life. The management of this cache can be found in .../nptl/allocatestack.c.
The size of the cache is configured as follow:
/* Maximum size in kB of cache. */ static size_t stack_cache_maxsize = 40 * 1024 * 1024; /* 40MiBi by default. */ |
The author is an engineer in computer sciences located in France. He can be contacted here.