pete > courses > CS 202 Spring 24 > Lecture 29: The Heap
Lecture 29: The Heap
Goals
- use fgets to receive input
- describe the purpose of the Heap
- use malloc and free to work with the Heap
- define the purpose of the void * type
- use strcpy to copy data into a string
- use memcpy to copy arbitrary data into arbitrary memory
last time, we looked at a program (stack-frame-games.c) that had a weird behavior but one of the (implicit) take-away messages, which I will now make explicit, is that values of local variables should not be used after a function returns
this is because the stack frame in which those local variables are stored is discarded when the function returns
and the memory in which that stack frame lived could be used for another stack frame
if we want a function to create data (which, necessarily, must live in memory) that survives after the function returns, that data must not be a local variable
so where can we put it?
first, a digression
(that will lead us to our answer)
we talked out how to perform output in C, using printf
now let’s talk about input, for which we’re going to use fgets
we’ll start with an example:
first, we declare a character array
since this is a local variable, this array lives in main’s stack frame
then we print out the address where this array is stored (remember that the %p format specifier to printf formats the corresponding parameter like a pointer)
then we print out a prompt
since the prompt doesn’t end in a newline character ('\n'), we have to call fflush to cause the text to appear (don’t worry about why, just trust that we do)
then we call fgets to read whatever the user types
it takes three parameters:
- the first parameter is where to put the data; in this case, we want it to go into that array we allocated a few lines earlier
- we want to read at most 25 characters, which is the size of the array
- and we want to read it from "standard input", which is the official name for data typed at the keyboard
finally, the program prints the string we entered, surrounded by double quotes
we can compile and run it:
$ gcc -o fgets-into-stack fgets-into-stack.c $ ./fgets-into-stack string_array is stored at address 0x7ffcf787a5d0 Enter a string: hello there You entered "hello there "
perhaps the only thing sort of unexpected is that the output at the end has a newline character inside the double quotes: this is because I pressed Enter when I was done with my input, which the computer translated to the newline character, and fgets put that newline character into the array along with all the other characters I typed
if we paused the program after fgets finished but before the final printf and looked at the stack frame for main, it would look something like this:
+------------------------+ \ | | | . . | . . | . . | | | | main's frame +------------------------+ \ | | | | | . . | string_array | . . | | . . | | | e r e \n \0 | | | | h e l l o t h | | | +------------------------+ / | | | | . . | . . | . . | | | | +------------------------+ /
in that diagram, the lower-numbered addresses of memory are at the bottom left
each row represents 8 bytes of memory
here’s another, very similar program:
there is one big difference: instead of a character array, we declare a character pointer, static_string
and we assign to static_string… a static string
before we compile and run it, let’s think carefully about what this is doing
start with the variable declaration:
char *static_string;
this declares a local variable
we can (and should) think of this as allocating memory: it sets aside memory for us to use to store data
first question: how much memory does it set aside?
it sets aside exactly enough memory to store a value of type char *—ie, a character pointer
every pointer has the same size, the word size of the machine: on ARM32, the word size is 32 bits (hence the name of the ISA); on modern x86 chips, the word size is 64 bits (8 bytes)
since my machine runs a modern x86 chip, this declaration will set aside 8 bytes of memory
where?
it’s a local variable, so it must be in the stack frame for main
this gives us a diagram like this (remember: each row is 8 bytes):
+------------------------+ \ | | | . . | . . | . . | | | | main's frame +------------------------+ \ | | | | static_string | +------------------------+ / | | | | . . | . . | . . | | | | +------------------------+ /
now, the assignment
before we look at the actual assignment used in the code, consider this alternative:
static_string = 12;
if we saw this line of code, we’d conclude (correctly) that the result is that the value 12 would be stored in main’s stack frame in the 8-byte chunk allocated to static_string
the real line of code follows a very similar form:
static_string = "This is a static string.";
so it would be reasonable to assume that it will also result in some value being put in main’s stack frame in the 8-byte chunk allocated to static_string
but what data goes in there?
one hint to this is in the type of static_string: it’s a char *, so the data that goes into that 8-byte chunk ought to be an address
looking at the code, it seems like it should be something that identifies the string "This is a static string."
this implies that the bytes representing that string are in memory, too, which makes a lot of sense because it’s data the program wants to operate on, and all such data lives in memory
but where does this string live in memory?
so far, we’ve seen two kinds of things in memory: instructions and the stack
this is neither of those
the address of this string will be stored in the stack after the assignment is done, but where do the bytes of the string itself live?
the answer is… somewhere else
the details of the somewhere else aren’t important, just that they are indeed somewhere else
+------------------------+ \
| | |
. . |
. . |
. . |
| | | main's frame
+------------------------+ \ |
+--| | | static_string |
| +------------------------+ / |
| | | |
| . . |
| . . |
| . . |
| | | |
| +------------------------+ /
|
| ...
|
| | \0 |
| | s t r i n g . \n |
| | a s t a t i c |
+->| T h i s i s |
here, the arrow represents the fact that the value stored in static_string is the address where the string itself is stored
and that place is outside The Stack
let’s compile and run it
$ gcc -o fgets-into-static-string fgets-into-static-string.c $ ./fgets-into-static-string static_string is stored at address 0x7ffdd2cb3540 static_string points to address 0x55cff42d6008 Enter a string: hello there Segmentation fault (core dumped)
it crashes!
why?
because, while we can read from the memory in which the string is stored, we are not allowed to write to it
recall that a segmentation fault happens when we try to perform a memory operation that is not allowed: fgets is trying to write to that memory
the fact that we can’t write to that memory is a fatal shortcoming of the second example
the first example echoes a shortcoming we saw last time: the memory in which local variables are stored is no longer usable after a function returns
this is because its stack frame is discarded
and, when another function is called, that same memory could be used by another function’s frame
we need a way to allocate memory such that the memory isn’t reclaimed when the function returns
my guess is you’ve seen such behavior in Java or Python: a function creates, eg, an object and then passes that object back to the caller; the object still exists even after the callee returns
we could accomplish something like this by having the caller create a local variable to hold the value
but that requires us to know at compile-time how much memory the program will need, which is not a reasonable thing to ask
especially if memory needs of a program change over time (unlike the vast majority of programs you’ve written, many programs need to stay running for a very long time: days, weeks, months)
so we need a way to dynamically allocate memory: that is, allocate chunks of memory of arbitrary size at arbitrary points during the executing of a program
the memory from which we carve out these chunks needs to be separate from instructions and from The Stack, because both of those already have proscribed purposes and behaviors
so we have a new area of memory called The Heap
we use the function malloc and free to access the heap
malloc asks for a chunk of memory and free relinquishes a chunk of memory previously granted by malloc
recall that in the first Linux lecture, I said that many command-line programs are documented in manual pages that can themselves be accessed from the command-line
the same goes for library functions like malloc and free
so, to read how malloc works (eg, what parameters it takes, what it returns, what header file it’s defined in, how it behaves) you can type
$ man malloc
(as before, arrow keys scroll and 'q' quits)
we can see at the top that, to use either malloc or free we need to #include <stdlib.h>
we also see that the function definitions (or prototypes) are as follows:
void *malloc(size_t size); void free(void *ptr);
(ignore calloc and realloc for this course)
just looking at the prototype for malloc, it seems logical to assume that its parameter is the size of the chunk of memory we’re requesting
so if we want 10 bytes, we’ll call malloc(10) and if we want 10000 bytes, we’ll call malloc(10000)
but what about that return value?
it makes sense that malloc would return a pointer, because that’s the most convenient way to identify memory
(note that, if malloc succeeds, the memory returned will be in a single chunk: if you ask for 2000 bytes, it will not give you a pair of separate, 1000-byte chunks)
but why a void pointer? what even is void anyway?
consider that malloc has no idea how you’re going to use this memory
you could want to store integers there, in which case it should be an int *
or you could want to store characters, in which case it should be a char *
or you could store instances of a custom data type, in which case it should be something like a struct widget *
to be as flexible as possible, then, malloc returns a void *
which means "a pointer to an unspecified type"
in a sense, the void type is both nothing and everything at once
so you can always assign a value of type void * to a variable of any pointer type
here’s how we would re-write our program from before using malloc and free:
honestly not that exciting
malloc returns a pointer to 25 bytes of memory
we pass that pointer as parameter to fgets
(at the end, we free the memory because we want to be responsible programmers; strictly speaking, this isn’t necessary, as all memory is reclaimed when a program ends, but it’s a good habit to get into)
and everything works as we would like:
$ gcc -o fgets-into-heap fgets-into-heap.c $ ./fgets-into-heap string_in_heap is stored at address 0x555c4d7882a0 Enter a string: hello there You entered "hello there "
here’s another example of malloc and free in action that demonstrates how the allocation is still usable after the function that created it returns: malloc.c
in foo, we declare a character pointer, called s
then we ask for 6 bytes on the heap
and we put the address of those 6 bytes into the variable s
then we can modify the contents of that memory using the dereference operator, as we’ve seen before
(note that we DO have to manually put in the null byte that C expects at the end of strings)
(note also that we have to allocate enough memory to make room for the null byte)
this is the same as all the other pointer stuff we’ve done
with the exception that this memory exists outside the stack, so it is not tied to any specific stack frame
then we can return it and code in main (in this case printf) can access that memory just fine
note that the address of this memory is saved in the variable t, whose value is in main’s stack frame
and the value of t points to where the allocation in The Heap
further on the topic of pointers and memory… consider this code:
char *s = malloc(6); s = "hello";
will this cause the string "hello" to be copied into the memory granted by malloc?
conceptually, we are assigning a new value to the variable s
s is a pointer
so we are causing s to point to a new place in memory (ie, the location of the string "hello")
we are not copying those characters into the existing place where s points
so, short of copying bytes one at a time as in the program above, how can we copy data into memory?
enter the function strcpy (short for "string copy")
here’s an example: string.c
it will copy the string "hello" into the memory pointed to by s
note that strcpy will copy the null byte at the end of the string (read the manpage to see for yourself)
strcpy will copy from the second parameter into the first parameter
when will it stop copying?
when it reaches a null byte in the second parameter
what if we modified the example above to copy a super-long string into s?
eg,
strcpy(s, "hellohellohello");
we only have 6 bytes at s; will this fail?
nope, it will happily succeed, overwriting the memory adjacent to the six bytes of s
this could lead to bad, bad things happening
the better option is to use strncpy instead, which takes a parameter that specifies the number of bytes to copy
eg,
strncpy(s, "hellohellohello", 6);
this will copy at most 6 bytes from the second parameter into the first parameter
(note that, however, according to the strncpy manpage, if there is no null byte in the first 6 bytes of the source string, it will not put a null byte in the destination!)
what if I change the above code, to embed a null byte in the source operand:
strncpy(s, "he\0lo", 6);
recall that strings in C are terminated by the null byte
so only the first three bytes will be copied!
but what if we want to copy data that might have null bytes in the middle?
we can’t use strncpy (and, incidentally, these would not be strings, they’d just be arbitrary data)
for this, we use the function memcpy
which behaves pretty much the same as strncpy except it doesn’t stop when it encounters a null byte
put another way, it always copies exactly the number of bytes you request
memcpy(s, "he\0lo\0", 6);
(but memcpy will not add the null-termination for you, which is why I added the null byte at the end of the string there)
following on from the previous example of malfeasance, consider this: stack-games.c
s is a 10-character array on the stack
and we copy 32 bytes of data into it
the first 10 of those bytes will go in the memory set aside for s
but where do the final 22 bytes go?
in the memory immediately following s!
what’s immediately following s?
things like the caller’s frame pointer… and the return address
so if we ovewrite the return address with garbage, what happens when the function returns?
it will attempt to return to a garbage address, almost certainly resulting in a segfault
but what if we’re smart about the data with which we overwrite the return address?
we can return wherever we want