pete > courses > CS 202 Fall 26 > Lecture 23: memory layout and I/O in C

Lecture 23: memory layout and I/O in C

Goals

over the past few lectures, we’ve looked at a lot of ARM32 assembly code and the C code that it corresponds to

and much of that assembly code dealt with loading values from memory into registers or storing values from registers into memory

for structs and arrays in particular, the pointful assembly code was all about where their constituent parts were stored in memory

therefore, to solidify those concepts to you and to expand your knowledge of C, we’re going to write and run some C programs that use the constructs we’ve already seen

but we’re going to incorporate behavior that gets the program itself to tell us where these things are stored

just as before, I’m going to share some very simple C programs that demonstrate the concepts

unlike before, however, we’re not going to look at the assembly

instead, I’m going to compile the C code into a runnable program, run it, and see what it prints out

(note that, because my laptop has an x86 chip, the code produced by the compiler will be in x86, not ARM)

to accomplish this, we need to be able to perform output

initially demonstrated in printf.c

the C function usually used to print text to the screen is called printf, where the print part is hopefully self-explanatory and the f means "formatted" (we’ll see why in a bit)

the program above demonstrates this

this is the first time we’ve seen a function call in C

if you’re used to Java, it’s exactly the same: the name of the function, the comma-separated list of parameters enclosed in parentheses, then a semi-colon

in this case, its single parameter is the string literal to output to the screen

by "string literal" I mean it is a value that evaluates to itself: it’s not like a variable that evaluates to whatever has been previously assigned to it

so running this program should cause "Hello world." to be printed to the screen

we compile it:

$ gcc -o printf printf.c

the $ represents the shell prompt: that is, the text the shell prints to indicate to the user that it’s ready for input

the program we’re running is now using gcc (and not arm-none-eabi-gcc) because we want to produce machine code that will run on the native hardware—ie, the x86 chip inside my computer

the "printf.c" part makes sense: it tells gcc which file we want to compile

but the "-o printf" part is new: here, we’re telling gcc what filename it should use for the resulting executable

this has to come in a pair: the "-o" indicates "the next word will be the name of the output file"

upon pressing enter, we see nothing printed to the screen, which is common for UNIX programs when they succeed at a task that doesn’t require printing to the screen

and we run it:

$ ./printf
Hello world.$

here, we had to explicitly say "run the program named printf in the current directory"

recall that "." means "the current directory"

this is to make it very clear that we don’t mean the printf program that might be installed in the main system directories

note the "$" right after the output string: this is the next shell prompt

because the printf program finished its job, the shell noticed, and so printed the next prompt

it would have been more neighborly for the shell prompt to appear on a different line

for that, we have to change the string that gets printed: we have to print a newline character, which gives exactly the effect we want: the next text will start on the next line

to do so, we add the characters "\n" to the end of the string literal being printed:

    printf("Hello world.\n");

how do we interpret this?

it turns out that some characters can’t be directly represented in a C string

one of these is the newline character, which causes the cursor to move to the next line

because we do sometimes want to print a newline character, we need some other way to specify it inside a C string

hence, backslash escaping

a backslash in a string indicates that the following character should be interpreted using special rules

in this case, because it’s an "n", it indicates that a newline character should be inserted

there are others: "\t" indicates a tab character,"\r" indicates a carriage return, which causes the cursor to move to the beginning of the line (but does not move to the following line, which "\n" does do)

the manpage for ASCII (run man ascii from the shell) shows the rest of the available backslash-escape sequences

we recompile and re-run and the output is much nicer;

$ gcc -o printf printf.c
$ ./printf
Hello world.
$

one last thing to point out about that program

specifically, this line:

#include <stdio.h>

absent any other information, the C compiler knows nothing about printf: most specifically, the number and types of its parameters and the type of its return value

these important bits of information are contained within the file stdio.h, which this line of code incorporates in the program

the generic term for the kind of file that contains this kind of information is a header file and they almost always end in .h

if we comment out this line of code and try to compile, here’s what we see:

printf.c: In function ‘main’:
printf.c:9:5: error: implicit declaration of function ‘printf’ [-Wimplicit-function-declaration]
    9 |     printf("Hello world.\n");
      |     ^~~~~~
printf.c:1:1: note: include ‘<stdio.h>’ or provide a declaration of ‘printf’
  +++ |+#include <stdio.h>
    1 | /*
printf.c:9:5: warning: incompatible implicit declaration of built-in function ‘printf’ [-Wbuiltin-declaration-mismatch]
    9 |     printf("Hello world.\n");
      |     ^~~~~~
printf.c:9:5: note: include ‘<stdio.h>’ or provide a declaration of ‘printf’

the first warning translates to "you’re calling printf but I have no idea what this printf thing even is"

the second warning translates to "since you didn’t tell me anything about printf, I assumed it took a single int as parameter, but you’re giving it a string as its parameter and that doesn’t match my assumption"

and the notes helpfully tell us how to fix it

the next program is only a little more complicated: print-variable.c

we declare a variable x and then print its value, now taking advantage of the formatting features of printf

printf works by pairing up "format specifers" (anything starting with a % symbol, in this case "%d") with parameters

so the first parameter, x, will be formatted using the "%d" rules, which are to interpret it as a decimal number and print that

(we’ll see examples of multiple parameters soon)

and now in action:

$ gcc -o print-variable print-variable.c
$ ./print-variable
x is 12

this behavior is likely expected and feels good based on your past programming experience

but based on what we’ve learned in the preceding weeks in this course, we can (and should) be more explicit about what’s going on

let’s start with this line in the program:

    int x = 12;

it’s doing two things

first, the int x part: - at a high level, we’re introducing a new variable called x, which is going to store a signed integer - at a lower level, we need to reserve ("allocate") enough memory to store a 32-bit value represented in two’s complement, and associate the name x with it, so that any changes to x will modify that reserved memory and any uses of the variable x will read from that reserved memory

(the fact that it’s a 32-bit value and not something else is because The Rules of C state that values of type int are 32-bit unsigned integers… with some exceptions that are not worth getting into here)

the second thing this does is store the value 12 in that memory, encoded as two’s complement

so if we went deep inside our computer, we’d see the following bit pattern in memory… somewhere

0000 0000 0000 0000 0000 0000 0000 1100

otherwise known as

0x0000000c

this segues nicely to talking more about what the printf function itself does

in this program, because we use the "%d" format specifier, it’s going to take the value of x (which, again, is two’s complement) and output a sequence of ASCII characters which, when read by a human, will communicate the number 12

this is shown in the output above

we could ask the printf function to format it differently

for example, if we change that line to this:

    printf("x is %08x\n", x);

the output will format the value of x as 8 digits of hexadecimal, padded with zeroes

to break down the format specifier ("%08x"): - it always starts with "%" - the "0" means "pad with zeroes" - the "8" means "print 8 characters, padded if necessary" - the "x" means "format as hex"

here we go:

$ gcc -o print-variable print-variable.c
$ ./print-variable
x is 0000000c

(note that it didn’t print a "0x" prefix: this is because the "%x" format specifier doesn’t do that; if we wanted it, we’d have to add it ourselves)

I started this whole business by foreshadowing the idea of printing out where things are stored in memory

print-address.c

two new things here: "%p" and "&x"

the "%p" format specifier causes the parameter to be formatted like a memory address

the "&" operator returns the memory address at which the operand is stored

thus "&x" is the address where x is stored

in action:

$ gcc -o print-address print-address.c 
$ ./print-address 
x is stored at 0x7ffead94132c

(if you run any of the programs from this lecture multiple times, chances are very good that you’ll get different addresses: this is expected; we’ll see why in the next couple lectures—the important thing is the values of these addresses relative to one another within a single run of the program)

note that the address is really big: it’s much bigger than 32 bits

this is because my computer (and likely yours, if you run this code yourself) uses a 64-bit word size and therefore memory addresses are much larger

remember that we can (and should) think of memory as a massive array, where each element stores a single byte of data

since x has type int, it needs 4 bytes of memory

the address printed out is the first of those bytes; the rest of the value is stored at the next 3 larger addresses

so, given the output above, we can conclude that all four of these addresses are used to store x:

next up: new data types in C!

C doesn’t actually have many built-in data types, so the ones we’ll see end up being variations on integers

the idea is that C is very close to assembly (which doesn’t have types at all) and thus if you want to build complex, expressive data types, you use those building blocks

thus, I give you:

many-variables.c

a char is 8 bits

a uint16_t is a 16-bit unsigned integer

a int32_t is a 32-bit signed integer (2’s complement, naturally)

how big is an int ?

you might be tempted to say 32 bits, because that’s how big they are on ARM32

and historically, this is the correct connection: an "int" (without specifying how large) would be the same size as the word size of the machine (convenient because it means a single int takes up a single register)

HOWEVER

because 32-bit machines were so prevalent for so long, and because we still use a bunch of software written during those times, an integer on modern machines is usually assumed to be 32 bits

that "usually" is important, because you can’t always depend on it

consider the implications of different sizes for int: if I try to put the value 2 billion into an int on ARM32, it’ll work, whereas it will overflow on a machine that uses 16 bits for an int

this is BAD: unexpected or unreliable behavior is a recipe for disaster

therefore, it’s a good idea to explicitly specify the number of bits to use

which is where the stdint.h header file comes in

it defines the uint16_t and int32_t types demonstrated above, along with others

signed and unsigned variants for 8, 16, 32, and 64 bits

(you can see this file for yourself on a Linux system: /usr/include/stdint.h)

but let’s compile and run this program

$ gcc -o many-variables many-variables.c 
$ ./many-variables 
x is stored at 0x7ffc9aa25bb0
c is stored at 0x7ffc9aa25bad
i is stored at 0x7ffc9aa25bae
y is stored at 0x7ffc9aa25bb4

now we begin to see something interesting

here’s a diagram of memory: the first column is the address and the second describes what’s stored there

(note the lowest-numbered address is at the bottom)

0x7ffc9aa25bb7  \
0x7ffc9aa25bb6  |   y
0x7ffc9aa25bb5  |
0x7ffc9aa25bb4  /
0x7ffc9aa25bb3  \
0x7ffc9aa25bb2  |   x
0x7ffc9aa25bb1  |
0x7ffc9aa25bb0  /
0x7ffc9aa25baf  \   i
0x7ffc9aa25bae  /
0x7ffc9aa25bad  >   c

this is pretty much as we might expect, except for one weird observation: the order of variables in memory is different than the order in which they were declared in the program!

the reason for this goes back down layers of abstraction

the circuitry that connects memory to the register file is constructed in such a way that it’s much more efficient transferring multi-byte values when those values begin on an address that is an even multiple of their size

for instance, a 4-byte value (like an int) should start at an address that is an integer multiple of 4

a 2-byte value (like a `uint16_t) should start at an address that is an integer multiple of 2

in the output above, the compiler chose not to allocate memory in the order in which they were declared, but rather in an order that a) caused all values to be aligned and b) didn’t leave unused gaps between

this is acceptable because there is no relation between those variables, and so their relative locations in memory don’t matter

which brings us to structs, which do define a relation between fields

struct-addresses.c

define a type "struct foo" at the top, with the fields a, b, c, d, and e

then, in main, we declare a variable of that type, which we call bar, and print out the locations of the fields

$ gcc -o struct-addresses struct-addresses.c 
$ ./struct-addresses 
bar.a is at 0x7fff781dab60
bar.b is at 0x7fff781dab62
bar.c is at 0x7fff781dab64
bar.d is at 0x7fff781dab68
bar.e is at 0x7fff781dab6c

here another picture will come in handy

0x7fff781dab6f  \
0x7fff781dab6e  | bar.e
0x7fff781dab6d  |
0x7fff781dab6c  /
0x7fff781dab6b  \
0x7fff781dab6a  | empty!
0x7fff781dab69  /
0x7fff781dab68  > bar.d
0x7fff781dab67  \
0x7fff781dab66  | bar.c
0x7fff781dab65  |
0x7fff781dab64  /
0x7fff781dab63  \ bar.b
0x7fff781dab62  /
0x7fff781dab61  > empty
0x7fff781dab60  > bar.a

note that here the compiler has retained the order but inserted empty space to maintain alignment!

we also see that the "first" item in the struct (ie, field a) is lowest in memory, which matches what we saw in the ARM32 assembly that worked with structs

we can ask the compiler to take out the holes by adding an annotation to the struct definition

packed-struct.c

$ gcc -o packed-struct packed-struct.c 
$ ./packed-struct 
bar.a is at 0x7ffc6f3cf020
bar.b is at 0x7ffc6f3cf021
bar.c is at 0x7ffc6f3cf023
bar.d is at 0x7ffc6f3cf027
bar.e is at 0x7ffc6f3cf028

it has retained the ordering, but ditched the gaps

draw the memory diagram to convince yourself this is the case

given that unpacked structs are likely more efficient, why would you ever want a packed struct?

sometimes data formats have been defined where the fields are packed tightly, and it’s convenient to represent values in those formats as structs

though many formats, notably the protocols that make the Internet work, were engineered with 32-bit alignment in mind

let’s look at arrays now

here’s a program very similar to one I showed last lecture, except that it prints out the address of each array element instead of assigning to it

array-addresses.c

now we see printf with two parameters: the first format specifier (ie, "%d") is matched up with the first parameter after the string (ie, i); the second format specifier ("%p") is matched up with the second parameter ("&(a[i])")

$ gcc -o array-addresses array-addresses.c 
$ ./array-addresses 
a[0] is stored at 0x7fff664acca0
a[1] is stored at 0x7fff664acca4
a[2] is stored at 0x7fff664acca8
a[3] is stored at 0x7fff664accac
a[4] is stored at 0x7fff664accb0
a[5] is stored at 0x7fff664accb4
a[6] is stored at 0x7fff664accb8
a[7] is stored at 0x7fff664accbc
a[8] is stored at 0x7fff664accc0
a[9] is stored at 0x7fff664accc4

given our assembly-spelunking from Monday, this is pretty much expected output

a[0] is stored at the lowest address, and the rest of the array is stored at 4-byte increments above (ie, higher than) that

they’re at 4-byte increments because the array was declared to store values of type uint32_t, which is 4 bytes

change the array to store values of type uint8_t and see how the output changes

also try uint16_t

last topic: multi-dimensional arrays

check out the program first:

multi-dimensional-array.c

we declare a to be a 10x20 array and then iterate through it using a nested loop

but here’s the question: where do these things sit in memory?

recall that memory is, effectively, a one-dimensional array, where addresses are the indices, and each cell holds a single byte

how can we build a 2-dimensional array out of this?

run the program and find out!

buuuuut, it produces way too much data and is thus unenlightening, so I’m not going to reproduce it here

how can we make it useful?

use a smaller array!

change the dimensions to 2 and 3

but then we have to also change the loop invariants to reflect this new array size

this may sound like a simple operation, and in a small program like this, it is

but in a larger program, remembering to perform that sort of modification is both easy to do and potentiall disastrous (we’ll see how later)

a much better approach is to to declare the dimensions as constants at the top of the program, as illustrated here

multi-dimensional-array-with-constants.c

you might ask why not just use variables

the first reason is that such a use would violate the abstraction variables give us: variables are meant to be changed while the program is running, this is purely a compile-time decision

secondly, the compiler can perform certain optimizations when it knows values never change

so here’s what it tells us

$ gcc -o multi-dimensional-array-with-constants multi-dimensional-array-with-constants.c 
$ ./multi-dimensional-array-with-constants 
a[0][0] is stored at 0x7ffcd23e1f50
a[0][1] is stored at 0x7ffcd23e1f54
a[0][2] is stored at 0x7ffcd23e1f58
a[1][0] is stored at 0x7ffcd23e1f5c
a[1][1] is stored at 0x7ffcd23e1f60
a[1][2] is stored at 0x7ffcd23e1f64

as it turns out, the two rows of the array are laid out one after the other in memory

draw the picture to convince yourself this is the case

this extrapolates predictably to arrays of higher dimensions

Last modified: