pete > courses > CS 202 Spring 24 > Lecture 23: memory layout and I/O in C

Lecture 23: memory layout and I/O in C

Goals

over the past few lectures, we’ve looked at a lot of assembly code

and much of that assembly code dealt with loading values from memory into registers or storing values from registers into memory

for structs and arrays in particular, the pointful assembly code was all about where their constituent parts were stored in memory

therefore, to solidify those concepts to you and to expand your knowledge of C, we’re going to write and run some C programs that use the constructs we’ve already seen

but we’re going to incorporate behavior that gets the program itself to tell us where these things are stored

which means we need to be able to perform output

initially demonstrated in printf.c

the C function usually used to print text to the screen is called printf, where the print part is hopefully self-explanatory and the f means "formatted" (we’ll see why in a bit)

the program above demonstrates this

this is the first time we’ve seen a function call in C

if you’re used to Java, it’s exactly the same: the name of the function, the comma-separated list of parameters enclosed in parentheses, then a semi-colon

in this case, its single parameter is the string literal to output to the screen

by "string literal" I mean it is a value that evaluates to itself: it’s not like a variable that evaluates to whateve has been previously assigned to it

so running this program should cause "Hello world.\n" to be printed to the screen

but what’s that "\n" thing?

it turns out that some characters can’t be directly represented in a C string

one of these is the newline character, which causes the cursor to move to the next line

because we do sometimes want to print a newline character, we need some other way to specify it inside a C string

hence, backslash escaping

a backslash in a string indicates that the following character should be interpreted using special rules

in this case, because it’s an "n", it indicates that a newline character should be inserted

there are others: "\t" indicates a tab character,"\r" indicates a carriage return, which causes the cursor to move to the beginning of the line (but does not move to the following line, which "\n" does do)

so if we compile and run this program, then we should see the string "Hello world." all on its own line, followed by the next Linux shell prompt

since I want to run this program on my computer, which uses the x86 ISA, I compile it with "plain" gcc instead of the ARM-specific version of gcc you’ve seen me use in the past

$ gcc -o printf printf.c

the $ represents the shell prompt: that is, the text the shell prints to indicate to the user that it’s ready for input

the "printf.c" part makes sense: it tells gcc which file we want to compile

but the "-o printf" part is new: here, we’re telling gcc what filename it should use for the resulting executable

this has to come in a pair: the "-o" indicates "the next word will be the name of the output file"

now, to run it:

$ ./printf
Hello world.
$

(I’ve shown both the prompt at which I entered and ran the command as well as the following prompt to show that the program finished; I won’t do this in the future, I just wanted to show it for completeness here)

the "./" part is new: when we ran programs previously, we just gave the name of the program (eg, gcc, ls)

that’s because many programs on Linux are installed in well-known directories, which the system knows to search whenever you run programs

our printf program isn’t in one of those directories, so we have to explicitly tell it to look in the current directory, which is "."

(recall that ".." is the parent directory; same idea here)

one last thing to point out about that program

specifically, this line:

#include <stdio.h>

absent any other information, the C compiler knows nothing about printf: most specifically, the number and types of its parameters and the type of its return value

these important bits of information are contained within the file stdio.h, which this line of code incorporates in the program

the generic term for the kind of file that contains this kind of information is a header file and they almost always end in .h

if we remove this line of code and try to compile, here’s what we see:

$ gcc -o printf printf.c
rintf.c: In function ‘main’:
printf.c:9:5: warning: implicit declaration of function ‘printf’ [-Wimplicit-function-declaration]
    9 |     printf("Hello world.");
      |     ^~~~~~
printf.c:9:5: warning: incompatible implicit declaration of built-in function ‘printf’
printf.c:1:1: note: include ‘<stdio.h>’ or provide a declaration of ‘printf’
  +++ |+#include <stdio.h>
    1 | /*

the first warning translates to "you’re calling printf but I have no idea what this printf thing even is"

the second warning translates to "since you didn’t tell me anything about printf, I assumed it took a single int as parameter, but you’re giving it a string as its parameter and that doesn’t match my assumption"

and the note at the bottom helpfully tells us how to fix it

the next program is only a little more complicated: print-variable.c

we declare a variable x and then print its value, now taking advantage of the formatting features of printf

printf works by pairing up "format specifers" (anything starting with a % symbol, in this case "%d") with parameters

so the first parameter, x, will be formatted using the "%d" rules, which are to interpret it as a decimal number and print that

(we’ll see examples of multiple parameters soon)

and now in action:

$ gcc -o print-variable print-variable.c
$ ./print-variable
x is 12

I started this whole business by foreshadowing the idea of printing out where things are stored in memory

print-address.c

two new things here: "%p" and "&x"

the "%p" format specifier causes the parameter to be formatted like a memory address

the "&" operator returns the memory address at which the operand is stored

thus "&x" is the address where x is stored

in action:

$ gcc -o print-address print-address.c 
$ ./print-address 
x is stored at 0x7ffead94132c

(if you run any of the programs from this lecture multiple times, chances are very good that you’ll get different addresses: this is expected; we’ll see why in the next couple lectures—the important thing is the values of these addresses relative to one another within a single run of the program)

note that the address is really big: it’s much bigger than 32 bits

this is because my computer (and likely yours, if you run this code yourself) uses a 64-bit word size and therefore memory addresses are much larger

next up: new data types in C!

C doesn’t actually have many built-in data types, so the ones we’ll see end up being variations on integers

the idea is that C is very close to assembly (which doesn’t have types at all) and thus if you want to build complex, expressive data types, you use those building blocks

thus, I give you:

many-variables.c

a char is 8 bits

a uint16_t is a 16-bit unsigned integer

a int32_t is a 32-bit signed integer (2’s complement, naturally)

how big is an int ?

you might be tempted to say 32 bits, because that’s how big they are on ARM32

and historically, this is the correct connection: an "int" (without specifying how large) would be the same size as the word size of the machine (convenient because it means a single int takes up a single register)

HOWEVER

because 32-bit machines were so prevalent for so long, and because we still use a bunch of software written during those times, an integer on modern machines is usually assumed to be 32 bits

that "usually" is important, because you can’t always depend on it

consider the implications of different sizes for int: if I try to put the value 3 billion into an int on ARM32, it’ll work, whereas it will overflow on a machine that uses 16 bits for an int

this is BAD: unexpected or unreliable behavior is a recipe for disaster

therefore, it’s a good idea to explicitly specify the number of bits to use

which is where the stdint.h header file comes in

it defines the uint16_t and int32_t types demonstrated above, along with others

signed and unsigned variants for 8, 16, 32, and 64 bits

(you can see this file for yourself on a Linux system: /usr/include/stdint.h)

but let’s compile and run this program

$ gcc -o many-variables many-variables.c 
$ ./many-variables 
x is stored at 0x7ffc9aa25bb0
c is stored at 0x7ffc9aa25bad
i is stored at 0x7ffc9aa25bae
y is stored at 0x7ffc9aa25bb4

now we begin to see something interesting

here’s a diagram of memory: the first column is the address and the second describes what’s stored there

(note the lowest-numbered address is at the bottom)

0x7ffc9aa25bb7  \
0x7ffc9aa25bb6  |   y
0x7ffc9aa25bb5  |
0x7ffc9aa25bb4  /
0x7ffc9aa25bb3  \
0x7ffc9aa25bb2  |   x
0x7ffc9aa25bb1  |
0x7ffc9aa25bb0  /
0x7ffc9aa25baf  \   i
0x7ffc9aa25bae  /
0x7ffc9aa25bad  >   c

this is pretty much as we might expect, except for one weird observation: the order of variables in memory is different than the order in which they were declared in the program!

(we’ll see why in a bit)

let’s look at arrays now

here’s a program very similar to one I showed last time, except that it prints out the address of each array element instead of assigning to it

array-addresses.c

now we see printf with two parameters: the first format specifier (ie, "%d") is matched up with the first parameter after the string (ie, i); the second format specifier ("%p") is matched up with the second parameter ("&(a[i])")

$ gcc -o array-addresses array-addresses.c 
$ ./array-addresses 
a[0] is stored at 0x7fff664acca0
a[1] is stored at 0x7fff664acca4
a[2] is stored at 0x7fff664acca8
a[3] is stored at 0x7fff664accac
a[4] is stored at 0x7fff664accb0
a[5] is stored at 0x7fff664accb4
a[6] is stored at 0x7fff664accb8
a[7] is stored at 0x7fff664accbc
a[8] is stored at 0x7fff664accc0
a[9] is stored at 0x7fff664accc4

given our assembly-spelunking from Monday, this is pretty much expected output

a[0] is stored at the lowest address, and the rest of the array is stored at 4-byte increments above (ie, higher than) that

they’re at 4-byte increments because the array was declared to store values of type uint32_t, which is 4 bytes

change the array to store values of type uint8_t and see how the output changes

next up: structs

struct-addresses.c

define a type "struct foo" at the top, with the fields a, b, c, d, and e

then, in main, we declare a variable of that type, which we call bar, and print out the locations of the fields

$ gcc -o struct-addresses struct-addresses.c 
$ ./struct-addresses 
bar.a is at 0x7fff781dab60
bar.b is at 0x7fff781dab62
bar.c is at 0x7fff781dab64
bar.d is at 0x7fff781dab68
bar.e is at 0x7fff781dab6c

here another picture will come in handy

0x7fff781dab6f  \
0x7fff781dab6e  | bar.e
0x7fff781dab6d  |
0x7fff781dab6c  /
0x7fff781dab6b  \
0x7fff781dab6a  | empty!
0x7fff781dab69  /
0x7fff781dab68  > bar.d
0x7fff781dab67  \
0x7fff781dab66  | bar.c
0x7fff781dab65  |
0x7fff781dab64  /
0x7fff781dab63  \ bar.b
0x7fff781dab62  /
0x7fff781dab61  > empty
0x7fff781dab60  > bar.a

this mostly makes sense… except for the spaces that appear to be empty

and it’s true: nothing is stored there

why waste space like this?

because operations like loads and stores are more efficient when they are aligned

imagine that, instead of a 1-dimensional array of bytes, memory is a 2-dimensional array (ie, a grid) where each row is 4 bytes

a single 4-byte value could either take up a whole row unto itself or it could span two adjacent rows

when a value takes up a whole row, it is called aligned; when it does not, it is called unaligned

loads and stores on aligned values are much more efficient than on unaligned values

(and the same goes for 16-bit/2-byte values on a grid that’s 2 bytes wide)

therefore, the compiler has inserted those empty places precisely to make sure all the fields of the struct are aligned, and are thus more efficient to access

(this is in contrast to the variables in the many-variables program above: the compiler rearranged the variables, but NOT the fields of the struct)

we also see that the "first" item in the struct (ie, field a) is lowest in memory, just like the beginning of the array was lowest in memory

we can actually ask the compiler to take out the holes by adding an annotation to the struct definition

packed-struct.c

$ gcc -o packed-struct packed-struct.c 
$ ./packed-struct 
bar.a is at 0x7ffc6f3cf020
bar.b is at 0x7ffc6f3cf021
bar.c is at 0x7ffc6f3cf023
bar.d is at 0x7ffc6f3cf027
bar.e is at 0x7ffc6f3cf028

draw the memory diagram to convince yourself this is the case

given that unpacked structs are likely more efficient, why would you ever want a packed struct?

sometimes data formats have been defined where the fields are packed tightly, and it’s convenient to represent values in those formats as structs

though many formats, notably the protocols that make the Internet work, were engineered with 32-bit alignment in mind

last topic: multi-dimensional arrays

check out the program first:

multi-dimensional-array.c

we declare a to be a 10x20 array and then iterate through it using a nested loop

but here’s the question: where do these things sit in memory?

recall that memory is, effectively, a one-dimensional array, where addresses are the indices, and each cell holds a single byte

how can we build a 2-dimensional array out of this?

run the program and find out!

buuuuut, it produces way too much data and is thus unenlightening, so I’m not going to reproduce it here

how can we make it useful?

use a smaller array!

change the dimensions to 2 and 3

but then we have to also change the loop invariants to reflect this new array size

this may sound like a simple operation, and in a small program like this, it is

but in a larger program, remembering to perform that sort of modification is both easy to do and potentiall disastrous (we’ll see how later)

a much better approach is to to declare the dimensions as constants at the top of the program, as illustrated here

multi-dimensional-array-with-constants.c

you might ask why not just use variables

the first reason is that such a use would violate the abstraction variables give us: variables are meant to be changed while the program is running, this is purely a compile-time decision

secondly, the compiler can perform certain optimizations when it knows values never change

so here’s what it tells us

$ gcc -o multi-dimensional-array-with-constants multi-dimensional-array-with-constants.c 
$ ./multi-dimensional-array-with-constants 
a[0][0] is stored at 0x7ffcd23e1f50
a[0][1] is stored at 0x7ffcd23e1f54
a[0][2] is stored at 0x7ffcd23e1f58
a[1][0] is stored at 0x7ffcd23e1f5c
a[1][1] is stored at 0x7ffcd23e1f60
a[1][2] is stored at 0x7ffcd23e1f64

as it turns out, the two rows of the array are laid out one after the other in memory

draw the picture to convince yourself this is the case

this extrapolates predictably to arrays of higher dimensions

Last modified: