pete > courses > CS 315 Fall 26 > lecture 10: mmap

Lecture 10: mmap

Goals

last time, we looked at the process of dynamic linking

we have previously looked at the ELF metadata within a program

among which data we can find enumerated the collection of libraries that need to be linked at runtime

but this doesn’t tell us precisely which file on disk fulfills this bibliographical need

the ldd program will do this for us

applying it to the hello-world program from the previous lecture…

$ ldd hello-world
    linux-vdso.so.1 (0x00007ffe305f0000)
    libc.so.6 => /usr/lib/libc.so.6 (0x00007f29ebd98000)
    /lib64/ld-linux-x86-64.so.2 => /usr/lib64/ld-linux-x86-64.so.2 (0x00007f29ebf8f000)

we see that the Standard C library (libc.so.6) is to be found in the file /usr/lib/libc.so.6

if we ran nm on this file, we’d see symbols for all the functions you’ve seen with section-3 manpages

(we won’t, but you can on your own)

by this time, we’ve spent a lot of time looking at ELF and the various structures within it that allow us to store different types of data within a program

eg, instructions, PLT, GOT, symbol table, libraries to be linked

today, we’re going to look at how this stuff manifests itself in a running process

along the way, we’ll also see a little peek at a bunch of systemsy things (I’ll note them explicitly below) that we’ll get into in greater detail later in the semester

but first, we have to find a process to examine

you may be used to the Task Manager on Windows or Activity Monitor on OS X

the equivalent UNIX command is ps

$ ps
  PID TTY          TIME CMD
14623 pts/11   00:00:00 bash
16622 pts/11   00:00:00 ps

columns, left to right: - PID: process id, a unique number assigned to each process - TTY: identifies the terminal window in which the command is running - TIME: the length of time the process has actually been running on the CPU (recall that processes take turns using the CPU to make them appear to run simultaneously) - CMD: the command that started the process

this list is really short, though clearly there are other processes running on my machine

this is because, with no arguments, ps only lists processes running inside the current terminal session

(historical note: "TTY" and "terminal" are terms from antiquity, in which one made a physical connection to a computer to interact with it)

if you want to see all the running processes, ps has options for that

the particular set of options I use for that purpose is

$ ps auxww

(full output omitted due to length)

read the manpage to see exactly what they do, or just blindly use this command because it Just Works

here’s the first line of the output, which describes the columns:

USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND

new columns: - USER: the name of the user that ran the process - %CPU: portion of the CPU the process has used over the past couple seconds - %MEM: portion of system memory the process is currently consuming - VSZ and RSS: measures of process memory usage - START: when the process was started

what if I want information about a single process?

it would be nice if I could take the output of that command and remove all the lines I don’t care about

the grep program will do that: it takes a string (called a pattern) as a command-line argument and then reads from its input

every line the pattern appears in, it will print

that’s great, but now we need a way to pass the output of ps to the input of grep

this is precisely what the pipe does

$ ps auxww | grep firefox
pete      1345 17.6  6.6 3218148 1079924 ?     Sl   Oct02 645:13 /usr/lib/firefox/firefox -P default
pete     16676  0.0  0.0  11184  2384 pts/11   S+   22:21   0:00 grep -E firefox

note that it shows not just the firefox process, but also the grep process that is filtering for the "firefox" string

all of this has been leading up to the ability to figure out the PID for a particular process, because we need it to examine its running state

so pick a PID at random and let’s go to /proc

(for illustrative purposes in the notes, I will assume we’re choosing pid 12345, though whether a process with that pid actually exists on your machine when you try to duplicate these instruction is anybody’s guess—use ps to find one)

what is /proc, you ask?

it’s… weird, in that it doesn’t actually exist

think of the myread function you wrote

instead of actually reading data from the file, it could very easily have just fabricated random data and given it to the user

that’s essentially what all the files in /proc are: they don’t exist on disk anywhere, but when you ask for their contents, the kernel consults its internal data structures and generates the file data on the fly

among some other things, most of the directories in /proc are numerical: each one corresponds to a running process

therefore, the files in /proc/12345 will describe the process with pid 12345

you can see there are quite a few files in there

the file I want us to look at is maps, which shows all the memory in use by a running process—otherwise known as its memory map

$ cat /proc/12345/maps

which shows us lots of inscrutable output

we’ll look at a few items individually

first, at the top:

00400000-004c6000 r-xp 00000000 fe:00 56760614                           /usr/bin/bash

this says that the contents of the file /usr/bin/bash exist in memory at addresses 00400000 through 004c6000 and that this memory is both readable and executable (r-xp)

these are the instructions!

the address range might look familiar from your pokings-around with gdb

the next two lines:

006c5000-006c6000 r--p 000c5000 fe:00 56760614                           /usr/bin/bash
006c6000-006ca000 rw-p 000c6000 fe:00 56760614                           /usr/bin/bash

the first line here represents read-only data (r--p) defined by the program

the second is both readable and writeable (rw-p)—stuff like the GOT and global variables and static variables

a few lines later we can see the heap!

00759000-008a3000 rw-p 00000000 00:00 0                                  [heap]

and still further on we can see that libc has been loaded:

7f6815b87000-7f6815d35000 r-xp 00000000 fe:00 56756359                   /usr/lib/libc-2.26.so
7f6815d35000-7f6815f34000 ---p 001ae000 fe:00 56756359                   /usr/lib/libc-2.26.so
7f6815f34000-7f6815f38000 r--p 001ad000 fe:00 56756359                   /usr/lib/libc-2.26.so
7f6815f38000-7f6815f3a000 rw-p 001b1000 fe:00 56756359                   /usr/lib/libc-2.26.so

it, too, has parts that are readable and executable but not writeable; only readable; and readable and writeable but not executable

I don’t know off the top of my head why there is some associated memory that is neither readable, nor writeable, nor executable

you can also see that this particular process has a bunch of libraries loaded

finally, near the bottom, we see that it has a stack:

7fffb562c000-7fffb564d000 rw-p 00000000 00:00 0                          [stack]

and again the addresses jibe with what we’ve seen in gdb

this begs the question: how does the data from those files get into memory at those addresses?

one possible method is, on process start, for each file (executable or library):

this is not the way it’s done (because efficiency)

instead, the process uses the mmap system call to map a set of memory addresses to bytes within the file on disk

first, let’s make the vocab clear, because I think it will help clarify what’s going on

this term comes from mathematics, where functions are often described as mapping one set of values to another

for instance, consider the function f(x) = x * 2

it maps (translates) an input to an output

in the same way, a memory mapping translates an address in memory to a byte within the underlying file

therefore, given this mapping

55a777f2c000-55a777f34000 r-xp 00000000 fe:00 56762708                   /usr/bin/cat

if the process attempts to read memory address 55a777f2c000, it will a) succeed and b) give the zeroth byte of the file /usr/bin/cat

similarly, if the process attempts to read memory address 55a777f2c010, it will read also succeed and give the 16th byte of the file /usr/bin/cat (because those numbers are hex)

the upshot of all of this is that, when a file is memory-mapped, we can perform file I/O using pointers

so how does mmap work?

it’s a system call, so man 2 mmap should be your first stop

unfortunately, it’s a pretty imposing description (as befits one of the most important system calls) so let’s look at some examples of it in action

we’ll start with mmap-test.c

and the dummy input file test-file

and you might find the Makefile for the rest handy

I wrote this program to demonstrate how a mapping is set up

it pauses before creating the mapping (at which point we will examine the mappings and note that the file in question is not mapped)

then I’ll hit enter and it will perform the mapping (at which point we will examine the mappings and note that it is now mapped)

let’s walk through the source code before we compile and run it

note that I was conscientious about error checking in this program, to set a good example

in the later programs today, I was not conscientious to make sure the code all fits on one screen

you need to be conscientious in your code

to the code!

first we open(2) the file:

fd = open("test-file", O_RDONLY)

note that the permissions (in this case, O_RDONLY) must match how you want to use the memory being mapped

that is, if you want to be able to write to the memory (we’ll see an example in a bit) you need to open it either O_WRONLY or O_RDWR

then we find its size, because we need that when we get around to calling mmap

fstat(fd, &sb)

(sb is an instance of struct stat—check the fstat(2) manpage)

then the pause, tackily implemented using write(2) and read(2), with a shameful lack of error checking

and finally the call to mmap itself:

file_data = mmap(NULL, sb.st_size, PROT_READ, MAP_PRIVATE, fd, 0);

the first parameter allows you to request a particular address for the contents of the file to mapped to (though you might not get it)

the second parameter is the length of the mapping in bytes

the third parameter is the permissions of the mapping, in this case it will be read-only (we’ll see read/write in a bit)

the fourth parameter determines what happens to changes you make to the memory (we’ll also see this in action in a bit)

the fifth parameter is the open file descriptor corresponding to the file to be mapped

the sixth and parameter is where within the file to begin mapping—in this case, the beginning (offset 0)

mmap returns a pointer to the memory in which the file’s data will be available

to sum up, in this call, and in a possibly more-intuitive order: - we’re asking to map the file represented by fd (arg 5) - starting at the beginning of the file (arg 6) - continuing to the end (arg 2) - we want to read the data (arg 3) - we don’t want anybody else to see any changes we might make to the data (yes, even though we just said we’re only reading it) (arg 4) - and we don’t care where in memory the data appears (arg 1)

and we save the memory address at which the data is available in the variable file_data

so we can get access to that data through that pointer

which is exactly what we do with the next bit of code:

write(1, file_data, sb.st_size)

this writes the file data to file descriptor 1, which is stdout

then, to show that we can really interact with the data using pointers…

write(1, file_data + 13, sb.st_size - 13)

which outputs the file data starting 13 bytes in

finally, we de-mmap it (which is called "munmap")

munmap(file_data, sb.st_size)

and close the file

close(fd)

time to see it in action

compile it:

$ gcc -Wall -pedantic -o mmap-test mmap-test.c

and run it:

$ ./mmap-test
hit enter to mmap

it is now waiting to mmap

we will use this opportunity to check out the maps for the running process

first we find the pid:

$ ps auxww | grep mmap
pete     20582  0.0  0.0   6212   780 pts/7    S+   22:30   0:00 ./mmap-test

then look at the maps:

$ less /proc/20582/maps

we can see that test-file is not mapped ('q' to quit from less)

press enter, and we see the contents of the file as expected, followed by the second prompt:

This is some random test data.
random test data.
hit enter to munmap

since the file is still mapped, we can again check the maps and see it is indeed mapped:

7fc5e06b8000-7fc5e06b9000 r--p 00000000 fe:00 1442208                    /home/pete/tmp/315-mmap/test-file

not only that, but it’s mapped readable (r--p) and if we printed out the address of file_data in the program, it would be 7fc5e06b8000

so that’s basic mmap usage, now it’s time to get more interesting

check out writer.c

$ make writer
gcc -Wall -pedantic -c -o writer.o writer.c
gcc -ldl -o writer writer.o

(again, lack of error checking so it all fits on screen—I do not condone this behavior in real code)

we go through the same setup routine, but the main event this time is actually modifying the data by writing to the mapped memory:

memcpy(file_data + 8, "MINE", 4);

if we run the program, would we expect to see the data change?

$ cat test-file
This is some random test data.
$ ./writer
hit enter to write
hit enter to munmap file
$ cat test-file
This is some random test data.

no change

the reason lies in the MAP_PRIVATE flag provided to mmap(2), which is also the source of the "p" in the permissions I have shown and will now finally explain

recall the mapping of test-file shown above, in which the permissions are r--p

now, in writer.c, change MAP_PRIVATE to MAP_SHARED

then recompile, and re-run

$ make writer
gcc -Wall -pedantic -c -o writer.o writer.c
gcc -ldl -o writer writer.o
$ ./writer

find its pid and check its map:

$ ps auxww | grep writer
pete     22131  0.0  0.0   6216   708 pts/7    S+   23:14   0:00 ./writer
$ grep test-file /proc/22131/maps
7fa587339000-7fa58733a000 rw-s 00000000 fe:00 1442208                    /home/pete/tmp/315-mmap/test-file

and oh look the p has become an s

in the window running writer, press enter once (it should now saw "hit enter to munmap file") and check out the file contents:

$ cat test-file
This is MINE random test data.

and the data has changed

this exemplifies the difference between a private mapping and a shared mapping

when modifications are made to a private mapping, those modifications are NOT visible outside the process

when modifications are made to a shared mapping, those modifications ARE visible outside the process

which begs the question: what if another process has mapped the same file? will it see changes the first process makes if it’s a shared mapping?

let’s find out!

before we start, revert to the original contents of test-file (it should contain "This is some random test data.")

check out reader.c

compile it:

$ make reader
gcc -Wall -pedantic -c -o reader.o reader.c
gcc -ldl -o reader reader.o

open two terminal windows, one for the writer and one for the reader (the first line in each output will indicate which terminal it refers to—it is not being shown on the screen)

first, get the writer going

WRITER
$ ./writer 
hit enter to write

then get the reader going

READER
$ ./reader
hit enter to output file contents

press enter to see what the READER thinks the file says

READER
This is some random test data.

go press enter in the WRITER window

WRITER
hit enter to munmap file

now go see if the READER saw that by pressing enter in its window

READER
This is MINE random test data.

conclusion: when the mapping is shared, other processes do see the changes

and they seem the more or less immediately

this is kind of crazy

and potentially really powerful

but, as Peter Parker learned, with great power comes great responsibility

and we’ll learn about that responsibility in a couple weeks

Last modified: