CS 150 - Prelab 10

Due: Friday 5/5 at the beginning of lab

For our last (yay!) prelab we're going to be revisiting our data that we collected from lab 8. If you'd like to work in a pair on the lab and the prelab, you're welcome (and encouraged) to do so.

The data

This week, we're going to be extending our weather aggregator program from Lab 8. Recall that the end result of the program was a file of the form:
<date>  <hour>  <temp>
<date>  <hour>  <temp>
where each entry is separated by a tab. For example, if we collected data for a few hours we might get:
4-18-2017       15      49.55
4-18-2017       16      50.23
4-18-2017       17      51.78
4-18-2017       18      51.52
4-18-2017       19      49.41
4-18-2017       20      42.83
4-18-2017       21      41.59
4-18-2017       22      40.72
If you set up this program to run every hour for a particular zip code, you would eventually end up with a file spanning multiple days, with one entry per hour containing the temperature for that date/hour. Here are two examples of files containing such "raw" temperature data: Take a quick look at them just to make sure you understand what is in these files.

Matlab friendly data

Matlab is not very good at processing text files, so often when working with real data you will manipulate the data with another language (e.g., Python) to get it into a format that's easier for Matlab to handle.

The goal of this week's lab will be to plot the temperature data as well as generate some statistics characterizing the weather. Before we can do this, though, we need to convert our data into something more Matlab-friendly. I created a Python program that reads through the data in a raw file output from our Lab 8 data and creates a file where each line in the file represents a days worth of data. For example, for test-raw.txt the output would be:

32.21 32.31 32.30 32.08 34.10 35.30 36.08 36.97 39.83 39.84 39.96 40.87 41.29 42.20 41.73 41.55 41.48 41.69 42.19 43.03 43.68 44.10 44.15 43.28
43.11 43.18 43.83 43.92 44.23 44.82 45.25 46.06 46.69 48.33 52.78 58.24 66.22 75.99 84.73 92.93 100.80 101.11 95.74 77.92 63.69 51.85 44.90 36.58
33.45 32.79 31.97 31.04 30.90 30.56 30.11 29.81 29.32 29.07 28.60 28.36 28.35 28.06 27.91 27.61 27.09 26.50 26.18 26.06 25.69 25.42 25.38 25.19
There are three days worth of data. Each line/row is a day and each column is an hour (0-23).

For both of the raw files I am providing corresponding matrix files that are much easier for Matlab to read:

In the wisc-matrix.txt file, what does the 7th entry on the 5th line represent? Write your answer on a piece of paper and bring it to class on Friday. Be specific!

Due diligence

I've posted the lab for this week. Read through the entire handout. This will make sure your time gets best utilized for the lab on Friday.

If you haven't done so yet, make sure you've worked through the examples in section 5 of the matlab basics handout that we looked at on Monday. These examples will be similar to the types of things you'll do for the lab.

(Optional) The conversion

If you want more practice with Python (and in particular Python data processing and file manipulation) you can try to write a program that does the data conversion described above. The program should read the data from a raw file and then write a file in the matrix format with one day per line.

If you get stuck (or if you're just curious), I've put a copy of a program called process_weather.py in the same directory as the data files that does this conversion. There you can also find another program, wisc_data.py, that I used to convert the .csv file downloaded from the Wisonconsin site to the "raw" file to match the format of weather_aggregator.