CSc 422 Assignment 4

Hit Or Miss

Assigned
Due

Nov 13
90 pts
Dec 11

This is a simulation assignment in implementing replacement algorithms. You are given an existing C++ program which reads actual memory reference traces produced by a program called valgrind. The existing simulation implements the LRU, NRU and FIFO algorithms discussed in the text. You are to add clock and page aging.

Though the trace files must be generated on Linux, this assignment can be solved anywhere you can compile standard C++.

Downloads

The existing program is built from nine C++ files, which are linked here. But the easiest way is just to download afiles.tgz or afiles.zip. These are each archives of all the needed code.

Each of the example programs requires a valgrind address trace to run. There's some instructions below about making your own, but here are some traces I made: tgz, zip. The one called a3_lslog.txt is a short trace from running the Unix file listing command ls, the others are small GUIs, a toy (xeyes) and a a calculator tool (xcalc). It is quite possible to make longer ones, but these already take a while for the program to process.

Download the code and make it compile, and download the traces. The executable runs from the command line, and it takes four or five arguments. The name of the trace file to process, the name of the process to use, the number of real memory pages, the number of bits in the page offset, and the frequency of running the maintenance operation. The maintenance operation parameter is only used by NRU, and refers tells how often to run the procedure to clear all the reference bits. The units are references, so if the parameter is given as 1000, NRU clears the referenced bits every 1000 memory references. The number of page offset bits is the number of bits in the rightmost part of the address when it is broken up for translation. For 4k pages, that is 12 bits. This is also the default value. After running, it prints statistics. For instance,

[tom@tomslap v4]$ vm xcalc_trace fifo 35 Algorithm: FIFO Memory: 35 frames of 4096 bytes each for 143360 total. Performance: 39241289 references produced 336226 faults, rate: 0.00856817

Assignment

As noted, you are to add the two additional algorithms, one to simulate clock, and one for page aging. Aging will use the maintenance frequency to decide how often to update the ages; clock will ignore this parameter.

Study the existing code to see how the algorithms are coded. Each algorithm has its own class, derived from the abstract base class ReplacementAlg defined in replacement.h. The base class takes care of reading the file of memory references, maintaining the page table, and deciding if each reference causes a fault. If so, it calls the abstract virtual method fault which returns the number of the real page chosen to use to bring in the new page. This is where the replacement algorithm runs.

You will need to add two new classes, one for clock and one for aging, and you will need to modify the vm.cpp program to included your new class files, and to recognize their names when given on the command line. You should find code for these things already present, commented out.

For clock, you might want to start with the FIFO class and modify it. Clock is just FIFO with second chance, so modify the program to check the ref bit before evicting a page. If it's set, clear it and go on. As the hand moves through each page number, use the base class method get_PTE_by_frame to see what page (if any) is using that frame, and to access its referenced bit.

For the page aging class, your constructor will need to allocate an array of 8-bit ages per page. Implement the virtual method maint to update the ages. This is called periodically by the base class according to the maintenance frequency setting. Your method will scan through each real page number, and update each age, again using get_PTE_by_frame to get and clear the reference bit. You might look at the one in NRU for a starting point. Your fault method will then scan through the ages looking for the smallest one.

Page aging should make one important modification to the algorithm as described in the text: When you bring in a new page after a fault, initialize its age to 128 rather than waiting for it to be updated at the next maintenance scan. This will keep the page from being immediately selected again if the next fault occurs before the next regular age update.

For both algorithms, fill all unused pages before replacing pages in use. This may happen naturally, but make sure it does.

The trace files can be quite large, containing many millions of memory references. These small simulations can therefore take a while to finish. I have also noticed that aging is very sensitive to the frequency parameter, and usually doesn't do well when the number of pages is small. Clock seems to perform consistently well on these traces. Here are the results from a few of mine:

[tom@tomslap v4]$ vm a3_lslog.txt clock 10 Algorithm: clock Memory: 10 frames of 4096 bytes each for 40960 total. Performance: 1387453 references produced 37971 faults, rate: 0.0273674 [tom@tomslap v4]$ vm a3_lslog.txt aging 10 12 1000 Algorithm: aging, maint freq 1000 Memory: 10 frames of 4096 bytes each for 40960 total. Performance: 1387453 references produced 108918 faults, rate: 0.0785021 [tom@tomslap v4]$ vm a3_lslog.txt aging 10 12 5000 Algorithm: aging, maint freq 5000 Memory: 10 frames of 4096 bytes each for 40960 total. Performance: 1387453 references produced 195674 faults, rate: 0.141031 [tom@tomslap v4]$ vm a3_lslog.txt aging 10 12 300 Algorithm: aging, maint freq 300 Memory: 10 frames of 4096 bytes each for 40960 total. Performance: 1387453 references produced 87997 faults, rate: 0.0634234 [tom@tomslap v4]$ vm xcalc_trace aging 30 12 1000 Algorithm: aging, maint freq 1000 Memory: 30 frames of 4096 bytes each for 122880 total. Performance: 39241289 references produced 1397161 faults, rate: 0.0356044 [tom@tomslap v4]$ vm xcalc_trace clock 30 12 Algorithm: clock Memory: 30 frames of 4096 bytes each for 122880 total. Performance: 39241289 references produced 334728 faults, rate: 0.00853 [tom@tomslap v4]$ vm xcalc_trace aging 30 12 2000 Algorithm: aging, maint freq 2000 Memory: 30 frames of 4096 bytes each for 122880 total. Performance: 39241289 references produced 2658639 faults, rate: 0.0677511 [tom@tomslap v4]$ vm xcalc_trace clock 20 10 Algorithm: clock Memory: 20 frames of 1024 bytes each for 20480 total. Performance: 39272083 references produced 1232272 faults, rate: 0.0313778

Simulator Organization

The figure below shows the general organization of the simulation.

The main program is in vm.cpp. It mostly handles arguments, creates objects, and calls the trace reader in class ReplacementAlg, which is what drives the simulation. The ReplacementAlg object contains the reader, and also the page table. It handles most of the mechanics of maintaining the page table and translating references. This is all done and working.

The “Specific Algorithm” box implements the particular replacement algorithm in question. In fact, there are three of these given, to implement LRU, FIFO and NRU. You must create two more, to implement Clock and Aging. A specific algorithm class is derived from ReplacementAlg, and implements the specifics by overriding the methods name(), fault() and possibly maint(), according to the algorithm implemented. The base class processes page references, manages the page table, identifies faults, and provides a number of utility methods to help implement your overrides.

You will need to implement (override) these methods:

string name()
Return the name of algorithm implemented.
unsigned long fault(const Ref &r)
This method is called by the simulation when a fault occurs. The parameter r gives the virtual address which caused the fault. The page where r resides must be loaded into memory, and the fault method must return the page frame (physical page) number where that page should go. The base class will use this return value to update the page table and various statistics. If the frame is already occupied, the previous occupant is removed before r is installed.
void maint()
This is called periodically. Implement it for an algorithm that requires some periodic operation, such as clearing the referenced bit or updating ages.

There are several inherited methods available to interact with the base class. The first one is the constructor, which takes a number of page frames (real memory size), number of page offset bits (which implies the page size), and (optionally) a maintenance frequency, which tells how often to call the maint method, in memory references. A frequency of zero disables the call. For this, your class should mostly receive and forward on the same values. See the constructors of the existing replacement algorithms for examples of how to do this. In addition, you have a number of inherited methods to use implementing your overrides. Some of the major ones are:

unsigned long get_num_frames()
int get_page_bits()
int get_maint_freq()
Each of these return of the values sent to the constructor, as described above.
unsigned long pageno(const Ref &r)
Take a reference (perhaps the argument sent to fault), and return the virtual page number where it is located.
const PTE *get_PTE_by_page(unsigned long p)
Get a pointer to a PTE object for the indicated virtual page number p. You may not update this PTE.
const PTE *get_PTE_by_frame(unsigned long f)
Get a pointer to a PTE object for the indicated frame number (real page number), f. You may update this PTE, but note that the returned pointer will be NULL if frame f is unused.
unsigned long increal(unsigned long &rpno)
Increment rpno modulus the number of real pages, and return the old value of rpno.
The PTE object has several simple methods to extract and control its translation real page number and status bits: is_valid(), get_frame(), get_ref(), get_mod(), set_ref(), set_mod(), clr_ref(), clr_mod(). The valid bit and translation (frame) cannot not be set, only checked, since they are managed by the base class. The modified bit can be cleared, but it is probably a bad idea to do so.

Submission

When your program works, is well-commented and nicely indented, submit over the web here.

Where Do Traces Come From?

They come from using the valgrind tool with a plugin called (for some reason) lackey. Valgrind is available for Linux and Mac; there's no Windows port. If you have a supported platform, and have acquired valgrind, you can create a trace with this command:
valgrind --tool=lackey --trace-children=yes --trace-mem=yes command 2> filename
This will run any command under the tracer, and direct the standard error output to filename. The trace is written to the error stream. The command will generally run quite slowly, and the trace file will be quite large. I've also had trouble getting any large GUI program to work; I suspect there is some issue that comes from slowing things down so much, but I don't know for certain if that's the problem.