Assignment to evaluate methods of understanding a system's behavior

This assignment takes approximately 60 minutes to complete. All sections are timed -- please allocate an uninterrupted time to complete the assignment. The assignment collects some basic demographic information (e.g., age). Your identity and personal information will never be revealed beyond the research group that created this assignment.

Tell us more about yourself

Runtime Logs - Tutorial

This page describes runtime logs, which can be used to understand a system's behavior. At the end of this mini-tutorial, you will answer a few questions to check your understanding of these types of logs.

A runtime log records information about a system and its executions. Each system execution produces a single execution trace, and a log consists of multiple such traces. Each log line in a trace describes an event -- the action the system executed -- and a performance metric that conveys some additional information about the system at the time of the event. In this exercise, a header line at the top of each log will explain the log format.

Consider a simple system that allows clients to log in and after, to read or write data. A runtime log of this system is shown below, including a header line and four distinct execution traces, one per client (each with a unique Client ID). In the log, client a3983's trace (“login→read→logout”) is in bold. The performance metric in this log is the timestamp, in seconds. For example, timestamps associated with a3983's trace indicate that a read event took 4 seconds:
(13010 - 13006 = 4)

Checking your understanding

The questions below are intended to test your understanding of runtime logs described above. Each question has a single correct answer.

You won't be able to move on until you answer these questions correctly.

Behavioral Models - Tutorial

This page describes behavioral models, which can be used to understand a system's behavior. At the end of this mini-tutorial, you will answer a few questions to check your understanding of this technique.

A behavioral model is a graph that describes a system’s behavior. Each node in this graph represents an event: an action the system can take. A path through the model that starts at INITIAL and ends at TERMINAL represents an execution trace: a series of events corresponding to one execution of the system.

The behavioral model below describes two possible executions, one of which is “login→read→logout” (highlighted in red). The absence of a path in a model indicates an invalid execution. For example, there is no “login→logout” path in the model below, so according to the model, the system cannot produce such an execution.

Checking your understanding

The questions below are intended to test your understanding of behavioral models described above. Each question has a single correct answer.

You won't be able to move on until you answer these questions correctly.

Performance Models - Tutorial

This page describes performance models, which can be used to understand a system's behavior. At the end of this mini-tutorial, you will answer a few questions to check your understanding of this technique.

A performance model is a graph that summarizes a system’s behavior. A node in this graph represents an event: an action the system can take, and outgoing edges labeled with performance metrics convey additional information about system performance between the events (e.g., how much time can elapse between the events). A path through the model that starts at INITIAL and ends at TERMINAL represents an execution trace: a series of events corresponding to one execution of the system.

Each node in a performance model is labeled with an event. The edges are labeled with a performance metric, such as, for example, the amount of time that elapses between the two events connected by the edge. Other performance metrics include the number of bytes transferred, bytes written, a measure of power consumed, etc.

In the performance model below, the path “login→1→read→[2,4]→logout” (highlighted in red) indicates that for this execution trace, login takes 1 second, and read takes between 2 and 4 seconds.

Checking your understanding

The questions below are intended to test your understanding of performance models described above. Each question has a single correct answer.

You won't be able to move on until you answer these questions correctly.

Assignment Questions

On this page, you will use a [runtime log / behavioral model / performance model] to answer questions about a simple system.

Connection Tester

Connection Tester is a tool that diagnoses network issues. It tests a client's bandwidth, executes a series of queries, and then classifies the network path as "normal" or "abnormal" based on the results.

Runtime Log

Behavioral Model

Performance Model

System Understanding Questions

Based on the runtime log and behavioral model above, answer the following questions.

Assignment Questions

On this page, you will use a [runtime log / behavioral model / performance model] to answer questions about a simple system.

RADIUS Authentication

The RADIUS protocol authenticates clients on a network. To authenticate, a client first sends a username and password in an Access-Request message. The server can respond in three different ways. It can grant the user access with an Access-Accept message, or it can reject access with an Access-Reject message. In either case, authentication ends. Alternatively, the server can request additional credentials with an Access-Challenge message, after which the above steps repeat: the client sends another Access-Request message, and the server again responds.

In the following log, Total bytes transferred represents the number of bytes communicated in either direction before the current message is sent.

Runtime Log

Behavioral Model

Performance Model

System Understanding Questions

Based on the runtime log above, answer the following questions.

Assignment Questions

On this page, you will use a [runtime log / behavioral model / performance model] to answer questions about a simple system.

Caching Web Browser

Web browsers often cache previously-viewed pages and resources to disk so that subsequent requests to the same pages are faster. A browser might record when it caches a page to disk (cache-page), retrieves a page from disk (retrieve-page), caches an image to disk (cache-image), and retrieves an image from disk (retrieve-image). The log/model below represents captured traces for such caching and retrieving events along with the total number of KB read from and written to disk.

In the following log, Total KB transferred represents the number of KB read/written in either direction before the current event.

Runtime Log

Behavioral Model

Performance Model

System Understanding Questions

Based on the runtime log and performance model above, answer the following questions.

Before you go...

Please explain your experiences with logs, behavioral models, and performance models. Thank you!

Thank you!

You have successfully completed the assignment.