1 Course Information

1.1 Course Description

In the information age, we are surrounded with data, much of which contains an element of randomness, or "noise". Taking advantage of the abundance of information requires the ability to make principled inferences and predictions under noisy conditions. A wide variety of inference and prediction can be understood in one way or another as applications of conditional probability. In the first part of the course, we will develop some of the mathematical tools we need before we can study conditional probability rigorously. These include proof by induction, combinatorics, set algebra, and probability spaces. In the middle section, we develop some theory of probability and discrete random variables, with an emphasis on conditional probability and Bayes’ rule. In the final segment, we look at some of the ways conditional probability can be applied to study random environments. Topics in this section include basic information theory, discrete state Markov chains (a kind of random process), and basic elements of Bayesian statistics and decision theory.

1.2 Prerequisites

Grade of B or better in any one of ISTA 116, MATH 163, MATH 263, or equivalent. One of CS 245, LING/MATH/PHIL 202, MATH 215 or MATH 243. These requirements may be waived for students with strong academic records with permission of the instructor.

1.3 Locations and Times

When: Mondays and Wednesdays, 4:30-5:45 P.M.
Where: TBA
Website: http://sista.arizona.edu/~cdawson/ista311/

1.4 Instructor Information

Colin Reimer Dawson, Ph.D.
Office: Gould-Simpson 850
Email: cdawson@email.arizona.edu
Office Hours: By appointment

1.5 Readings

I haven’t found a single textbook that covers everything we need at the right level. Readings will therefore be drawn from multiple sources, with my notes as a central “trunk”. We will read most or all of the following chapters, which can be obtained electronically.

Chapter 1 from:
Jaynes, E.T. (2003). Probability Theory: The Logic of Science. Cambridge University Press. (Available for free electronically from UA Library)

Chapters 4-8 and 10 from:
Bolstad, William M. (2007). Introduction to Bayesian Statistics. Hoboken, New Jersey: John Wiley and Sons.

Chapters 1-6 and 10 from:
Applebaum, David (2008). Probability and Information: An Integrated Approach, 2^nd Edition. Cambridge University Press.

1.6 Course Goals

This course has two main purposes: First, after taking this course, you should be accustomed to reasoning probabilistically, incorporating uncertainty in every day inferences, and should be able to translate real-world problems into mathematical language. Second, you should become comfortable with the structure of mathematical arguments, and should be able to assess whether conclusions follow deductively from definitions and premises. After completing this course, you’ll be prepared for 400-level classes in probability, statistics, artificial intelligence and machine learning.

1.7 Format of the Class

Officially we are holding the class as an “independent study”, but in reality it will be structured more as part tutorial series and part seminar. I’ll present new theory along with some examples for part of the class, and the rest of the time will be devoted to discussion of more examples and collaborative problem-solving. We’ll meet twice a week, Mondays and Wednesdays from 4:30-5:45, with another hour set aside at a mutually agreeable time for you to go over problems together with some guidance from me.

2 Course and Grading Policies

2.1 Components of the Grade

Since the class is officially an independent study, it is graded on a pass/fail basis. The grade will depend on attendance, participation, and a final project, with each of these three components given roughly equal weight. There are no exams.

I will assign homework problems, which it is very important that you do, but rather than turn it in and have me grade it, we’ll divvy up the problems for you to present the solutions in our discussion meeting. Presentations will be factored in to the participation grade, but you should not be afraid to make mistakes.

2.2 Attendance and Participation

Most people find the material in this class quite challenging. Practicing using concepts and techniques with homework problems is critical for internalizing the material; but it is not enough on its own, particularly since it is sometimes possible to complete homework mechanically, with only a shallow understanding of what’s going on. A deeper understanding occurs when you have to try to explain a concept or problem to someone else. The goal of the discussion sessions is for you to engage with each other in a way that allows the kind of understanding which is only possible through explaining to take hold.

2.3 Homework Assignments

I will assign a few problems every week for you to do at home and then present to the group. These will mostly be drawn from the various texts, but some might be made up by me. You are encouraged to work together on these. Since they’re not graded, I assume there’s little incentive to mindlessly copy someone else’s work, but just in case that’s wrong, don’t do it! It will show when you go to present it, anyway. If it becomes clear that students are regularly skipping out on homework, I may start to collect it to at least check for honest effort.

Feedback I’m not planning to collect and mark homework, but I’m more than happy to meet with you to discuss it in advance of the group meeting. Just make an appointment.

2.4 Final Project

The final project involves identifying a real-world phenomenon that can be represented using a probabilistic model, where the distribution of an observed variable depends on the value of an unobserved variable. For example, the unobserved variable could be a parameter in the distribution of the observed variable; it may govern transitional probabilities in a probabilistic process; or it may simply correspond to which in a set of competing hypotheses accurately describes the phenomenon. The goal is to (a) define the relevant variables and explain how they relate, (b) identify a set of questions about the problem that can be addressed with data, (c) gather or find some data relevant to the chosen questions, and (d) attempt to answer the question either analytically (solving equations on paper) or by computer simulation.

The writeup for the final project will consist of a short paper (4-6 pages), supplemented with any necessary mathematical derivations and/or computer code.

3 University Policies

Classroom Behavior. Students are expected to behave respectfully toward each other and to the instructor. Disrespectful behavior includes the use of cell phones or other electronic devices in the classroom during class hours.

The Arizona Board of Regents Student Code of Conduct is here: http://dos.web.arizona.edu/uapolicies/scc5308abcd.html#sccphilosophy

ABOR Policy 5-308, prohibits threats of physical harm to any member of the University community, including to oneself. See: http://policy.web.arizona.edu/~policy/threaten.shtml.

Special Needs and Accommodations. Students who need special accommodation or services should contact the

Disability Resources Center
1224 East Lowell Street, Tucson, AZ 85721
(520) 621-3268
FAX (520) 621-9423
email: uadrc@email.arizona.edu
web: http://drc.arizona.edu/.

You must register and request that the Center or DRC send official notification of your accommodations needs as soon as possible. Please plan to meet with the instructor by appointment or during office hours to discuss accommodations and how the course requirements and activities may impact your ability to fully participate. The need for accommodations must be documented by the appropriate office.

Student Code of Academic Integrity. Students are encouraged to share intellectual views and discuss freely the principles and applications of course materials. However, graded work/exercises must be the product of independent effort unless otherwise instructed. Students are expected to adhere to the UA Code of Academic Integrity as described in the UA General Catalog. See: http://dos.web.arizona.edu/uapolicies/.

Confidentiality of Student Records. See http://www.registrar.arizona.edu/ferpa/default.htm

Subject to Change Statement. Information contained in this syllabus, other than the grade and absence policy, may be subject to change by the instructor, with advance notice.

4 Schedule of Topics

Note: J refers to the Jaynes book, B to the Bolstad book and A to the Applebaum book. The numbers are chapters and sections: for example, B4.1-3 means section 4.1 through 4.3 from Bolstad. I will have notes for all sections, but those that are marked “Notes” do not have outside readings to go along with them.