CMSI 4320 - Cognitive Systems Design
Professor Andrew Forney • Andrew.Forney[at]lmu.edu
Cognitive Systems are loosely defined as automated agents that exhibit animalistic or higher-level human intellectual capacities such as the ability to explain their choices, think counterfactually, and learn from examples, rewards, and punishment. The ever-diversifying field of AI boasts two primary subdisciplines that investigate how to develop, research, and theorize over the nature of such agents. This course examines these two fields from a formal, computational standpoint, while weaving in some considerations from the philosophy of science as well; in particular, we will explore the theory, applications, and opportunities to concert:
Reinforcement Learning: a paradigm for training sophisticated, but somewhat animalistic, agents based on the principles of positive and negative reinforcement from online, active experimentation.
Causal Inference: a theoretical framework built atop probability and statistics that allows for an interpretable and rich set of inference queries that follow from modeling systems of causes and effects.
Learning Outcomes
By the course's end, students will:
Gain a deep understanding of modern theories of Causal Inference (CI), including the causal hierarchy, do-calculus, causal modeling, structural counterfactuals, and transportability.
Obtain a practical understanding of some problems and approaches to Reinforcement Learning, including Multi-Armed Bandit problems (and applications to CI), Markov Decision Processes, Q-Learning, and Deep Q Networks.
Appreciate the conceptual bridge that connects human cognition, attempts to automate it, and portions of experimental design as they relate to knowledge acquisition on an individual-basis.
Experience working with popular reinforcement learning environments like with Pytorch and AI Gym, in causal modeling with Structural Causal Models, and a deepened understanding of probability and statistics.
Identify some potential avenues for research, especially at the intersection of causal inference and reinforcement learning.
Prerequisites
Note: before taking this course, it is assumed that you have a matured grasp on the Python programming language, as well as the conceptual topics from Algorithms (CMSI 2130) and Artificial Intelligence (CMSI 3300). We will spend little time in-class reviewing each!
Algorithms / AI Refresher: need a review? You can consult my course archives for the last offerings of CMSI 2130, 3300.
In particular, you should *at least* be familiar with the following concepts:
CMSI 2130: Classical Search, Minimax, Dynamic Programming, Hillclimbing.
CMSI 3300: Probability Theory, Bayesian Networks, Sampling, Machine Learning (esp. learning algorithms like SGD).
Python Refresher: consult your CMSI 1010 instructor's materials or any number of online tutorials like Berkeley's listed below. For practice, see our LMU CMSI resource page, also linked below.
Git Refresher: you should be comfortable with the fundamentals of Git and managing GitHub repositories as GitHub Classroom will be used to distribute and collect all assignments in the course. Need a refresher? See the Git links below.
Texts
Given that there are two main, and rather distinct, topics covered in this class, we will employ 2 optional textbooks:
Causal Inference [CI] Text: Causal Inference in Statistics: A Primer, by Pearl, Glymour, and Jewell
Reinforcement Learning [RL] Text: Reinforcement Learning: An Introduction [Free]
Although these specific texts are not mandatory for the course, they contains many concepts that we will be covering in great detail, and your assignments will expect that you understand these concepts.
Additionally, I will provide abbreviated course notes for our lectures on my course page (see Notes tab)
Resources will be provided throughout the course detailing any additional topics that are not sufficiently contained in the above text. These resources will be free and publicly available.
Lecture Attendance
Not mandatory to attend: lecture attendance is strongly advised since you'll otherwise miss out on all of our inside-jokes. Less importantly, you'll be missing out on lecture content that may not be summarized in the course notes, like questions asked by peers, classwork exercises, and hints for homework and exams.
Lectures are participatory: come prepared to ask and answer questions to make the most out of your education!
Laptops discouraged, notetaking encouraged: take notes by hand and summarize the important points -- don't copy verbatim.
* No need to inform me of missed lecture: I'll just assume you'll be making things up by reading online / asking friends.
(* unless you'll be missing any form of assessment -- see Grading section below)
Technology permitting, I will attempt to record the lectures, but you should not rely on lecture recordings for keeping up with content!
Workload Expectations
The university's workload expectations are such that every unit in which you are enrolled translates to 3 hours of weekly work, including time in-class, spent studying, and working on assignments.
As this is a 4-Unit class, it is expected that you will be allocating an average of 9 hours outside of lectures per week working on course assignments, reading, and studying. You should manage your time appropriately.
Some weeks will have less material to keep you busy, and others more, but be aware that this is a work-intensive course that may require you to spend a lot of time studying and programming!
Finding Help for the Course
LMU CMSI has a ton of resources and people who are thrilled to help you succeed! See the following opportunities and avenues for getting help in this class' material.
Slack Messaging
Alongside email exchanges, you may reach me via the Slack messaging service (Slack is a chat client that can be
downloaded here) in the LMUCS channel (lmucs.slack.com) and username forns.
Feel free to reach out any time -- in the worst case, I have to step away from my desk and will answer your message when I return!
Please try to send your note / question in a single message rather than splitting it into like 9 different messages that make my computer sound like it's making popcorn.
Student / Office Hours are times during which I'm available for nothing but you, the students! I can help you: review course material, debug your programs, provide guidance on further exploration, and more!
R 10:15am - 11:45am, DOO 201A or 217 (they're right next to each other)
W 10:00am - 1:00pm, via Zoom (see link in course welcome email)
Questions by Slack / Email ANY time!
Steps for Remote Office Hours Attendance:
I do not publicly post my Zoom office-hours link, but just send me a note on Slack *in advance of office hours* to see where I'm at (the "Starter Pack" email I send at the beginning of the semester also has this information).
My office hours tend to be well attended, so if you join into a waiting room, it means I'm helping other students and will admit you as soon as I'm free.
Once you've been admitted, you / your group will be placed into a Zoom Breakout Room with priority given to the order in which you joined. Wait in here and I'll be with you as soon as I can!
Teaching Assistants (TAs)
Teaching Assistants (TAs) are student employees who are on-staff during certain hours to help you review material, debug code, and answer questions about the class!
Warning: this schedule is subject to change and may fluctuate with weekly TA availability; check the #tutoring channel on the LMUCS
Slack for the latest updates to the TAs' schedules!
Respecting our TAs! Remember that TAs are students like you with their own obligations and schedules; here are some tips for how to make respectful use of their generous time and skill.
| Do's | Don'ts |
|---|---|
Ask a TA to explain a concept from class or the homework, draw things out, walk you through the steps, or hint at your bugs. |
Ask a TA to write your code or do your debugging for you -- they've been explicitly told not to do so. |
Message TAs on Slack or email with questions *during their assigned tutoring hours.* |
Message TAs *outside of* their assigned tutoring hours -- send me a Slack message instead. |
Come to the Keck Lab to work on assignments during a TA's office hours; try to do so well in advance of deadlines! |
Demand TA attention and time while a queue of students are attending, or when a deadline is near. |
Bottom Line: remember that TAs are teaching assistants, but if you ever have questions or concerns that you don't wish to discuss with TAs, you should always feel free to contact the professor!
Tentative Schedule of Topics
The following constitutes the tentative schedule for topics to be covered in the course. This is largely an approximation for what we will cover each lecture, and is subject to change. For each topic, the relevant chapter of the textbook is also given, however, you are only required to understand the sections of each chapter that correlate with the material in the course notes -- the rest are for your curiosity and edification!
* CN = Course Notes, indicating topics that will be covered and listed on our course website but are not found in the textbook, even though *all* topics will have an accompanying page in our course notes.
Note: For any lecture tagged as "(Recorded Video)", do NOT come to the classroom -- in-person class will not be held these days.
| Item | Date | Topic | Text Chap. |
|---|---|---|---|
| Lecture 1-1 | |||
| Lecture 1-2 | |||
| Lecture 2-1 | |||
| Lecture 2-2 | |||
| Lecture 3-1 | |||
| Lecture 3-2 | |||
| Lecture 4-1 | |||
| Lecture 4-2 | |||
| Lecture 5-1 | |||
| Lecture 5-2 | |||
| Lecture 6-1 | |||
| Lecture 6-2 | |||
| Lecture 7-1 | |||
| Lecture 7-2 | |||
| Break | |||
| Break | |||
| Lecture 8-1 | |||
| Lecture 8-2 | |||
| Lecture 9-1 | |||
| Lecture 9-2 | |||
| Lecture 10-1 | |||
| Lecture 10-2 | |||
| Lecture 11-1 | |||
| Lecture 11-2 | |||
| Lecture 12-1 | |||
| Lecture 12-2 | |||
| Lecture 13-1 | |||
| Lecture 13-2 | |||
| Lecture 14-1 | |||
| Lecture 14-2 | |||
| Lecture 15-1 | |||
| Lecture 15-2 | |||
| Lecture 16-1 | |||
| Final |
Assignments & Grading
Grade Decomposition
Grades will be assigned based on the following weighted coursework:
[65%] Assignments: Catch-all category for classwork + homework combo exercises, which will be assigned roughly every 3 weeks and be weighted equivalently. You can expect roughly 5 large assignments throughout the course of the semester, some of which will overlap with the period in which you will be completing your final project (see below).
[35%] Final Project: a final project primarily in the domain of reinforcement learning but concerting elements of causal inference. The final project spec is available at the start of the course, but you will have the tools to begin work roughly 1/2 way through the semester.
Warning: your assignment and final project grades will be a composite of correctness and PostCommit Quiz results (see PostCommit section below).
PostCommit
PostCommit Quizzes: PostCommit is an app developed here in LMU CMSI and aims to harness the power of AI to help you prepare for technical interviews and conference presentation by answering interview-style technical questions about your own assignment submissions! Here's how these work:
AFTER the first submission deadline, a notice will be sent to the class to enroll in PostCommit by visiting the following link. Thereafter, you will use the same link for taking your PostCommit quizzes.
If you have already enrolled in PostCommit from another class, you're all set -- just log in to the site as usual!
Complete each assignment by the deadline given.
Important: If working on Group assignments, you MUST join the group via GitHub classroom.
On most programming assignments that you submit during the year, we will create a customized quiz to probe your depth of understanding for the code that you commit. These will be quiz questions that you answer in plain-English (though with technical vocab) similar to how you might explain code during a technical interview.
You must be (1) present in class when the quiz is offered (sometime following the assignment deadline, TBA in class), (2) bring your own laptop or lease one from the department / ITS, and (3) take the quiz on your device. You may NOT take each quiz on iPads or mobile devices.
Once the class has all taken the quiz, individualized feedback will be given through the app based on the quality and depth of your responses as well as advice for what might be expected in technical interviews. You can review the feedback through the app at any time. Each of your responses is graded on a 0-2 point scale:
0 = Missing, your response was incomplete, inaccurate, or otherwise greatly missed the mark of what the question asked of you.1 = Gist, your response had many correct pieces or was in the direction of the correct answer, but had small inaccuracies or missing pieces that were important to mention. This is often the case if you do not recruit the proper vocabulary or class material into your response.2 = Mastery, your response used the right vocabulary, was free of inaccuracy, and properly demonstrated your computing background from the course material as was relevant to the question.
Your score on the PostCommit Quiz (PCQ) will be converted to a percentage of the total possible (
pcq_percent = 2 * total number of questions) and compared to your correctness score on the assignment (correctness_percent = total assignment score / 100).Your understanding of the assignment's code-to-concept mapping is of the most paramount importance for your knowledge and experience to generalize beyond the class. As such, to motivate the understanding of your code, there will be a penalty assessed to your homework score if you produce functional code that you do not understand and cannot explain adequately as assessed through the PCQs (see FAQ below for rationale).
However, this penalty should only be assessed if there is a large difference ("delta" \(\Delta\)) between your correctness score and your PCQ score; it is not expected that you have a high code-to-concept mapping if your code is not fully functional. As such, we compute a PostCommit Delta value \(\Delta PC = \text{pcq_percent - correctness_percent}\).
You will receive a penalty if \(\Delta PC\) exceeds a certain threshold... however, you may also receive a BONUS if you didn't get fully functional code but can explain it well or articulate what you wanted to do, but couldn't quite get to.
The following table diagrams the possible penalties / bonuses:
\(\Delta PC\) Value
Assignment Score Adjustment
\(\Delta PC \lt -0.70\)
\(-20\)
\(-0.70 \le \Delta PC \lt -0.60\)
\(-15\)
\(-0.60 \le \Delta PC \lt -0.50\)
\(-10\)
\(-0.50 \le \Delta PC \lt -0.40\)
\(-5\)
\(-0.40 \le \Delta PC \lt 0.20\)
\(\pm 0\)
\(0.20 \le \Delta PC \lt 0.40\)
\(+5\)
\(0.40 \le \Delta PC\)
\(+10\)
Note that there is also an optional research component to PostCommit! Although everyone will participate in the quizzes, you have the option to not participate in the research component. See attached Informed Consent and Experimental Bill of Rights for more info:
PostCommit FAQ
What is the motivation for the penalties possible from PostCommit quizzes?
Because this is a foundational computing course (especially one whose topics are often queried in technical interviews), most of its assignments can be trivially solved using GenAI, looking up answers on the internet, consulting with friends, etc. which may complete the code without gifting you any of the intended knowledge, experience, debugging practice, etc. that are intended by them. The code you write in 5+ years may look very different from the code you write in this class, but the meta-skills and generalizable knowledge of *process* will support you if you do the work yourself as intended, and will be missing if you do not. Your grade should thus reflect your level of code-to-concept understanding, and continuing in the curriculum *without* it will harm only you.
How do I prepare for PostCommit Quizzes?
The best way is to be the one who labored over the assignment's solution yourself -- if you are the true author of your code who spent the intended time thinking, planning, connecting to lecture, outlining, implementing, debugging, and testing, the PCQ questions should come naturally. If they do not, it could mean that you are completing them with too much outside assistance, and that could even mean from the TAs! Before each quiz, take some time to review the spec, relevant course notes, and of course, your own code, focusing on big questions of "why" you chose to implement features the way you did.
Any tips for achieving Mastery level scores on PCQ questions?
Great question because these tips are similar to the best ways to answer technical interview style questions! Your answers should strive to be complete (not lacking any critical detail needed to demonstrate your technical background), but concise (not spewing every CMSI fact you know in the hopes of hitting the bullseye with the shotgun technique). Although you won't be penalized for mentioning extra relevant information in your answers, you may risk stating something that is inaccurate if your answers are too long, which can cause point deductions. Use of appropriate vocabulary from the lectures is one of the strongest ways to demonstrate your mastery; try to use the proper technical vocab rather than if you were talking to someone outside of the field.
What if there are issues with the PCQ questions asked of me or I disagree with the grade assigned?
We review each created question and each assigned score + feedback, but it's possible that the graders will miss something. Although you are always free to contact the instructor with any discrepancies or further explanation, we will only entertain questions about content, not fairness of assigned grade (these have already been considered). E.g., you should absolutely contact the instructor if there is an error in the feedback, but not because you were assigned a 1 on a question and believe it should be a 2 unless you can dispute the feedback's reason for that assignment.
What if I miss a PostCommit Quiz?
You are allowed 1 make-up PCQ IF (1) you contact me in advance of missing the class or (2) have documentation certifying your absence. You may not make up a quiz missed in any other fashion, which will count as a 0 for the purposes of the \(\Delta PC\) computation above.
Final Grades
Final letter grades are given based on the university scale of grade percentages:
A: 93 - 100
A-: 90 - 92
B+: 86 - 89
B: 83 - 85
B-: 80 - 82
C+: 76 - 79
C: 73 - 75
C-: 70 - 72
D: 65 - 69
F: 64 and below
Fractional grade percentages at 0.5 or over will be rounded up, so an 89.5% will be considered a 90% (A-), but an 89.4% will be considered an 89% (B+) on the above scale.
That said, these are only the guaranteed grade assignments: if your "final" grade is an 82%, you are guaranteed a B- or better, but you might still get an A if 82% is the top score.
Extra Credit
Extra credit opportunities will be sporadically available during lectures / exams; if you are present and can hand in an attempt at the extra credit opportunities, you will receive bonus points!
If you receive all extra credit opportunities, you can gain a maximum +2% on your final grade. These bonuses are applied post-curve, so no student will be punished for not attending lectures relative to their peers.
Submission Standards
Each of your submitted homeworks and projects are subject to the following constraints:
Group Size Limit: You are free to work in groups for the course's assignments. The maximum group size for this semester is 3 students.
Moreover, it is expected that ALL groupmembers contribute to any project and although you are free to divy-up responsibilities, ALL members are responsible for UNDERSTANDING what another groupmember has written.
Assignment posting: all assignments (classwork, homework, or otherwise) will be announced in class and posted on this site with due dates. You are responsible for attending the lectures / checking this site / your email for any updates to assignments and their associated deadlines.
Late Policy: assignments are due at exactly the time indicated by the method specified. Assignments that are late by 0 - 24 hours over the deadline receive only 80% of the credit they otherwise would have. Assignments late by more than 24 hours receive a 0. There are no exceptions to this rule, plan accordingly.
Time management is one of the most important meta-skills to learn in this class -- avoid the risk and try to finish at *least* one day in advance of each deadline.
Good habits: read the specs the *day* they come out to let the topics simmer, put blocks in your calendar that you plan on working on the assignments.
Submission Errors: "I forgot to push to GitHub" is not a valid excuse for late assignments; always double check your remote repository to make sure that your submission is present before the deadline. Moreover, assignments that contain syntax errors or deviate from given skeletons will receive a 0.
Style: assignments which are sloppily submitted without proper formatting, style, and comments are subject to penalties. We will discuss what constitutes such offenses in the style guide, though you should be familiar with clean coding standards from your introductory courses.
Understanding: it is expected that you have fully understood any work that you submit as your own. If you submit any work that you cannot explain in technical detail, you may not receive credit for this work and may be liable for academic dishonesty. It is up to the instructor's discretion to quiz you, interview you, or use your PostCommit answers as evidence for understanding of submissions (or lack thereof). See Academic Honesty and PostCommit sections of syllabus for more detail and rationale.
Academic Honesty
Students are encouraged to talk and think about problems in groups, but each submission should be their own, containing no code that has been copy-pasted from another source. We take plagiarism very seriously, and each submission will be run through a similarity-checking mechanism to ensure fairness. Code or homework which we believe to be shared innapropriately between students or copied from the internet is subject to severe disciplinary action (not that you would, just sayin'). The following are examples of unacceptable behavior in this course:
Copying non-trivial amounts of code from the ANY other source into an assignment (excluding small things from StackOverflow like how to use Python list comprehension to simplify some code). This includes code found on the internet or through the use of a generative AI model like ChatGPT.
Copying code from another student's submission, sharing detailed pseudocode, or having them verbally explain their submission line-by-line and then copying that.
Copying another group's solutions on a classwork exercise (sharing between group members is encouraged... that's what it means to... you know... group up).
Submitting code that you understand very little; the point of this class is not just to produce a working submission, but in the journey and labor of its development.
If you plan to copy anything from the internet, ask first!
The following slides provide more CMSI-specific standards for academic honesty; it is expected that by reading this syllabus you are aware of these standards.
CMSI Academic Honesty Standards
The use of Generative AI (Gen-AI) services like ChatGPT and Cursor can accelerate, augment, and amplify your learning... so long as they are never used to avoid it!
|
This class allows for GenAI under certain, but not all, circumstances. If you choose to use GenAI in ways not sanctioned below, you are liable for
academic dishonesty. The following contains non-exhaustive do's and don'ts for GenAI use in this class:
|
Rule of thumb: if anyone but you is responsible for your submitted work, you are liable for academic dishonesty! Remember that it is YOU taking this course to learn the tools of your chosen trade, your degree will be useless otherwise!
Resources
Note: this is an abnormally impacted section of this class! Your patience and hard work are appreciated in advance; we'll make the most out of this semester together!
With that said, here are some things you can expect from this class, and what I expect from you:
Community Agreement
Just as I have certain expectations of you in this course, you may have certain expectations of me. Here is a high-level list of each:
| My Responsibilities | Your Responsibilities |
|---|---|
Answer your questions and curiosities in office hours, over Slack, and via email with a turnaround of at most 36 hours. |
Be responsible with use of your time, class time, office hours, email, and the Keck Lab TAs. Last minute pleas for help on assignments or exams may not be enough time to get you the assistance you need. |
Provide you with an educational experience superior to that which could be gleaned from a textbook or online source alone. |
Keeping up with material, doing outside study, and asking questions when you're stuck -- I'm happy to help, but can't read minds. |
Give you feedback on your submissions and individualized instruction on how to maximize your gain from this class. |
Don't provide answers on exams or solutions to assignments that are not your own. |
Ensure that any group work is productive and encouraging; I'm always happy to mediate if conflict occurs in a group. |
Be respectful of group mates: do not expect someone to do all of the work (including yourself) and try to hear ALL group members' voices. |
Be as welcoming, inviting, and judgment free as possible -- I want you to learn and get excited about this class! |
Be respectful and supportive of classmates -- we're all learning together. |
Tips for Success in this Class
Here are a few tips for succeeding in this class (and college, in general); they're not meant to sound patronizing, but rather, are simply pearls of wisdom to hand down from someone who has recently trodden your path!
Ask questions: this is a brave space where inquiry is the heart of your education -- don't waste the opportunity!
Chances are good that someone else has the same question, free karma!
I might have made a mistake / typo and your question can correct it -- we're all human (I think)!
Don't feel like asking in-class? Jot a
?in your notes to ask later via email, Slack, or in Office Hours!
Fear not the specter of error: it may not feel great to be wrong, especially publicly in a college class -- but I'm here to tell you there's no better time to be wrong!
You'll never be wrong about what you were wrong about again -- free studying!
In-class is the perfect time to be wrong -- there are no stakes for being wrong, unlike in an interview, exam, or while you're coding on the job!
We celebrate being wrong in here because everyone can learn from mistakes -- you'll be admired as the brave one who volunteered.
Indulge your curiosity: and no, that's not some tagline stolen from a Vegas day-spa (actually, it might be, don't quote me on that).
We only have time to scratch the surface of class topics -- don't let your learning stop there, especially on topics that resonate!
Google related projects, experiment with questions of "what if" on different algorithms -- go forth and explore!
Remember that you belong: no matter what you look like, believe, etc. remember that if you have a passion for computing and the hunger to learn, you are a welcome member of this class, department, and institution.
This includes having a growth mindset: even if you feel like you're behind or unprepared, know that everyone who has the drive can learn computing -- we believe in you!
No need to feel like impostors -- this material is challenging but will help you grow -- if it were easy, you wouldn't get anything from it!
Always feel free to let me know if you feel otherwise on any of the above: if you're feeling excluded for ANY reason, let me know, because if I can't help you, I'll find someone who can.
University Resources
Here are some LMU links for University Resources that might be of interest:
Computer Science Department - Student Guide: There are some great review and practice guides in here too!
Academic Degree Requirements and Policies: Pay particular attention to honor code and process.