This assignment has it all! You'll get conceptual practice with the first two layers of the causal hierarchy, as well as causal discovery with structure learning, heterogenous data, identifiability, adjustment, and programmatic approaches to it all!

This is a Group Assignment! Feel free to work in groups up to the group-size limit listed in the syllabus.

And if you thought for a second that my assignments would be named with some modicum of professionalism, guess again!

Start with the solution skeleton in-hand! In the following project, I've given you only the desired directory structure and datasets required for the problems that follow -- the rest is up to you!

Inside you'll find the directories:

• doc in which you should place your final report.pdf containing your answers to every question requesting a written response, computation, etc. I suggest you compose this document using either Word with its equations editor, or LaTeX (preferably). Enumerate your answers corresponding to each problem, clearly indicating what answer corresponds to what problem!

• dat in which I've provided some datasets that you'll employ in a couple problems.

• In the main directory you'll find the Python modules used during the programmatic sections that follow -- they'll be familiar since most are from Ass-1!

GitHub Classroom Link

	GenAI use is BANNED for all questions from here on until stated otherwise as it will shortcut your learning and generalizable skills though is always OK for understanding error messages, interpreting skeleton code, rephrasing what a question is asking, etc. as outlined in the syllabus.

What's the Joke?

The Tiers of Causality: let's start with an underhand pitch: enjoy the following comic by SMBC, and then explain the joke in reference to the causal hierarchy within your report.

Formatting Causal Queries

Formatting Queries: Causal models have a lot of utility in policy analysis, wherein their parameters are based off of large datasets that can be used in pursuit of informing systemic decisions, and determining how the effects of those decisions can affect a variety of variables. Some (actual) studies have pursued causal modeling for assessing political issues like (in the present example) causal factors that lead to terrorism.

Consider the following toy example with factors that have been studied as predictors of terrorist activity:

• \(P\) = political system

• \(F\) = foreign intervention

• \(W\) = weapon availability

• \(R\) = regional instability

• \(E\) = education level

• \(T\) = terrorist activity

From these, we collect data across a variety of studies and produce the following Causal Bayesian Network (CBN):

For each of the following plain English queries, convert it into a probabilistic expression that could be computed from the Markovian CBN above.

1. What is the likelihood of there being terrorism \((T=1)\) in nations with low education \((E=0)\) and ample access to weapons \((W=1)\)?

2. What would be the likelihood of terrorism \((T=1)\) if a UN Peacekeeping mission provided region stability \((R=1)\) to regions with low education \((E=0)\)?

3. What causal effect does Education (0 = no college, 1 = college) have on the presence of Terrorism \((T = 1)\)?

4. At what greater risk of Terrorism \((T = 1)\) is an unstable region \((R = 0)\) compared to a stable one \((R = 1)\)?

Queries in Full-SCMs

Queries in Fully-Specified SCMs: We're examining the hiring practices of a particular organization that has a reputation for being elitist and rampantly afflicted by croneyism (*Andrew puts down his word-of-the-day calendar*). From the company's hiring data and reasonable assumptions about the causal structure about the system, you're able to specify some variables of interest:

• \(S \in \{0,1\}\), the socio-economic status of the job candidate with \(0\) being low-SES, \(1\) being high.

• \(E \in \{0,1\}\), the education level of the candidate with \(0\) being low-status (e.g., ill-reputed undergrad institution or low GPA) and \(1\) being high.

• \(H \in \{0,1\}\), whether or not that candidate was hired with \(0\) being not.

• \(U_E \in \{0,1\}\)*, exogenous errors lumping in all the reasons someone in each SES may or may not go to college, like opting instead to pursue a career as a Twitch streamer.

• \(U_H \in \{0,1\}\)*, exogenous errors lumping in all the reasons someone may or may not be hired, like being a college roommate of the CEO, despite being dumb as a rock.

* Note: often, exogenous variables are seen as ways to model "normalcy" such that the system operates according to known influences *except for* when the exogenous variables represent an unmodeled exception (see structural equations demonstrating this below).

The model for this system (including structural equations for endogenous variables):

Hint: start by finding \(P(v)\) from \(P(u)\) and \(F\), and if you don't know what those things are, consult your notes!

Queries in Partial-SCMs

Queries in Partially-Specified SCMs: reconsider our Vaping-Heart-Disease network and then, using the do-modified steps of enumeration inference discussed in class, compute the likelihood of attaining heart-disease \((H=1)\) IF one were to start exercising \((do(E=1))\), given that they're Stressed \((S=1)\). Show all work and enumerate each step in enumeration inference.

ID & Adjustment

Identifiability and Adjustment: examine the following network structure, assuming that we know this to be a defendable model.

Note the dashed edges! \(U, W\) are simply labels for unobserved confounders (UCs) in the model!

1. \(P(Z|do(X))\)

2. \(P(B|do(A))\)

3. \(P(Z|do(C))\)

4. \(P(Z|do(A))\)

Causal Discovery from Data

Structure Learning with Heterogeneous Data: you are studying the relationships of various factors on... well... studying! In particular, you are interested in the causal relationships of a variety of factors on reported Test Anxiety. The factors of interest are:

• \(X \in \{0,1\}\), whether or not an individual Xercised (OK, so maybe I chose the variable names first and didn't want to remake the datasets) the day before the exam.

• \(W \in \{0,1\}\), whether or not an individual got 8+ hours of sleep the day before the exam.

• \(Y \in \{0,1\}\), whether or not an individual participated in a study group the day before the exam.

• \(Z \in \{0,1\}\), whether or not the individual experienced anxiety during the exam.

Your Task: determine the *true* structure of the causal relationships between each of these variables.

To do so, we'll be involving our old friend, Tetrad, which provides some structure learning tools that we touched on in AI.

Here's the information that we have available for analysis:

1. Test Anxiety Survey [survey.csv]: a survey was given to students leaving an exam and asked them to report on each of the above mentioned variables.

2. Sleep Interruption Study [sleep_study.csv]: students participated in a sleep study in which participants were sorted into two experimental groups at random: Group A was monitored in a sleep institute and awoken only after their brain rhythms had indicated a full 8 hours of sleep. Group B (or "Grump-B" as the experimenters dubbed them) were awoken after only 6 hours of sleep. Participants then took an exam from the experimenters in which they were told that the quality of their performance would be awarded with more cash the better they did (in reality all were paid the same amount). After the exam, the students were questioned on the un-intervened variables mentioned above (e.g., if they had exercised that day, participated in a study group, etc.).

3. Study Group... Study [study_study.csv]: In a similar experiment, participants were brought to stay at the same institute as in the previous study the night before an experimenter-administered exam, and were given the same material to study for the following day. However, they were also randomly assigned to 2 experimental conditions: Group A was asked to study in their room, quietly, upon arrival, while Group B was clustered into a variety of different study groups. The next day, the experimenters administered the exam, and afterwards, questioned the students on the un-intervened variables mentioned above.

4. Background Knowledge: Suppose, in addition to the datasets provided above, you find a wealth of studies from the life sciences that support the idea that Exercise leads to better quality of Sleep, and that no unobserved confounders connect Exercise with Test Anxiety.

[!] Each of the studies' results of the above can be found in the dat/study_studies directory!

Part 1: Using Tetrad, navigate to the menu and choose Pipelines > Load Data and Search. Double click the Data module to load one of the datasets, and then double click the Search module and use the PC algorithm (with default parameters) to search for the network structure within that data.

Once you've done this for each of the datasets, record the following in your report:

1. Record the graph recovered by Tetrad's structure learning (which examines only independence relationships, so the graphs here tell you ONLY associational-layer dependences leaving it up to you to deduce the causal ones) in your report.

2. Tetrad will be unable to orient some of the edges, in which observationally equivalence edges will be represented with undirected edges. Explain why, in each dataset, those edges were not able to be oriented.

To consider how these datasets, when viewed in unison, can be helpful, considering the following example of a Markovian causal model relating 4 variables, {A, B, C, D}, and two studies: an observational dataset and interventional on C with respect to how we can then deduce the true model structure.

Part 2: Using your knowledge about heterogeneous datasets, deduce the *true* Semi-Markovian network structure in the observational / unintervened model of the Studying Study datasets by combining clues from each of the studies and pieces of background knowledge individually. Record the true model in your report and explain your steps in deducing it.

Some hints:

• Note the differences in edges that can and cannot be directed in each of the different datasets.

• Remember the structural effects of interventions in the experimental data! This will help you deduce where the do-operator is... doing something.

• (Optionally) Record the important conditional independence relationships that may be different in each study.

• There may be at least 1 unobserved confounder present in the true model!

	GenAI use from this point of the assignment onwards is allowed! The main challenge of this part will be in fighting obnoxious syntax, less in implementing what you want to happen, which makes GenAI a great tool for the job.

Computing Queries Programmatically: you are examining medical data related to a particular set of treatments' effects on recovery from a condition. In particular, you possess data on:

• \(X \in \{0,1\}\), administered treatment between one of two drugs: Drug 0 and Drug 1.

• \(Y \in \{0,1\}\), whether or not the patient recovered (\(Y=1\) indicates that the patient recovered after treatment).

• \(Z, R, S, W \in \{0,1\}\), a variety of binary contexts / covariates describing patient characteristics that are related to the prescription's assigned by physicians and their outcome.

Using structure learning strategies like in the Study Studies problem, suppose we know the true network structure of this system and it's Markovian, but you also have FDA data on the treatments under which interventions \(do(X=x)\) took place; we'll refer to this model as the Medical CBN:

Specifications

---

Before beginning this section, familiarize yourself with the

pgmpy Causality Toolbox

Using the models specified above (and which will be provided to you as Causal Bayesian Networks), complete the methods in analysis.py with associated unit tests in analysis_tests.py that accomplish the following.

do Param

Let's start by learning how to use the do operator programmatically; turns out it's pretty straightforward with pgmpy, just examine a certain named parameter in the query method linked above!

With this in hand, use the do-operator to implement the following:

  def compute_query_from_G_using_do(cbn: "BayesianNetwork", do_var: str, do_val: int, out_var: str, out_val: int) -> float:
  """
  Computes P(Y = y | do(X = x)) in the given pre-intervention CBN for the given
  intervention value X = x using the given do-operator in pgmpy.
  
  Parameters:
      cbn (BayesianNetwork):
          The Causal Bayesian Network modeling the system in this exercise.
      do_var (str):
          The variable in the given CBN being intervened upon.
          -> In our query example, this would be X
      do_val (int):
          The value that the intervened variable is being forced to.
          -> In our query example, this would be the x in X = x
      out_var (str):
          The outcome / query variable
          -> In our query example, this would be Y
      out_val (int):
          The outcome / query variable value being requested
          -> In our query example, this would be the y in Y = y
  
  Example:
      # P(Y = 1 | do(X = 0))
      do_x_0_result = compute_query_from_G_using_do(cbn, do_var="X", do_val=0, out_var="Y", out_val=1)
  """

Ready to test? Spin up pytest -k test_compute_query_from_G_using_do. This tests the results of \(P(Y = 1 | do(X = x))~\forall~x\), i.e., the average causal recovery rates under each treatment.

do Derived

Recall, however, that sometimes we might need to perform adjustment in order to control for unobserved confounding should our CBN be Semi-Markovian.

Although there are some methods to perform adjustment in the causal toolbox, let's try things from (semi-) first-principles.

You'll now answer the SAME query \(P(Y = 1 | do(X = x))\) but WITHOUT using do-operator as provided by pgmpy. You may still use the query method but without the do named parameter.

Hint: think adjustment formulae!

  def compute_query_from_G_using_adjustment(cbn: "BayesianNetwork", do_var: str, do_val: int, out_var: str, out_val: int) -> float:
      """
      Computes P(Y = y | do(X = x)) in the given pre-intervention CBN for the given
      intervention value X = x WITHOUT using the do-operator in pgmpy; in other words,
      must use some form of adjustment to compute the query.
      
      [!] Simplifying assumption: assume any adjustment sets contain a maximum of 1 variable
      [!] Simplifying assumption: assume all variables are binary
      
      Parameters:
          cbn (BayesianNetwork):
              The Causal Bayesian Network modeling the system in this exercise.
          do_var (str):
              The variable in the given CBN being intervened upon.
              -> In our query example, this would be X
          do_val (int):
              The value that the intervened variable is being forced to.
              -> In our query example, this would be the x in X = x
          out_var (str):
              The outcome / query variable
              -> In our query example, this would be Y
      
      Example:
          # P(Y = 1 | do(X = 0)) = ??? (what do-free expressions can be used to compute this?)
          do_x_0_result = compute_query_from_G_using_adjustment(cbn, do_var="X", do_val=0, out_var="Y", out_val=1)
      """

Ready to test? Spin up pytest -k test_compute_query_from_G_using_adjustment. This tests the results of \(P(Y = 1 | do(X = x))~\forall~x\), computed using the adjustment formula; we should get the same answer as before if things went right!

do Subgraph

Tired of answering the same question over and over? Not yet!

Now, as opposed to answering the query \(P(Y = 1 | do(X = x))\) in graph \(G\), let's show how it can be answered in graph \(G_x\) if we instead made a CBN for the experimental data!

The twist: since this is already a CBN in the interventional subgraph, we don't even need to use the do-operator nor adjustment to get what we want because here: \(P_{G_x}(Y = 1) = P_{G_x}(Y = 1 | do(X = x)) = P_{G_x}(Y = 1 | X = x)\).

As such, confirm our query findings from earlier by implementing the following method assuming that we're passed a CBN already in the interventional subgraph of the intervention whose effects we're studying:

  def compute_query_from_G_x_using_no_do(cbn: "BayesianNetwork", do_var: str, do_val: int, out_var: str, out_val: int) -> float:
      """
      Computes P(Y = y | do(X = x)) in the given POST-intervention CBN for the given
      intervention value X = x WITHOUT using the do-operator in pgmpy.
      
      Parameters:
          cbn (BayesianNetwork):
              The Causal Bayesian Network modeling the system in this exercise.
              [!] Note: in this problem, the CBN represents the interventional-
                  subgraph, do(X = x)
          do_var (str):
              The variable in the given CBN being intervened upon.
              -> In our query example, this would be X
          do_val (int):
              The value that the intervened variable is being forced to.
              -> In our query example, this would be the x in X = x
          out_var (str):
              The outcome / query variable
              -> In our query example, this would be Y
      
      Example:
          # P(Y = 1 | do(X = 0)) = P_{G_X=x}(Y = 1)
          do_x_0_result = compute_query_from_G_x_using_no_do(cbn_do_x, do_var="X", do_val=0, out_var="Y", out_val=1)
      """

Ready to test? Spin up pytest -k test_compute_query_from_G_x_using_no_do. This tests the results of \(P_{G_X=x}(Y = 1)~\forall~x\), computed using the interventional subgraph \(G_x\); we should get the same answer as BOTH previous computations before if things went right!

Punchline: look at all the different ways you can compute the same query depending on what you have and what you want: P7 can be used for Causal Inference in Markovian models, P8 when you have a Semi-Markovian model and identifiable query, and P9 when you have interventional data!

Obs. Choices

Here's another perk of having both observational and experimental data: we get to be critical of how things are and how they should be!

Let's first implement a method aimed at determining what the most likely treatment would be prescribed for a given patient in "the wild" (i.e., outside of the laboratory setting where physicians are free to prescribe whatever they see fit).

Turns out that given the same patient, different physicians can sometimes choose radically different treatments, and with the pace of medical science, the "best" treatment today might be replaced or even found dangerous tomorrow!

To help us see how often each treatment \(X=x\) is being prescribed for patients presenting with certain attributes, implement the following method that computes \(argmax_x P(X = x | C = c)\) for choice variable X and contexts C:

  def get_most_likely_obs_choice(cbn: "BayesianNetwork", choice_var: str, context: dict[str, int]) -> dict[str, int]:
      """
      Computes argmax_x P(X = x | C = c) for choice variable X and given evidential context C.
      
      Parameters:
          cbn (BayesianNetwork):
              The Causal Bayesian Network modeling the system in this exercise.
              [!] This is the pre-interventional model
          choice_var (str):
              The variable whose values we are determining which is the more likely
              choice in the given circumstances.
              -> In our query example, this would be X
          context (dict[str, int]):
              Mapping of evidence variables to their observed values
              -> In our query example, this would be C = c for each C in evidence
      
      Example:
          # What's the most likely treatment for a patient presenting with {"R": 0, "S": 1}?
          # i.e., find argmax_x P(X = x | C = c) with C = {"R": 0, "S": 1}?
          context_result = get_most_likely_obs_choice(cbn, choice_var="X", context={"R": 0, "S": 1})
      """

Ready to test? Spin up pytest -k test_get_most_likely_obs_choice. This tests both to make sure you're able to determine: (1) the most-prescribed treatment on average \(argmax_x~P(X=x)\), and (2) the most-prescribed treatment for those exhibiting attributes {"R": 0, "S": 1}, i.e., \(argmax_x~P(X = x | C = c)\) with C = {"R": 0, "S": 1}

Exp. Choice

Finally, let's compare the choices that physicians are making on-average in "the wild" vs. the "best" choices they should be making by causal criteria.

Implement the following method that, given experimental data / the interventional subgraph \(G_x\), computes \(argmax_x~P(Y = 1 | do(X = x) C = c)\) for choice variable X, evidential context C, and outcome variably Y whose "desired" value is Y = 1.

  def get_best_causal_choice(cbn: "BayesianNetwork", choice_var: str, out_var: str, context: dict[str, int]) -> dict[str, int]:
      """
      Computes argmax_x P(Y = 1 | do(X = x) C = c) for choice variable X, evidential context C,
      and outcome variably Y whose "desired" value is Y = 1.
      
      Parameters:
          cbn (BayesianNetwork):
              The Causal Bayesian Network modeling the system in this exercise.
              [!] This is the pre-interventional model
          choice_var (str):
              The variable whose values we are determining which is the more likely
              choice in the given circumstances.
              -> In our query example, this would be X
          out_var (str):
              The outcome / query variable
              -> In our query example, this would be Y
          context (dict[str, int]):
              Mapping of evidence variables to their observed values
              -> In our query example, this would be C = c for each C in evidence
      
      Example:
          # What's the best treatment for a patient presenting with {"R": 0, "S": 1}?
          context_result = get_best_causal_choice(cbn, choice_var="X", out_var="Y", context={"R": 0, "S": 1})
      """

Ready to test? Spin up pytest -k test_get_best_causal_choice. This tests to make sure you're returning the best action in each context.

Note an important discrepancy between this test and the last! Doctors are prescribing \(X = 0\) more on average, but the best treatment by causal criteria is actually \(do(X=1)\)!

Now that we're familiar with all of the Causal Inference tools offered by programmatic implementations, let's connect to our other topic in the course: reinforcement learning!

There are many ways in which causal reinforcement learning is becoming a hot topic, but we'll see just a sliver of interesting explorations here.

This portion will take place in the Multi-Armed Bandit (MAB) setting that we had waaaay back in assignment 1 -- need a refresher? Head to the Assignments tab!

In fact, you'll have the chance for review very shortly because you're going to need parts of your Ass-1 implementation to start!

MAB ASR Port

This is just to get started up in the MAB domain again; head on over to mab_agent.py and perform the following:

1. Copy over your MABAgent, TSAgent, ContextualAgent classes, replacing those in the current skeleton.

2. If you hadn't implemented it as-such already, your ContextualAgent must use Thompson Sampling for each different context that it sees. For example, if given the context c_t = {"S": 0, "W": 1}, this is essentially an entirely different TS bandit environment compared to c_t = {"S": 1, "W": 0}.

3. Run pytest -k test_old_mab_agents to make sure that all of the above has ported over successfully.

Contextual Subset MAB

Now that all of our previous work is in place, let's empower it. How you might ask? Well let's get some intuition on where Causality has some things to say!

Observation 1: In the Contextual Bandit Problem, NOT ALL context variables are either needed / should be used by the optimal policy to manage the explore vs. exploit dilemma effectively!

To demonstrate this, you'll implement a variant of the ContextualAgent that we'll call ContextualSubsetAgent described as follows.

  class ContextualSubsetAgent(ContextualAgent):
      '''
      Custom agent that plays the Contextual MAB game, accepting some
      context covariates c_t before each decision, but only some of
      which may feature into this *selective* agent's decisions!
      
      [!] Still inherits from ContextualAgent, but you may wish to override
      parts of the superclass' constructor or other methods!
      '''
      
      def __init__ (self, dec_vars: list[str], dec_cards: list[int], context_vars: list[str], context_cards: list[int], ignore_contexts: list[str]):
        '''
        See @ContextualAgent.__init__
        
        Can be used to initialize any other attributes needed for this agent's ASR.
        
        Added Parameters on top of those in Superclass:
            context_vars (list[str]):
                A list of context variables that are available to the agent to make their
                decision at each trial, e.g., ["W", "S"]
            context_cards (list[int]):
                A list of variable cardinalities belonging to each context variable by
                index in which they appear in context_vars.
            ignore_contexts: (list[str]):
                A list of context variable names that can be excluded on causal grounds.
                Variables specified here should not be conditioned on while either making
                an action choice nor when getting feedback.
        
        Example:
            context_vars = ["A", "B", "C"]
            ignore_contexts = ["B"]
            
            When making a Greedy choice / getting feedback, only use contexts "A" and "C"
        '''

To implement the above:

1. Implement the given constructor (hint: you can use the Superclass relationship to save work here).

2. Override the give_contextual_feedback, contextual_choose methods to ignore any context variables passed in for ignore_contexts during construction.

Observe the performance of the correctly-implemented ContextualSubsetAgent in the Medical CBN with the following sets of

Causal MAB

Finally (whew!), let's consider the world where causality can save us from having to do any learning at all and just draw from prior experience!

The twist: just like in real life, in this variant of the contextual bandit game, you will be given some SUBSET of contexts at each trial! E.g., at trial 1 you might be given {"Z": 1, "W": 0} and at trial 2, {"R": 1}, etc.

The good news: you come equipped with prior experience in the form of an observational CBN that describes the environment that your agent can use to make the best decision given the context!

To demonstrate why we care about causal vs. observational queries, even in settings where we're chasing reward, you'll craft 2 agents that respond to the given contexts at each trial, \(z_t\):

1. CausalContextualAgent, which will make decisions by Causal Decision Theory, i.e., that makes choices using the CBN via:

   $$a_t = argmax_a~P(Y=1 | do(a), z_t)$$

2. ObservationalContextualAgent, which will make decisions by Evidential Decision Theory, i.e., that makes choices using the CBN via:

   $$a_t = argmax_a~P(Y=1 | a, z_t)$$

Once more, like the previous problem, you may assume:

• Reward \(Y=1\) is the desired outcome, a binary reward.

• Likewise, that there is one binary decision to make (even though the constructor takes in a dec_vars param to scale-proof)

• All context variables are binary.

For each of CausalContextualAgent and ObservationalContextualAgent, you need only implement its __init__, choose_action methods -- there is NO feedback that these agents use (and therefore no learning) since we already have the governing CBN for each.

Observe the performance of the correctly-implemented agents in the Medical CBN with the following sets of

Note: because we are not always given all contexts at each trial, it will not always be possible to select the optimal action! Still, we must make the best decision we can with the info given.

You will be submitting your assignments through GitHub Classroom!

What

Complete all requested sections of your report.pdf (making sure to clearly label problem numbers and answers) and place in your doc directory. Complete all requested Python scripts precisely where they appear in the skeleton directory.

How

To clone this assignment (if you need a refresher), consult the guide here:

GitHub Classroom Tutorial

To submit this assignment:

• Simply push your final, submission copy to the GitHub Classroom repository associated with your account.

• Place your name at the top of *all* submitted files AND in the accompanying readme file.