Thus far, in our formalizations of problem solving and search, we have made some assumptions about the environment that (although may apply to certain restricted problem sets) are not generalizable, and may limit the effectiveness of our approaches.

Moreover, we have been discussing uninformed search strategies...

What made search strategies uninformed?

Search strategies are uninformed when they integrate no problem-specific knowledge or constraints into the search operation.

Our uninformed search strategies are thus "blind" to some problem-specific information that may not only improve runtime performance improvements over those without this information, but can actually lead to more optimal solutions as well!

Recall our search example from last lecture in which we were searching for a path to get our agent to its goal in a grid-world.

Problems with Uninformed Search

---

To wit, suppose our agent must now be able to navigate mud tiles, \(M\) such that the cost of stepping on a mud tile (it's sticky, after all) is 3 rather than the standard 1 for movement onto a regular tile.

Moreover, there may be multiple goals to which our agent can navigate to satisfy the problem (see below).

Consider how our uninformed searches would perform on the above scenario, using the search tree that each might generate:

Above, is the optimal action always the most shallow (i.e., smallest depth) in the search tree?

No! With non-uniform cost problems, this is the case because depth is equivalent to cost (up to some scalar), but not so with non-uniform cost problems.

As such, will any of our uninformed search strategies be optimal?

No! None of DFS, IDDFS, or BFS will be guaranteed to find the optimal solution for non-uniform cost problems.

Problem 1: uninformed search strategies that consider only a set order of expanding nodes may not be optimal for non-uniform cost problems.

We'll have to find a way to take these differing costs into consideration!

That said, let's consider a separate issue: remember our pathfinding animation linked earlier in the week? Observe how wasteful BFS is in finding the route from the initial to goal state.

See all of those tiles that were explored North, South, and West of the initial state? i.e., the ones that were nowhere near closing in on the goal?

These are wasteful explorations and betray the naivety of our uninformed search -- it's simply striking out in all directions without regard for some moves being more likely successes at finding the goal than others.

Problem 2: uninformed search strategies may perform many wasteful explorations that are due to naivety about the problem at hand.

These can be both computationally and memory expensive, and for large search spaces, we want to cut down on both.

Addressing Problem 1: consider how we can empower our search to remain optimal for non-uniform cost problems.

If your intuition was to... I dunno... take the costs into consideration, you're right!

Best-first search strategies expand nodes in order from least- to highest-cost from the initial state.

In this way, we will never expand a node with higher total path cost before one with lower, thus preserving optimality guarantees.

So how do we accomplish this? We need to make 2 tweaks to our existing search strategies:

What extra piece of information will we need to record at each node in the search tree?

The total cost (from the root / initial state) to get to that node, i.e., \(g(n)\) for some node \(n\).

Here, the cost \(g(n)\) for node \(n\) can be intuited as the history / path cost to get to \(n\) and is easily computed by: $$g(n) = g(parent(n)) + cost(n)$$

Of course, having these costs stored in each node is not alone enough to preserve optimality: we now need to incorporate them into our search strategy.

How might we modify our frontier to now expand in order of least- to highest-cost?

Make it a priority queue treating \(g(n)\) as the priority!

We will call the priority associated with each node to be decided by its evaluation function \(f(n)\). For best-first, we have: $$f(n) = g(n)$$

Pretty dull evaluation function -- but powerful nonetheless! Let's see it in effect...

Trace the execution of best-first search on our example below:

Will best-first searches now be optimal / complete in non-uniform cost problems?

Yep! For the very reason that we never expand a node with higher cost than one with lower that's on the frontier.

Yet, we still need to address problem 2: the notion that even best-first searches may still exhibit the same behavior as uninformed searches in exploring many extraneous states.

Addressing Problem 2: consider how we can empower best-first search to omit wasteful explorations during search.

As we saw with our uninformed search strategies, we don't know about any descendants of a particular state (states reachable through some number of legal movements) without first expanding the states in between.

However, just because we don't know what states / costs might be incurred along a certain path, that doesn't mean we can't make educated guesses.

In search, a heuristic is an estimate of cost that is likely to be incurred along a certain path to a solution in the search tree.

Heuristics inform search strategies by providing problem-specific information that can guide the search, leading to optimal and more efficient search patterns.

In other words, heuristics represent our best guess for the future cost that would be encountered from a path starting at the evaluated node in a search tree.

For our current maze path-finding problem, what would be a good heuristic that, when given some node \(n\) in the search tree, would estimate how much cost would be incurred on the path to the nearest solution?

We can consider the Manhattan distance heurstic, which would compute the minimal number of movements required to get to the nearest goal from some node.

The Manhattan distance heuristic, parameterized by some node \(n\) in the search tree (given the set of all problem goal states, \(G\)), is functionally defined as: $$h(n) = argmin_{i \in G} |n.X - G_{i}.X| + |n.Y - G_{i}.Y|$$ (Note: the \(argmin\) operator is used to select the closest goal \(G_i\) in the set of all goals \(G\))

Note: heuristics are just that: rules of thumb! They are not always exact estimates of future cost.

Give examples for when a heuristic would yield exact vs. inexact estimates of future cost from some node in a search tree.

So how do we implement a heuristic in our Tree Search approach?

If, in best-first methods, we are already recording the history of costs in a node's \(g(n)\), and we have some heuristic \(h(n)\), then what would be a good way to judge the possible merit of a path emanating from some \(n\)?

The sum of the past costs and our estimate for the future ones!

This is precisely the tactic of a widely applied informed search called A* (pronounced "eh-star," like a Canadian astronomer).

A* search is an informed search algorithm by which nodes along the search tree's frontier are expanded according to the result of some evaluation function, \(f(n)\): $$f(n) = Cost_{past}(n) + Cost_{future}(n) = g(n) + h(n)$$ for history cost function \(g(n)\) and heuristic estimate function \(h(n)\)

In words, A* "scores" all nodes along the search frontier according to \(f(n)\) and then expands them in a logical order...

Apropos, if there is some score \(f(n) = g(n) + h(n)\) associated with every node \(n\) along the frontier, what would be a logical order of expansion for A* to perform given that we're interested in finding an optimal solution?

Expand the node with the lowest \(f(n)\), given that it will be our best chance of minimizing cost!

With uninformed searches, we modeled the frontier as a Queue for BFS and Stack for IDDFS, so likewise, we should consider how to implement the desired behavior of A* using a clever choice of data structure.

Because A* is just a variant of best-first search with a new evaluation function / priority for expansion, we can again use a priority queue frontier!

With these implementation details, let's diagram an example run using our troublesome problem for the uninformed searches:

Diagram the A* search tree and found goal state using the problem maze below.

Naturally, the next question for us should be analytical, and investigating the optimality of A*.

Is A* search complete? Is it optimal?

Yes and yes, even for non-uniform cost problems... as long as our heuristic is well chosen.

Visualization

---

Check out the A* visualization here:

Search Visualization

Heuristic Design

---

So what, exactly, constitutes a good heuristic?

Our choice of heuristic for A* search can not only affect its computational performance, but also whether or not it returns an optimal solution!

So, we of course need to vet our choices of \(h(n)\) to determine whether or not they are good implementations of problem-specific knowledge that can aid the search.

What are some desirable traits of heuristics? What must a heuristic never do?

(1) A heuristic must never overestimate the cost to a goal; it should (2) apply problem specific knowledge in pursuit of (3) estimating the true future cost as closely as possible.

Admissibility is a heuristic property stating that \(h(n)\) will never overestimate the cost of reaching the optimal goal from \(n\).

Why might an inadmissible heuristic compromise the optimality of A*?

An inadmissible heuristic may choose a non-optimal path in the search tree because it may erroneously believe the optimal one to have a cost higher than would be experienced in reality.

For this reason, admissibility is required for Tree Search heuristic implementations.

Which of the following are admissible / inadmissible heuristics? For inadmissible ones, provide an example problem that would cause A* to find a suboptimal goal.

  h1(n) = Manhattan distance away from the initial state
  
  h2(n) = (total # of goal tiles) 
        - (# of goal tiles in the same row as n)
        - (# of goal tiles in the same column as n)
        
  h3(n) = Manhattan distance of n from closest goal
  
  h4(n) = # of Mud tiles surrounding n

Plainly, given the above, we see that not all heuristics are created equal -- how, then, can we choose which to use in an implementation of A* and how effective will they be?

Heuristic Efficacy

---

Disclaimer: Heuristic Design is its own subfield of AI, and the below represents only the tip of the iceberg.

We'll look at a few... err... heuristic heuristics now!

Suppose we have two admissible heuristics \(h_1, h_2\), but \(h_1(n) \ge h_2(n)~\forall~n\). Which should we use, and why?

Prop 1: We should prefer \(h_1\) since it will expand at most as many nodes as \(h_2\), but possibly fewer.

However, it's rare that we ever have an admissible that is strictly greater than another admissible heuristic; suppose instead we have a more realistic scenario:

Suppose we have two admissible heuristics \(h_1, h_2\), but \(h_1(n) \not \ge h_2(n)~\forall~n\). How to exploit this scenario to have the best heuristic performance?

Prop 2: Create a third heuristic, \(h_3(n) = max(h_1(n), h_2(n))\). Note: \(h_3(n) \ge h_1(n), h_3(n) \ge h_2(n)\).

In other words, a heuristic that takes the maximum of two admissible heuristics will always be admissible and fit our first desirable property above.

However, what we have yet to address, is just how to measure how well a heuristic improves performance.

Suppose we knew the true future cost, \(h^*(n)\) associated with every node to the nearest goal state. How could we evaluate a heuristic's efficacy?

The difference between the true cost and a given heuristics across all states, namely: \(\sum_i h^*(n_i) - h(n_i)\). The larger this sum, the worse the heuristic is.

What is the problem with this approach?

We rarely have, or have the computational power necessary, to evaluate the sum for all possible states in the state space!

A more practical approach is to see how a heuristic performs empirically, i.e., across a wide variety of search problems in the same problem space.

The typical means of assessing an empirical measure of how good a heuristic is measures how much it limits the branching of the search.

The effective branching factor, \(b^*\), is simply an approximation of the branching factor that a uniform tree created by an A* solution would have needed to find a solution at depth d.

More formally, \(b^*\) has the mathematical definition (but no closed form) of:

  Let N be the number of nodes generated by an A* problem solution
  Let d be the depth at which a solution was found to said problem
  Then, b* is computed from the quantity:
  
  N + 1 = 1 + b* + (b*)^2 + ... + (b*)^d
  
  So, for example, if A* expanded 52 nodes and found a solution at
  depth d = 5, then:
  N + 1 = 1 + b* + (b*)^2 + ... + (b*)^d
  52 + 1 = 1 + b* + (b*)^2 + (b*)^3 + (b*)^4 + (b*)^5
                    |
                    | (maths)
                    v
  b* = 1.92

What would be the absolute best possible value for \(b^*\), and how can we judge the merit of a good heuristic from it?

The best possible value for \(b^* = 1\), as in the search tree that does a "bee-line" (pun intended) to the nearest goal. As such, we can judge a heuristic's merit by its effective branching factor's distance from 1.

In practice, good heuristics will tend to limit \(b^* \le 2\).

And that, folks, is informed search! Tune in next week when things get... adversarial...