Solved subset explanation.

Francisco Coelho
1 parent 53b3b48c
Showing 5 changed files with 185 additions and 40 deletions Show diff stats
biblio/asp/ASP-CORE-2.03b.pdf
code/asp/disj.lp
code/asp/pqueens.lp
text/pre-paper.pdf
text/pre-paper.tex
+% prob(a) = 0.3
 a ; -a.
 b ; c :- a.
 \ No newline at end of file
@@ -9,12 +9,12 @@ row(1 .. n).
 diag1(I, J, I - J + n) :- col(I), row(J).
 % Number ascending diagonals.
 diag2(I, J, I + J - 1) :- col(I), row(J).
-
-% WTF?
+%
+% Negative Restrictions
 :- D = 1 .. 2 * n - 1, not { queen(I, J) : diag1(I, J, D) } 1.
 :- D = 1 .. 2 * n - 1, not { queen(I, J) : diag2(I, J, D) } 1.
-
+%
 % Output this predicate.
 #show queen/2.
-#show diag1/3.
-#show diag2/3. 
 \ No newline at end of file
+%#show diag1/3.
+%#show diag2/3. 
 \ No newline at end of file
-\documentclass[a4paper]{article}
+\documentclass[a4paper, 12pt]{article}
+
+\usepackage[x11colors]{xcolor}
 \usepackage{hyperref}
 \hypersetup{
     colorlinks=true,
     linkcolor=blue,
 }
+
 \usepackage{commath}
+\usepackage{amsthm}
+\newtheorem{assumption}{Assumption}
 \usepackage{amssymb}
 %
 % Local commands
 %
-\newcommand{\todo}[1]{{\color{orange}TODO #1}}
+\newcommand{\note}[1]{\marginpar{\scriptsize #1}}
 \newcommand{\naf}{\ensuremath{\sim\!}}
 \newcommand{\larr}{\ensuremath{\leftarrow}}
 \newcommand{\at}[1]{\ensuremath{\!\del{#1}}}
@@ -22,7 +27,7 @@
 \newcommand{\langof}[1]{\ensuremath{\fml{L}\at{#1}}}
 \newcommand{\uset}[1]{\ensuremath{\left|{#1}\right>}}
 \newcommand{\lset}[1]{\ensuremath{\left<{#1}\right|}}
-\newcommand{\pr}[1]{\ensuremath{\mathrm{p}\at{#1}}}
+\newcommand{\pr}[1]{\ensuremath{\mathrm{P}\at{#1}}}
 \newcommand{\given}{\ensuremath{~\middle|~}}
 \title{Zugzwang\\\textit{Logic and Artificial Intelligence}}
@@ -42,84 +47,223 @@ Answer Set Programming (ASP) is a logic programming paradigm based on the Stable
 The Distribution Semantics (DS) is a key approach to extend logical representations with probabilistic reasoning. Probabilistic Facts (PF) are the most basic stochastic DS primitive and they take the form of logical facts, $a$, labelled with a probability, such as $p::a$; Each probabilistic fact represents a boolean random variable that is true with probability $p$ and false with probability $1 - p$. A (consistent) combination of the PFs defines a \textit{total choice} $c = \set{p::a, \ldots}$ such that
 \begin{equation}
-    P(c) = \prod_{a\in c} p \prod_{a \not\in c} (1- p).
+    \pr{C = x} = \prod_{a\in c} p \prod_{a \not\in c} (1- p).
+    \label{eq:prob.total.choice}
 \end{equation}
 Our goal is to extend this probability, from total choices, to cover the specification domain. We can foresee two key applications of this extended probability:
 \begin{enumerate}
     \item Support any probabilistic reasoning/task on the specification domain.
-    \item Also, given a dataset and a divergence measure, now the specification can be scored (by the divergence w.r.t. the \emph{empiric} distribution of the dataset), and sorted amongst other specifications. This is a key ingredient in algorithms searching, for example, an \textit{optimal specification} of the dataset. 
+    \item Also, given a dataset and a divergence measure, now the specification can be scored (by the divergence w.r.t.\ the \emph{empiric} distribution of the dataset), and sorted amongst other specifications. This is a key ingredient in algorithms searching, for example, an optimal specification of the dataset. 
 \end{enumerate}
-This goal faces a critical problem concerning situations where \textit{multiple} standard models result from a given total choice, illustrated by the following example. The specification 
-$$
-\begin{aligned}
-    0.3::a&,\cr
-    b \vee c& \leftarrow a.
-\end{aligned}
-$$
-has three stable models, $\set{\neg a}, \set{a, b}$ and $\set{a, c}$. While it is straightforward to set $P(\neg a)=0.7$, there is \textit{no further information} to assign values to $P(a,b)$ and $P(a,c)$. At best, we can use a parameter $\alpha$ such that
+This goal faces a critical problem concerning situations where multiple standard models result from a given total choice, illustrated by the following example. The specification
+\begin{equation}
+    \begin{aligned}
+        0.3::a&,\cr
+        b \vee c& \leftarrow a.
+    \end{aligned}
+    \label{eq:example.1}
+\end{equation}
+has three stable models, $\co{a}, ab$ and $ac$. While it is straightforward to set $P(\co{a})=0.7$, there is \textit{no further information} to assign values to $P(ab)$ and $P(ac)$. At best, we can use a parameter $x$ such that
 $$
 \begin{aligned}
-P(a,b) &= 0.3 \alpha,\cr
-P(a,c) &= 0.3 (1 - \alpha).
+P(ab) &= 0.3 x,\cr
+P(ac) &= 0.3 (1 - x).
 \end{aligned}
 $$
-This uncertainty in inherent to the specification, but can be mitigated with the help of a dataset: the parameter $\alpha$ can be estimated from the empirical distribution.
+This uncertainty in inherent to the specification, but can be mitigated with the help of a dataset: the parameter $x$ can be estimated from the empirical distribution.
-In summary, if an ASP specification is intended to describe some system that can be observed then:
+In summary, if an ASP specification is intended to describe some observable system then:
 \begin{enumerate}
-    \item The observations can be used to estimate the value of the parameters (such as $\alpha$ above and others entailed from further clauses).
+    \item The observations can be used to estimate the value of the parameters (such as $x$ above and others entailed from further clauses).
     \item With a probability set for the stable models, we want to extend it to all the samples (\textit{i.e.} consistent sets of literals) of the specification. 
     \item This extended probability can then be related to the \textit{empirical distribution}, using a probability divergence, such as Kullback-Leibler; and the divergence value used as a \textit{performance} measure of the specification with respect to the observations.
     \item If that specification is only but one of many possible candidates then that performance measure can be used, \textit{e.g.} as fitness, by algorithms searching (optimal) specifications of a dataset of observations.
 \end{enumerate}
-Currently, we are on the step two above: Extending a probability function (with parameters such as $\alpha$), defined on the stable sets of a specification, to all the samples of the specification. This extension must, of course, respect the axioms of probability so that probabilistic reasoning is consistent with the ASP specification. 
+Currently, we are on the step two above: Extending a probability function (with parameters such as $x$), defined on the stable sets of a specification, to all the events of the specification. This extension must, of course, respect the axioms of probability so that probabilistic reasoning is consistent with the ASP specification. 
 \section{Extending Probabilities}
-Given an ASP specification, we look at the \textit{total choices} $\fml{C}$, \textit{stable models} $\fml{S}$, \textit{atoms} and \textit{literals} $\fml{A}, \fml{L}$, \textit{samples} $z \in \fml{Z} \iff z \subseteq \fml{L}$ and \textit{events} $\fml{E}$ (consistent samples).
+Given an ASP specification, we consider the \textit{atoms} $a \in \fml{A}$ and \textit{literals}, $z \in \fml{L}$, \textit{events} $e \in \fml{E} \iff e \subseteq \fml{L}$ and \textit{worlds} $w \in \fml{W}$ (consistent events), \textit{total choices} $c \in \fml{C} \iff c = a \vee \neg a$ and \textit{stable models} $s \in \fml{S}$.
+
+% In a statistical setting, the outcomes are the literals $x$, $\neg x$ for each atom $x$, the events express a set of possible outcomes (including $\emptyset$, $\set{a, b}$, $\set{a, \neg a, b}$, \textit{etc.}), and worlds are events with no contradictions.
+
+Our path, traced by equations (\ref{eq:prob.total.choice}) and (\ref{eq:prob.stablemodel} --- \ref{eq:prob.events}), starts with the probability of total choices, $\pr{C = c}$, expands it to stable models, $\pr{S = s}$, and then to worlds $\pr{W = w}$ and events $\pr{E = e}$.
 \begin{enumerate}
-    \item \textbf{Given} $\pr{C = c}$ from the specification. Each total choice $C = c$ (together with the specification facts and rules) entails some stable models, $S_c$.
-    \item \textbf{Given} $\pr{S = s \given C = c}$ such that its value is zero if $s \not\in S_c$.
-    \item Each event $E = e$ either:
+    \item \textbf{Total Choices.} This case is given by $\pr{C = c}$, from equation \ref{eq:prob.total.choice}. Each total choice $C = c$ (together with the facts and rules) entails some stable models, $s \in S_c$, and each stable model $S = s$ contains a single total choice $c_s \subseteq s$.
+    \item \textbf{Stable Models.} Given a stable model $s \in \fml{S}$, and variables/values $x_{s,c} \in \intcc{0, 1}$, 
+    \begin{equation}
+        \pr{S = s \given C = c} = \begin{cases}
+            x_{s,c} & \text{if~} s \in S_c,\cr
+            0&\text{otherwise}
+        \end{cases}
+        \label{eq:prob.stablemodel}
+    \end{equation}
+    such that $\sum_{s \in S_c} x_{s,c} = 1$.
+    \item\label{item:world.cases} \textbf{Worlds.} Each world $W  = w$ either:
     \begin{enumerate}
-        \item Is a stable model. Then
+        \item Is a \textit{stable model}. Then 
         \begin{equation}
-            \pr{E = e \given C = c} = \pr{S = e \given C = c}.
+            \pr{W  = w \given C = c} = \pr{S = s \given C = c}.
+            \label{eq:world.fold.stablemodel}
         \end{equation}
-        \item \textit{Is contained} in some stable models. Then
+        \item \textit{Contains} some stable models. Then
         \begin{equation}
-            \pr{E = e \given C = c} = \sum_{s \supseteq e}\pr{S = s \given C = c}.
+            \pr{W  = w \given C = c} = \prod_{s \subset w}\pr{S = s \given C = c}.
+            \label{eq:world.fold.superset}
         \end{equation}
-        \item \textit{Contains} some stable models. Then
+        \item \textit{Is contained} in some stable models. Then
         \begin{equation}
-            \pr{E = e \given C = c} = \prod_{s \subseteq e}\pr{S = s \given C = c}.
+            \pr{W  = w \given C = c} = \sum_{s \supset w}\pr{S = s \given C = c}.
+            \label{eq:world.fold.subset}
         \end{equation}
         \item Neither contains nor is contained by a stable model. Then
         \begin{equation}
-            \pr{E = e \given C = c} = 0.
+            \pr{W  = w} = 0.
+            \label{eq:world.fold.independent}
         \end{equation}
     \end{enumerate}
-    \item For each sample $Z = z$
+    \item \textbf{Events.} For each event $E = e$,
     \begin{equation}
-        \pr{Z = z \given C = c} = \begin{cases}
-            \pr{E = z \given C = c} & z \in \fml{E}, \cr
+        \pr{E = e \given C = c} = \begin{cases}
+            \pr{W = e \given C = c} & e \in \fml{W}, \cr
             0 & \text{otherwise}.
             \end{cases}
+        \label{eq:prob.events}
     \end{equation}
 \end{enumerate}
-\begin{quotation}\marginpar{Todo} Prove the four event cases, and support the sum/product options, with the independence assumptions.
+Since stable model are minimal, there is no proper chain $s_1 \subset w \subset s_2$ so each world folds into exactly one ot the four cases of point \ref{item:world.cases} above.
+
+% PARAMETERS FOR UNCERTAINTY
+
+Equation (\ref{eq:prob.stablemodel}) expresses the lack of knowledge about the probability assignment when a single total choice entails more than one stable model. In this case, how to distribute the respective probability? Our answer to this problem consists in assigning an unknown probability, $x_{s,c}$, conditional on the total choice, $c$, to each stable model $s$. This approach allow the expression of an unknown quantity and future estimation, given observed data.
+
+% STABLE MODEL
+The stable model case, in equation (\ref{eq:world.fold.stablemodel}), identifies the probability of a stable model \textit{as a world} with its probability as defined previously in equation (\ref{eq:prob.stablemodel}), as a stable model.
+
+% SUPERSET
+Equation \ref{eq:world.fold.superset} results from conditional independence of the stable models $s \subset w$. Conditional independence of stable worlds asserts a least informed strategy that we make explicit:
+
+\begin{assumption}
+    Stable models are conditionally independent, given their total choices.
+\end{assumption}
+
+Consider the stable models $ab, ac$ from the example above. They result from the clause $b \vee c \leftarrow a$ and the total choice $a$. These formulas alone impose no relation between $b$ and $c$ (given $a$), so none should be assumed. Dependence relations are discussed in Subsection (\ref{subsec:dependence}).
+
+% SUBSET
+
+I'm not sure about what to say here.\marginpar{todo}
+
+\begin{equation*}
+    \pr{W  = w \given C = c} = \sum_{s \supset w}\pr{S = s \given C = c}.
+\end{equation*}
+
+But! $\pr{W = w \given C = c}$ already separates $\pr{W}$ into \textbf{disjoint}
+
+% INDEPENDENCE
+
+A world that neither contains nor is contained in a stable model describes a case that, according to the specification, should never be observed. So the respective probability is set to zero, per equation (\ref{eq:world.fold.independent}).
+%
+%   ================================================================
+%
+\subsection{Dependence}
+\label{subsec:dependence}
+
+Dependence relations in the underlying system can be explicitly expressed in the specification. 
+
+For example, $b \leftarrow c \wedge d$, where $d$ is an atomic choice, explicitly expressing this dependence between $b$ and $c$. One would get, for example, the specification
+$$
+0.3::a, b \vee c \leftarrow a, 0.2::d, b \leftarrow c \wedge d.
+$$
+with the stable models
+$
+\co{ad}, \co{a}d, a\co{d}b, a\co{d}c, adb
+$. 
+
+
+The interesting case is the subtree of the total choice $ac$. Notice that no stable model $s$ contains $adc$ because (1)  $adb$ is a stable model and (2) no stable model contains $adc$, because if $adc \subset s$ then $b \in s$ and $adb \subset s$.
+
+Following the case of equations (\ref{eq:world.fold.stablemodel}) and (\ref{eq:world.fold.independent}) this sets 
+\begin{equation*}
+    \begin{cases}
+        \pr{W = adc \given C = ad} = 0,\cr
+        \pr{W = adb \given C = ad} = 1
+    \end{cases}
+\end{equation*}
+which concentrates all probability mass from the total choice $ad$ in the $adb$ branch, including the node $W = adbc$. This leads to the following cases:
+$$
+\begin{array}{l|r}
+    x & \pr{x \given C = ad}\\
+    \hline
+    ad & 1 \\
+    adb & 1\\
+    adc & 0\\
+    adbc & 1    
+\end{array}
+$$
+so, for $C = ad$, $\pr{b} = 2/3, \pr{c} = 1/3, \pr{b,c} = 1/3 \not= \pr{b}\pr{c}$.
+
+\begin{quotation}\note{Todo}
+    
+    Prove the four world cases (done), support the product (done) and sum (tbd) options, with the independence assumptions.
 \end{quotation}
+\section{Developed Example}
+
+We continue with the specification from equation \ref{eq:example.1}. 
+
+\textbf{Step 1: Total Choices.} The total choices, and respective stable models, are
+\begin{center}
+    \begin{tabular}{l|r|r}
+        Total Choice ($c$) & $\pr{C = c}$ & Stable Models ($s$)\\
+        \hline 
+        $a$ & $0.3$ & $ab$ and $ac$.\\
+        $\co{a} = \neg a$ & $\co{0.3} = 0.7$ & $\co{a}$.
+    \end{tabular}
+\end{center}
+
+\textbf{Step 2: Stable Models.} Suppose now that 
+\begin{center}
+    \begin{tabular}{l|c|r}
+        Stable Models ($s$) & Total Choice ($c$)  & $\pr{S = c \given C = c}$\\
+        \hline 
+        $\co{a}$    & $1.0$ & $\co{a}$. \\
+        $ab$        & $0.8$ & $a$. \\
+        $ac$        & $0.2 = \co{0.8}$ & $a$.
+    \end{tabular}
+\end{center}
+
+\textbf{Step 3: Worlds.} Following equations \ref{eq:world.fold.stablemodel} --- \ref{eq:world.fold.independent} we get:
+\begin{center}
+    \begin{tabular}{l|c|l|c|r}
+        Occ. ($o$) & S.M. ($s$) & Relation & T.C. ($c$)  & $\pr{W  = w}$\\
+        \hline 
+        $\emptyset$ & all           & contained & $a$, $\co{a}$ & $1.0$ \\ 
+        $a$         & $ab$, $ac$    & contained & $a$ & $0.8\times 0.3 + 0.2\times 0.3 = 0.3$ \\
+        $b$         & $ab$          & contained & $a$ & $0.8\times 0.3 = 0.24$ \\ 
+        $c$         & $ac$          & contained & $a$ & $0.2\times 0.3 = 0.06$ \\ 
+        $\co{a}$    & $\co{a}$      & stable model & $\co{a}$ & $1.0\times 0.3 = 0.3$ \\
+        $\co{b}$    & none          & independent & none & $0.0$ \\
+        $\co{c}$    & none              & \ldots & & \\
+        $ab$        & $ab$          & stable model & $a$ & $0.24$ \\  
+        $ac$        & $ac$          & stable model & $a$ & $0.06$ \\  
+        $a\co{b}$   & none              & \ldots & & \\
+        $a\co{c}$   & none              & \ldots & & \\
+        $\co{a}b$   & $\co{a}$      & contains & $\co{a}$ & $1.0$ \\
+        $\co{a}c$   &  $\co{a}$         & \ldots & & \\
+        $\co{a}\co{b}$  & $\co{a}$      & \ldots & & \\
+        $\co{a}\co{c}$  & $\co{a}$      & \ldots & & \\
+        $abc$       & $ab$, $ac$    & contains & $a$ & $0.8\times 0.2 = 0.016$ \\
+    \end{tabular}
+\end{center}
- 
 \section*{References}
		1	+% prob(a) = 0.3
1	a ; -a.	2	a ; -a.
2	b ; c :- a.	3	b ; c :- a.
3	\ No newline at end of file	4	\ No newline at end of file
1	-\documentclass[a4paper]{article}	1	+\documentclass[a4paper, 12pt]{article}
		2	+
		3	+\usepackage[x11colors]{xcolor}
2	\usepackage{hyperref}	4	\usepackage{hyperref}
3	\hypersetup{	5	\hypersetup{
4	colorlinks=true,	6	colorlinks=true,
5	linkcolor=blue,	7	linkcolor=blue,
6	}	8	}
		9	+
7	\usepackage{commath}	10	\usepackage{commath}
		11	+\usepackage{amsthm}
		12	+\newtheorem{assumption}{Assumption}
8	\usepackage{amssymb}	13	\usepackage{amssymb}
9	%	14	%
10	% Local commands	15	% Local commands
11	%	16	%
12	-\newcommand{\todo}[1]{{\color{orange}TODO #1}}	17	+\newcommand{\note}[1]{\marginpar{\scriptsize #1}}
13	\newcommand{\naf}{\ensuremath{\sim\!}}	18	\newcommand{\naf}{\ensuremath{\sim\!}}
14	\newcommand{\larr}{\ensuremath{\leftarrow}}	19	\newcommand{\larr}{\ensuremath{\leftarrow}}
15	\newcommand{\at}[1]{\ensuremath{\!\del{#1}}}	20	\newcommand{\at}[1]{\ensuremath{\!\del{#1}}}
	@@ -22,7 +27,7 @@		@@ -22,7 +27,7 @@
22	\newcommand{\langof}[1]{\ensuremath{\fml{L}\at{#1}}}	27	\newcommand{\langof}[1]{\ensuremath{\fml{L}\at{#1}}}
23	\newcommand{\uset}[1]{\ensuremath{\left\|{#1}\right>}}	28	\newcommand{\uset}[1]{\ensuremath{\left\|{#1}\right>}}
24	\newcommand{\lset}[1]{\ensuremath{\left<{#1}\right\|}}	29	\newcommand{\lset}[1]{\ensuremath{\left<{#1}\right\|}}
25	-\newcommand{\pr}[1]{\ensuremath{\mathrm{p}\at{#1}}}	30	+\newcommand{\pr}[1]{\ensuremath{\mathrm{P}\at{#1}}}
26	\newcommand{\given}{\ensuremath{~\middle\|~}}	31	\newcommand{\given}{\ensuremath{~\middle\|~}}
27		32
28	\title{Zugzwang\\\textit{Logic and Artificial Intelligence}}	33	\title{Zugzwang\\\textit{Logic and Artificial Intelligence}}
	@@ -42,84 +47,223 @@ Answer Set Programming (ASP) is a logic programming paradigm based on the Stable		@@ -42,84 +47,223 @@ Answer Set Programming (ASP) is a logic programming paradigm based on the Stable
42	The Distribution Semantics (DS) is a key approach to extend logical representations with probabilistic reasoning. Probabilistic Facts (PF) are the most basic stochastic DS primitive and they take the form of logical facts, $a$, labelled with a probability, such as $p::a$; Each probabilistic fact represents a boolean random variable that is true with probability $p$ and false with probability $1 - p$. A (consistent) combination of the PFs defines a \textit{total choice} $c = \set{p::a, \ldots}$ such that	47	The Distribution Semantics (DS) is a key approach to extend logical representations with probabilistic reasoning. Probabilistic Facts (PF) are the most basic stochastic DS primitive and they take the form of logical facts, $a$, labelled with a probability, such as $p::a$; Each probabilistic fact represents a boolean random variable that is true with probability $p$ and false with probability $1 - p$. A (consistent) combination of the PFs defines a \textit{total choice} $c = \set{p::a, \ldots}$ such that
43		48
44	\begin{equation}	49	\begin{equation}
45	- P(c) = \prod_{a\in c} p \prod_{a \not\in c} (1- p).	50	+ \pr{C = x} = \prod_{a\in c} p \prod_{a \not\in c} (1- p).
		51	+ \label{eq:prob.total.choice}
46	\end{equation}	52	\end{equation}
47		53
48	Our goal is to extend this probability, from total choices, to cover the specification domain. We can foresee two key applications of this extended probability:	54	Our goal is to extend this probability, from total choices, to cover the specification domain. We can foresee two key applications of this extended probability:
49		55
50	\begin{enumerate}	56	\begin{enumerate}
51	\item Support any probabilistic reasoning/task on the specification domain.	57	\item Support any probabilistic reasoning/task on the specification domain.
52	- \item Also, given a dataset and a divergence measure, now the specification can be scored (by the divergence w.r.t. the \emph{empiric} distribution of the dataset), and sorted amongst other specifications. This is a key ingredient in algorithms searching, for example, an \textit{optimal specification} of the dataset.	58	+ \item Also, given a dataset and a divergence measure, now the specification can be scored (by the divergence w.r.t.\ the \emph{empiric} distribution of the dataset), and sorted amongst other specifications. This is a key ingredient in algorithms searching, for example, an optimal specification of the dataset.
53	\end{enumerate}	59	\end{enumerate}
54		60
55	-This goal faces a critical problem concerning situations where \textit{multiple} standard models result from a given total choice, illustrated by the following example. The specification
56	-$$
57	-\begin{aligned}
58	- 0.3::a&,\cr
59	- b \vee c& \leftarrow a.
60	-\end{aligned}
61	-$$
62	-has three stable models, $\set{\neg a}, \set{a, b}$ and $\set{a, c}$. While it is straightforward to set $P(\neg a)=0.7$, there is \textit{no further information} to assign values to $P(a,b)$ and $P(a,c)$. At best, we can use a parameter $\alpha$ such that	61	+This goal faces a critical problem concerning situations where multiple standard models result from a given total choice, illustrated by the following example. The specification
		62	+\begin{equation}
		63	+ \begin{aligned}
		64	+ 0.3::a&,\cr
		65	+ b \vee c& \leftarrow a.
		66	+ \end{aligned}
		67	+ \label{eq:example.1}
		68	+\end{equation}
		69	+has three stable models, $\co{a}, ab$ and $ac$. While it is straightforward to set $P(\co{a})=0.7$, there is \textit{no further information} to assign values to $P(ab)$ and $P(ac)$. At best, we can use a parameter $x$ such that
63	$$	70	$$
64	\begin{aligned}	71	\begin{aligned}
65	-P(a,b) &= 0.3 \alpha,\cr
66	-P(a,c) &= 0.3 (1 - \alpha).	72	+P(ab) &= 0.3 x,\cr
		73	+P(ac) &= 0.3 (1 - x).
67	\end{aligned}	74	\end{aligned}
68	$$	75	$$
69		76
70	-This uncertainty in inherent to the specification, but can be mitigated with the help of a dataset: the parameter $\alpha$ can be estimated from the empirical distribution.	77	+This uncertainty in inherent to the specification, but can be mitigated with the help of a dataset: the parameter $x$ can be estimated from the empirical distribution.
71		78
72	-In summary, if an ASP specification is intended to describe some system that can be observed then:	79	+In summary, if an ASP specification is intended to describe some observable system then:
73		80
74	\begin{enumerate}	81	\begin{enumerate}
75	- \item The observations can be used to estimate the value of the parameters (such as $\alpha$ above and others entailed from further clauses).	82	+ \item The observations can be used to estimate the value of the parameters (such as $x$ above and others entailed from further clauses).
76	\item With a probability set for the stable models, we want to extend it to all the samples (\textit{i.e.} consistent sets of literals) of the specification.	83	\item With a probability set for the stable models, we want to extend it to all the samples (\textit{i.e.} consistent sets of literals) of the specification.
77	\item This extended probability can then be related to the \textit{empirical distribution}, using a probability divergence, such as Kullback-Leibler; and the divergence value used as a \textit{performance} measure of the specification with respect to the observations.	84	\item This extended probability can then be related to the \textit{empirical distribution}, using a probability divergence, such as Kullback-Leibler; and the divergence value used as a \textit{performance} measure of the specification with respect to the observations.
78	\item If that specification is only but one of many possible candidates then that performance measure can be used, \textit{e.g.} as fitness, by algorithms searching (optimal) specifications of a dataset of observations.	85	\item If that specification is only but one of many possible candidates then that performance measure can be used, \textit{e.g.} as fitness, by algorithms searching (optimal) specifications of a dataset of observations.
79	\end{enumerate}	86	\end{enumerate}
80		87
81	-Currently, we are on the step two above: Extending a probability function (with parameters such as $\alpha$), defined on the stable sets of a specification, to all the samples of the specification. This extension must, of course, respect the axioms of probability so that probabilistic reasoning is consistent with the ASP specification.	88	+Currently, we are on the step two above: Extending a probability function (with parameters such as $x$), defined on the stable sets of a specification, to all the events of the specification. This extension must, of course, respect the axioms of probability so that probabilistic reasoning is consistent with the ASP specification.
82		89
83	\section{Extending Probabilities}	90	\section{Extending Probabilities}
84		91
85	-Given an ASP specification, we look at the \textit{total choices} $\fml{C}$, \textit{stable models} $\fml{S}$, \textit{atoms} and \textit{literals} $\fml{A}, \fml{L}$, \textit{samples} $z \in \fml{Z} \iff z \subseteq \fml{L}$ and \textit{events} $\fml{E}$ (consistent samples).	92	+Given an ASP specification, we consider the \textit{atoms} $a \in \fml{A}$ and \textit{literals}, $z \in \fml{L}$, \textit{events} $e \in \fml{E} \iff e \subseteq \fml{L}$ and \textit{worlds} $w \in \fml{W}$ (consistent events), \textit{total choices} $c \in \fml{C} \iff c = a \vee \neg a$ and \textit{stable models} $s \in \fml{S}$.
		93	+
		94	+% In a statistical setting, the outcomes are the literals $x$, $\neg x$ for each atom $x$, the events express a set of possible outcomes (including $\emptyset$, $\set{a, b}$, $\set{a, \neg a, b}$, \textit{etc.}), and worlds are events with no contradictions.
		95	+
		96	+Our path, traced by equations (\ref{eq:prob.total.choice}) and (\ref{eq:prob.stablemodel} --- \ref{eq:prob.events}), starts with the probability of total choices, $\pr{C = c}$, expands it to stable models, $\pr{S = s}$, and then to worlds $\pr{W = w}$ and events $\pr{E = e}$.
86		97
87	\begin{enumerate}	98	\begin{enumerate}
88	- \item \textbf{Given} $\pr{C = c}$ from the specification. Each total choice $C = c$ (together with the specification facts and rules) entails some stable models, $S_c$.
89	- \item \textbf{Given} $\pr{S = s \given C = c}$ such that its value is zero if $s \not\in S_c$.
90	- \item Each event $E = e$ either:	99	+ \item \textbf{Total Choices.} This case is given by $\pr{C = c}$, from equation \ref{eq:prob.total.choice}. Each total choice $C = c$ (together with the facts and rules) entails some stable models, $s \in S_c$, and each stable model $S = s$ contains a single total choice $c_s \subseteq s$.
		100	+ \item \textbf{Stable Models.} Given a stable model $s \in \fml{S}$, and variables/values $x_{s,c} \in \intcc{0, 1}$,
		101	+ \begin{equation}
		102	+ \pr{S = s \given C = c} = \begin{cases}
		103	+ x_{s,c} & \text{if~} s \in S_c,\cr
		104	+ 0&\text{otherwise}
		105	+ \end{cases}
		106	+ \label{eq:prob.stablemodel}
		107	+ \end{equation}
		108	+ such that $\sum_{s \in S_c} x_{s,c} = 1$.
		109	+ \item\label{item:world.cases} \textbf{Worlds.} Each world $W = w$ either:
91	\begin{enumerate}	110	\begin{enumerate}
92	- \item Is a stable model. Then	111	+ \item Is a \textit{stable model}. Then
93	\begin{equation}	112	\begin{equation}
94	- \pr{E = e \given C = c} = \pr{S = e \given C = c}.	113	+ \pr{W = w \given C = c} = \pr{S = s \given C = c}.
		114	+ \label{eq:world.fold.stablemodel}
95	\end{equation}	115	\end{equation}
96	- \item \textit{Is contained} in some stable models. Then	116	+ \item \textit{Contains} some stable models. Then
97	\begin{equation}	117	\begin{equation}
98	- \pr{E = e \given C = c} = \sum_{s \supseteq e}\pr{S = s \given C = c}.	118	+ \pr{W = w \given C = c} = \prod_{s \subset w}\pr{S = s \given C = c}.
		119	+ \label{eq:world.fold.superset}
99	\end{equation}	120	\end{equation}
100	- \item \textit{Contains} some stable models. Then	121	+ \item \textit{Is contained} in some stable models. Then
101	\begin{equation}	122	\begin{equation}
102	- \pr{E = e \given C = c} = \prod_{s \subseteq e}\pr{S = s \given C = c}.	123	+ \pr{W = w \given C = c} = \sum_{s \supset w}\pr{S = s \given C = c}.
		124	+ \label{eq:world.fold.subset}
103	\end{equation}	125	\end{equation}
104	\item Neither contains nor is contained by a stable model. Then	126	\item Neither contains nor is contained by a stable model. Then
105	\begin{equation}	127	\begin{equation}
106	- \pr{E = e \given C = c} = 0.	128	+ \pr{W = w} = 0.
		129	+ \label{eq:world.fold.independent}
107	\end{equation}	130	\end{equation}
108	\end{enumerate}	131	\end{enumerate}
109	- \item For each sample $Z = z$	132	+ \item \textbf{Events.} For each event $E = e$,
110	\begin{equation}	133	\begin{equation}
111	- \pr{Z = z \given C = c} = \begin{cases}
112	- \pr{E = z \given C = c} & z \in \fml{E}, \cr	134	+ \pr{E = e \given C = c} = \begin{cases}
		135	+ \pr{W = e \given C = c} & e \in \fml{W}, \cr
113	0 & \text{otherwise}.	136	0 & \text{otherwise}.
114	\end{cases}	137	\end{cases}
		138	+ \label{eq:prob.events}
115	\end{equation}	139	\end{equation}
116	\end{enumerate}	140	\end{enumerate}
117		141
118	-\begin{quotation}\marginpar{Todo} Prove the four event cases, and support the sum/product options, with the independence assumptions.	142	+Since stable model are minimal, there is no proper chain $s_1 \subset w \subset s_2$ so each world folds into exactly one ot the four cases of point \ref{item:world.cases} above.
		143	+
		144	+% PARAMETERS FOR UNCERTAINTY
		145	+
		146	+Equation (\ref{eq:prob.stablemodel}) expresses the lack of knowledge about the probability assignment when a single total choice entails more than one stable model. In this case, how to distribute the respective probability? Our answer to this problem consists in assigning an unknown probability, $x_{s,c}$, conditional on the total choice, $c$, to each stable model $s$. This approach allow the expression of an unknown quantity and future estimation, given observed data.
		147	+
		148	+% STABLE MODEL
		149	+The stable model case, in equation (\ref{eq:world.fold.stablemodel}), identifies the probability of a stable model \textit{as a world} with its probability as defined previously in equation (\ref{eq:prob.stablemodel}), as a stable model.
		150	+
		151	+% SUPERSET
		152	+Equation \ref{eq:world.fold.superset} results from conditional independence of the stable models $s \subset w$. Conditional independence of stable worlds asserts a least informed strategy that we make explicit:
		153	+
		154	+\begin{assumption}
		155	+ Stable models are conditionally independent, given their total choices.
		156	+\end{assumption}
		157	+
		158	+Consider the stable models $ab, ac$ from the example above. They result from the clause $b \vee c \leftarrow a$ and the total choice $a$. These formulas alone impose no relation between $b$ and $c$ (given $a$), so none should be assumed. Dependence relations are discussed in Subsection (\ref{subsec:dependence}).
		159	+
		160	+% SUBSET
		161	+
		162	+I'm not sure about what to say here.\marginpar{todo}
		163	+
		164	+\begin{equation*}
		165	+ \pr{W = w \given C = c} = \sum_{s \supset w}\pr{S = s \given C = c}.
		166	+\end{equation*}
		167	+
		168	+But! $\pr{W = w \given C = c}$ already separates $\pr{W}$ into \textbf{disjoint}
		169	+
		170	+% INDEPENDENCE
		171	+
		172	+A world that neither contains nor is contained in a stable model describes a case that, according to the specification, should never be observed. So the respective probability is set to zero, per equation (\ref{eq:world.fold.independent}).
		173	+%
		174	+% ================================================================
		175	+%
		176	+\subsection{Dependence}
		177	+\label{subsec:dependence}
		178	+
		179	+Dependence relations in the underlying system can be explicitly expressed in the specification.
		180	+
		181	+For example, $b \leftarrow c \wedge d$, where $d$ is an atomic choice, explicitly expressing this dependence between $b$ and $c$. One would get, for example, the specification
		182	+$$
		183	+0.3::a, b \vee c \leftarrow a, 0.2::d, b \leftarrow c \wedge d.
		184	+$$
		185	+with the stable models
		186	+$
		187	+\co{ad}, \co{a}d, a\co{d}b, a\co{d}c, adb
		188	+$.
		189	+
		190	+
		191	+The interesting case is the subtree of the total choice $ac$. Notice that no stable model $s$ contains $adc$ because (1) $adb$ is a stable model and (2) no stable model contains $adc$, because if $adc \subset s$ then $b \in s$ and $adb \subset s$.
		192	+
		193	+Following the case of equations (\ref{eq:world.fold.stablemodel}) and (\ref{eq:world.fold.independent}) this sets
		194	+\begin{equation*}
		195	+ \begin{cases}
		196	+ \pr{W = adc \given C = ad} = 0,\cr
		197	+ \pr{W = adb \given C = ad} = 1
		198	+ \end{cases}
		199	+\end{equation*}
		200	+which concentrates all probability mass from the total choice $ad$ in the $adb$ branch, including the node $W = adbc$. This leads to the following cases:
		201	+$$
		202	+\begin{array}{l\|r}
		203	+ x & \pr{x \given C = ad}\\
		204	+ \hline
		205	+ ad & 1 \\
		206	+ adb & 1\\
		207	+ adc & 0\\
		208	+ adbc & 1
		209	+\end{array}
		210	+$$
		211	+so, for $C = ad$, $\pr{b} = 2/3, \pr{c} = 1/3, \pr{b,c} = 1/3 \not= \pr{b}\pr{c}$.
		212	+
		213	+\begin{quotation}\note{Todo}
		214	+
		215	+ Prove the four world cases (done), support the product (done) and sum (tbd) options, with the independence assumptions.
119	\end{quotation}	216	\end{quotation}
120		217
		218	+\section{Developed Example}
		219	+
		220	+We continue with the specification from equation \ref{eq:example.1}.
		221	+
		222	+\textbf{Step 1: Total Choices.} The total choices, and respective stable models, are
		223	+\begin{center}
		224	+ \begin{tabular}{l\|r\|r}
		225	+ Total Choice ($c$) & $\pr{C = c}$ & Stable Models ($s$)\\
		226	+ \hline
		227	+ $a$ & $0.3$ & $ab$ and $ac$.\\
		228	+ $\co{a} = \neg a$ & $\co{0.3} = 0.7$ & $\co{a}$.
		229	+ \end{tabular}
		230	+\end{center}
		231	+
		232	+\textbf{Step 2: Stable Models.} Suppose now that
		233	+\begin{center}
		234	+ \begin{tabular}{l\|c\|r}
		235	+ Stable Models ($s$) & Total Choice ($c$) & $\pr{S = c \given C = c}$\\
		236	+ \hline
		237	+ $\co{a}$ & $1.0$ & $\co{a}$. \\
		238	+ $ab$ & $0.8$ & $a$. \\
		239	+ $ac$ & $0.2 = \co{0.8}$ & $a$.
		240	+ \end{tabular}
		241	+\end{center}
		242	+
		243	+\textbf{Step 3: Worlds.} Following equations \ref{eq:world.fold.stablemodel} --- \ref{eq:world.fold.independent} we get:
		244	+\begin{center}
		245	+ \begin{tabular}{l\|c\|l\|c\|r}
		246	+ Occ. ($o$) & S.M. ($s$) & Relation & T.C. ($c$) & $\pr{W = w}$\\
		247	+ \hline
		248	+ $\emptyset$ & all & contained & $a$, $\co{a}$ & $1.0$ \\
		249	+ $a$ & $ab$, $ac$ & contained & $a$ & $0.8\times 0.3 + 0.2\times 0.3 = 0.3$ \\
		250	+ $b$ & $ab$ & contained & $a$ & $0.8\times 0.3 = 0.24$ \\
		251	+ $c$ & $ac$ & contained & $a$ & $0.2\times 0.3 = 0.06$ \\
		252	+ $\co{a}$ & $\co{a}$ & stable model & $\co{a}$ & $1.0\times 0.3 = 0.3$ \\
		253	+ $\co{b}$ & none & independent & none & $0.0$ \\
		254	+ $\co{c}$ & none & \ldots & & \\
		255	+ $ab$ & $ab$ & stable model & $a$ & $0.24$ \\
		256	+ $ac$ & $ac$ & stable model & $a$ & $0.06$ \\
		257	+ $a\co{b}$ & none & \ldots & & \\
		258	+ $a\co{c}$ & none & \ldots & & \\
		259	+ $\co{a}b$ & $\co{a}$ & contains & $\co{a}$ & $1.0$ \\
		260	+ $\co{a}c$ & $\co{a}$ & \ldots & & \\
		261	+ $\co{a}\co{b}$ & $\co{a}$ & \ldots & & \\
		262	+ $\co{a}\co{c}$ & $\co{a}$ & \ldots & & \\
		263	+ $abc$ & $ab$, $ac$ & contains & $a$ & $0.8\times 0.2 = 0.016$ \\
		264	+ \end{tabular}
		265	+\end{center}
121		266
122	-
123	\section*{References}	267	\section*{References}
124		268
125		269