Solved subset explanation.

Francisco Coelho
1 parent 53b3b48c
Showing 5 changed files with 185 additions and 40 deletions Show diff stats
biblio/asp/ASP-CORE-2.03b.pdf
code/asp/disj.lp
code/asp/pqueens.lp
text/pre-paper.pdf
text/pre-paper.tex
+% prob(a) = 0.3
 a ; -a.
 b ; c :- a.
 \ No newline at end of file
@@ -9,12 +9,12 @@ row(1 .. n).
 diag1(I, J, I - J + n) :- col(I), row(J).
 % Number ascending diagonals.
 diag2(I, J, I + J - 1) :- col(I), row(J).
-
-% WTF?
+%
+% Negative Restrictions
 :- D = 1 .. 2 * n - 1, not { queen(I, J) : diag1(I, J, D) } 1.
 :- D = 1 .. 2 * n - 1, not { queen(I, J) : diag2(I, J, D) } 1.
-
+%
 % Output this predicate.
 #show queen/2.
-#show diag1/3.
-#show diag2/3. 
 \ No newline at end of file
+%#show diag1/3.
+%#show diag2/3. 
 \ No newline at end of file
-\documentclass[a4paper]{article}
+\documentclass[a4paper, 12pt]{article}
+
+\usepackage[x11colors]{xcolor}
 \usepackage{hyperref}
 \hypersetup{
     colorlinks=true,
     linkcolor=blue,
 }
+
 \usepackage{commath}
+\usepackage{amsthm}
+\newtheorem{assumption}{Assumption}
 \usepackage{amssymb}
 %
 % Local commands
 %
-\newcommand{\todo}[1]{{\color{orange}TODO #1}}
+\newcommand{\note}[1]{\marginpar{\scriptsize #1}}
 \newcommand{\naf}{\ensuremath{\sim\!}}
 \newcommand{\larr}{\ensuremath{\leftarrow}}
 \newcommand{\at}[1]{\ensuremath{\!\del{#1}}}
@@ -22,7 +27,7 @@
 \newcommand{\langof}[1]{\ensuremath{\fml{L}\at{#1}}}
 \newcommand{\uset}[1]{\ensuremath{\left|{#1}\right>}}
 \newcommand{\lset}[1]{\ensuremath{\left<{#1}\right|}}
-\newcommand{\pr}[1]{\ensuremath{\mathrm{p}\at{#1}}}
+\newcommand{\pr}[1]{\ensuremath{\mathrm{P}\at{#1}}}
 \newcommand{\given}{\ensuremath{~\middle|~}}
  
 \title{Zugzwang\\\textit{Logic and Artificial Intelligence}}
@@ -42,84 +47,223 @@ Answer Set Programming (ASP) is a logic programming paradigm based on the Stable
 The Distribution Semantics (DS) is a key approach to extend logical representations with probabilistic reasoning. Probabilistic Facts (PF) are the most basic stochastic DS primitive and they take the form of logical facts, $a$, labelled with a probability, such as $p::a$; Each probabilistic fact represents a boolean random variable that is true with probability $p$ and false with probability $1 - p$. A (consistent) combination of the PFs defines a \textit{total choice} $c = \set{p::a, \ldots}$ such that
  
 \begin{equation}
-    P(c) = \prod_{a\in c} p \prod_{a \not\in c} (1- p).
+    \pr{C = x} = \prod_{a\in c} p \prod_{a \not\in c} (1- p).
+    \label{eq:prob.total.choice}
 \end{equation}
  
 Our goal is to extend this probability, from total choices, to cover the specification domain. We can foresee two key applications of this extended probability:
  
 \begin{enumerate}
     \item Support any probabilistic reasoning/task on the specification domain.
-    \item Also, given a dataset and a divergence measure, now the specification can be scored (by the divergence w.r.t. the \emph{empiric} distribution of the dataset), and sorted amongst other specifications. This is a key ingredient in algorithms searching, for example, an \textit{optimal specification} of the dataset. 
+    \item Also, given a dataset and a divergence measure, now the specification can be scored (by the divergence w.r.t.\ the \emph{empiric} distribution of the dataset), and sorted amongst other specifications. This is a key ingredient in algorithms searching, for example, an optimal specification of the dataset. 
 \end{enumerate}
  
-This goal faces a critical problem concerning situations where \textit{multiple} standard models result from a given total choice, illustrated by the following example. The specification 
-$$
-\begin{aligned}
-    0.3::a&,\cr
-    b \vee c& \leftarrow a.
-\end{aligned}
-$$
-has three stable models, $\set{\neg a}, \set{a, b}$ and $\set{a, c}$. While it is straightforward to set $P(\neg a)=0.7$, there is \textit{no further information} to assign values to $P(a,b)$ and $P(a,c)$. At best, we can use a parameter $\alpha$ such that
+This goal faces a critical problem concerning situations where multiple standard models result from a given total choice, illustrated by the following example. The specification
+\begin{equation}
+    \begin{aligned}
+        0.3::a&,\cr
+        b \vee c& \leftarrow a.
+    \end{aligned}
+    \label{eq:example.1}
+\end{equation}
+has three stable models, $\co{a}, ab$ and $ac$. While it is straightforward to set $P(\co{a})=0.7$, there is \textit{no further information} to assign values to $P(ab)$ and $P(ac)$. At best, we can use a parameter $x$ such that
 $$
 \begin{aligned}
-P(a,b) &= 0.3 \alpha,\cr
-P(a,c) &= 0.3 (1 - \alpha).
+P(ab) &= 0.3 x,\cr
+P(ac) &= 0.3 (1 - x).
 \end{aligned}
 $$
  
-This uncertainty in inherent to the specification, but can be mitigated with the help of a dataset: the parameter $\alpha$ can be estimated from the empirical distribution.
+This uncertainty in inherent to the specification, but can be mitigated with the help of a dataset: the parameter $x$ can be estimated from the empirical distribution.
  
-In summary, if an ASP specification is intended to describe some system that can be observed then:
+In summary, if an ASP specification is intended to describe some observable system then:
  
 \begin{enumerate}
-    \item The observations can be used to estimate the value of the parameters (such as $\alpha$ above and others entailed from further clauses).
+    \item The observations can be used to estimate the value of the parameters (such as $x$ above and others entailed from further clauses).
     \item With a probability set for the stable models, we want to extend it to all the samples (\textit{i.e.} consistent sets of literals) of the specification. 
     \item This extended probability can then be related to the \textit{empirical distribution}, using a probability divergence, such as Kullback-Leibler; and the divergence value used as a \textit{performance} measure of the specification with respect to the observations.
     \item If that specification is only but one of many possible candidates then that performance measure can be used, \textit{e.g.} as fitness, by algorithms searching (optimal) specifications of a dataset of observations.
 \end{enumerate}
  
-Currently, we are on the step two above: Extending a probability function (with parameters such as $\alpha$), defined on the stable sets of a specification, to all the samples of the specification. This extension must, of course, respect the axioms of probability so that probabilistic reasoning is consistent with the ASP specification. 
+Currently, we are on the step two above: Extending a probability function (with parameters such as $x$), defined on the stable sets of a specification, to all the events of the specification. This extension must, of course, respect the axioms of probability so that probabilistic reasoning is consistent with the ASP specification. 
  
 \section{Extending Probabilities}
  
-Given an ASP specification, we look at the \textit{total choices} $\fml{C}$, \textit{stable models} $\fml{S}$, \textit{atoms} and \textit{literals} $\fml{A}, \fml{L}$, \textit{samples} $z \in \fml{Z} \iff z \subseteq \fml{L}$ and \textit{events} $\fml{E}$ (consistent samples).
+Given an ASP specification, we consider the \textit{atoms} $a \in \fml{A}$ and \textit{literals}, $z \in \fml{L}$, \textit{events} $e \in \fml{E} \iff e \subseteq \fml{L}$ and \textit{worlds} $w \in \fml{W}$ (consistent events), \textit{total choices} $c \in \fml{C} \iff c = a \vee \neg a$ and \textit{stable models} $s \in \fml{S}$.
+
+% In a statistical setting, the outcomes are the literals $x$, $\neg x$ for each atom $x$, the events express a set of possible outcomes (including $\emptyset$, $\set{a, b}$, $\set{a, \neg a, b}$, \textit{etc.}), and worlds are events with no contradictions.
+
+Our path, traced by equations (\ref{eq:prob.total.choice}) and (\ref{eq:prob.stablemodel} --- \ref{eq:prob.events}), starts with the probability of total choices, $\pr{C = c}$, expands it to stable models, $\pr{S = s}$, and then to worlds $\pr{W = w}$ and events $\pr{E = e}$.
  
 \begin{enumerate}
-    \item \textbf{Given} $\pr{C = c}$ from the specification. Each total choice $C = c$ (together with the specification facts and rules) entails some stable models, $S_c$.
-    \item \textbf{Given} $\pr{S = s \given C = c}$ such that its value is zero if $s \not\in S_c$.
-    \item Each event $E = e$ either:
+    \item \textbf{Total Choices.} This case is given by $\pr{C = c}$, from equation \ref{eq:prob.total.choice}. Each total choice $C = c$ (together with the facts and rules) entails some stable models, $s \in S_c$, and each stable model $S = s$ contains a single total choice $c_s \subseteq s$.
+    \item \textbf{Stable Models.} Given a stable model $s \in \fml{S}$, and variables/values $x_{s,c} \in \intcc{0, 1}$, 
+    \begin{equation}
+        \pr{S = s \given C = c} = \begin{cases}
+            x_{s,c} & \text{if~} s \in S_c,\cr
+            0&\text{otherwise}
+        \end{cases}
+        \label{eq:prob.stablemodel}
+    \end{equation}
+    such that $\sum_{s \in S_c} x_{s,c} = 1$.
+    \item\label{item:world.cases} \textbf{Worlds.} Each world $W  = w$ either:
     \begin{enumerate}
-        \item Is a stable model. Then
+        \item Is a \textit{stable model}. Then 
         \begin{equation}
-            \pr{E = e \given C = c} = \pr{S = e \given C = c}.
+            \pr{W  = w \given C = c} = \pr{S = s \given C = c}.
+            \label{eq:world.fold.stablemodel}
         \end{equation}
-        \item \textit{Is contained} in some stable models. Then
+        \item \textit{Contains} some stable models. Then
         \begin{equation}
-            \pr{E = e \given C = c} = \sum_{s \supseteq e}\pr{S = s \given C = c}.
+            \pr{W  = w \given C = c} = \prod_{s \subset w}\pr{S = s \given C = c}.
+            \label{eq:world.fold.superset}
         \end{equation}
-        \item \textit{Contains} some stable models. Then
+        \item \textit{Is contained} in some stable models. Then
         \begin{equation}
-            \pr{E = e \given C = c} = \prod_{s \subseteq e}\pr{S = s \given C = c}.
+            \pr{W  = w \given C = c} = \sum_{s \supset w}\pr{S = s \given C = c}.
+            \label{eq:world.fold.subset}
         \end{equation}
         \item Neither contains nor is contained by a stable model. Then
         \begin{equation}
-            \pr{E = e \given C = c} = 0.
+            \pr{W  = w} = 0.
+            \label{eq:world.fold.independent}
         \end{equation}
     \end{enumerate}
-    \item For each sample $Z = z$
+    \item \textbf{Events.} For each event $E = e$,
     \begin{equation}
-        \pr{Z = z \given C = c} = \begin{cases}
-            \pr{E = z \given C = c} & z \in \fml{E}, \cr
+        \pr{E = e \given C = c} = \begin{cases}
+            \pr{W = e \given C = c} & e \in \fml{W}, \cr
             0 & \text{otherwise}.
             \end{cases}
+        \label{eq:prob.events}
     \end{equation}
 \end{enumerate}
  
-\begin{quotation}\marginpar{Todo} Prove the four event cases, and support the sum/product options, with the independence assumptions.
+Since stable model are minimal, there is no proper chain $s_1 \subset w \subset s_2$ so each world folds into exactly one ot the four cases of point \ref{item:world.cases} above.
+
+% PARAMETERS FOR UNCERTAINTY
+
+Equation (\ref{eq:prob.stablemodel}) expresses the lack of knowledge about the probability assignment when a single total choice entails more than one stable model. In this case, how to distribute the respective probability? Our answer to this problem consists in assigning an unknown probability, $x_{s,c}$, conditional on the total choice, $c$, to each stable model $s$. This approach allow the expression of an unknown quantity and future estimation, given observed data.
+
+% STABLE MODEL
+The stable model case, in equation (\ref{eq:world.fold.stablemodel}), identifies the probability of a stable model \textit{as a world} with its probability as defined previously in equation (\ref{eq:prob.stablemodel}), as a stable model.
+
+% SUPERSET
+Equation \ref{eq:world.fold.superset} results from conditional independence of the stable models $s \subset w$. Conditional independence of stable worlds asserts a least informed strategy that we make explicit:
+
+\begin{assumption}
+    Stable models are conditionally independent, given their total choices.
+\end{assumption}
+
+Consider the stable models $ab, ac$ from the example above. They result from the clause $b \vee c \leftarrow a$ and the total choice $a$. These formulas alone impose no relation between $b$ and $c$ (given $a$), so none should be assumed. Dependence relations are discussed in Subsection (\ref{subsec:dependence}).
+
+% SUBSET
+
+I'm not sure about what to say here.\marginpar{todo}
+
+\begin{equation*}
+    \pr{W  = w \given C = c} = \sum_{s \supset w}\pr{S = s \given C = c}.
+\end{equation*}
+
+But! $\pr{W = w \given C = c}$ already separates $\pr{W}$ into \textbf{disjoint}
+
+% INDEPENDENCE
+
+A world that neither contains nor is contained in a stable model describes a case that, according to the specification, should never be observed. So the respective probability is set to zero, per equation (\ref{eq:world.fold.independent}).
+%
+%   ================================================================
+%
+\subsection{Dependence}
+\label{subsec:dependence}
+
+Dependence relations in the underlying system can be explicitly expressed in the specification. 
+
+For example, $b \leftarrow c \wedge d$, where $d$ is an atomic choice, explicitly expressing this dependence between $b$ and $c$. One would get, for example, the specification
+$$
+0.3::a, b \vee c \leftarrow a, 0.2::d, b \leftarrow c \wedge d.
+$$
+with the stable models
+$
+\co{ad}, \co{a}d, a\co{d}b, a\co{d}c, adb
+$. 
+
+
+The interesting case is the subtree of the total choice $ac$. Notice that no stable model $s$ contains $adc$ because (1)  $adb$ is a stable model and (2) no stable model contains $adc$, because if $adc \subset s$ then $b \in s$ and $adb \subset s$.
+
+Following the case of equations (\ref{eq:world.fold.stablemodel}) and (\ref{eq:world.fold.independent}) this sets 
+\begin{equation*}
+    \begin{cases}
+        \pr{W = adc \given C = ad} = 0,\cr
+        \pr{W = adb \given C = ad} = 1
+    \end{cases}
+\end{equation*}
+which concentrates all probability mass from the total choice $ad$ in the $adb$ branch, including the node $W = adbc$. This leads to the following cases:
+$$
+\begin{array}{l|r}
+    x & \pr{x \given C = ad}\\
+    \hline
+    ad & 1 \\
+    adb & 1\\
+    adc & 0\\
+    adbc & 1    
+\end{array}
+$$
+so, for $C = ad$, $\pr{b} = 2/3, \pr{c} = 1/3, \pr{b,c} = 1/3 \not= \pr{b}\pr{c}$.
+
+\begin{quotation}\note{Todo}
+    
+    Prove the four world cases (done), support the product (done) and sum (tbd) options, with the independence assumptions.
 \end{quotation}
  
+\section{Developed Example}
+
+We continue with the specification from equation \ref{eq:example.1}. 
+
+\textbf{Step 1: Total Choices.} The total choices, and respective stable models, are
+\begin{center}
+    \begin{tabular}{l|r|r}
+        Total Choice ($c$) & $\pr{C = c}$ & Stable Models ($s$)\\
+        \hline 
+        $a$ & $0.3$ & $ab$ and $ac$.\\
+        $\co{a} = \neg a$ & $\co{0.3} = 0.7$ & $\co{a}$.
+    \end{tabular}
+\end{center}
+
+\textbf{Step 2: Stable Models.} Suppose now that 
+\begin{center}
+    \begin{tabular}{l|c|r}
+        Stable Models ($s$) & Total Choice ($c$)  & $\pr{S = c \given C = c}$\\
+        \hline 
+        $\co{a}$    & $1.0$ & $\co{a}$. \\
+        $ab$        & $0.8$ & $a$. \\
+        $ac$        & $0.2 = \co{0.8}$ & $a$.
+    \end{tabular}
+\end{center}
+
+\textbf{Step 3: Worlds.} Following equations \ref{eq:world.fold.stablemodel} --- \ref{eq:world.fold.independent} we get:
+\begin{center}
+    \begin{tabular}{l|c|l|c|r}
+        Occ. ($o$) & S.M. ($s$) & Relation & T.C. ($c$)  & $\pr{W  = w}$\\
+        \hline 
+        $\emptyset$ & all           & contained & $a$, $\co{a}$ & $1.0$ \\ 
+        $a$         & $ab$, $ac$    & contained & $a$ & $0.8\times 0.3 + 0.2\times 0.3 = 0.3$ \\
+        $b$         & $ab$          & contained & $a$ & $0.8\times 0.3 = 0.24$ \\ 
+        $c$         & $ac$          & contained & $a$ & $0.2\times 0.3 = 0.06$ \\ 
+        $\co{a}$    & $\co{a}$      & stable model & $\co{a}$ & $1.0\times 0.3 = 0.3$ \\
+        $\co{b}$    & none          & independent & none & $0.0$ \\
+        $\co{c}$    & none              & \ldots & & \\
+        $ab$        & $ab$          & stable model & $a$ & $0.24$ \\  
+        $ac$        & $ac$          & stable model & $a$ & $0.06$ \\  
+        $a\co{b}$   & none              & \ldots & & \\
+        $a\co{c}$   & none              & \ldots & & \\
+        $\co{a}b$   & $\co{a}$      & contains & $\co{a}$ & $1.0$ \\
+        $\co{a}c$   &  $\co{a}$         & \ldots & & \\
+        $\co{a}\co{b}$  & $\co{a}$      & \ldots & & \\
+        $\co{a}\co{c}$  & $\co{a}$      & \ldots & & \\
+        $abc$       & $ab$, $ac$    & contains & $a$ & $0.8\times 0.2 = 0.016$ \\
+    \end{tabular}
+\end{center}
  
- 
 \section*{References}
	1	+% prob(a) = 0.3
1	2	a ; -a.
2	3	b ; c :- a.
3	4	\ No newline at end of file
...	...
...	...	@@ -9,12 +9,12 @@ row(1 .. n).
9	9	diag1(I, J, I - J + n) :- col(I), row(J).
10	10	% Number ascending diagonals.
11	11	diag2(I, J, I + J - 1) :- col(I), row(J).
12		-
13		-% WTF?
	12	+%
	13	+% Negative Restrictions
14	14	:- D = 1 .. 2 * n - 1, not { queen(I, J) : diag1(I, J, D) } 1.
15	15	:- D = 1 .. 2 * n - 1, not { queen(I, J) : diag2(I, J, D) } 1.
16		-
	16	+%
17	17	% Output this predicate.
18	18	#show queen/2.
19		-#show diag1/3.
20		-#show diag2/3.
21	19	\ No newline at end of file
	20	+%#show diag1/3.
	21	+%#show diag2/3.
22	22	\ No newline at end of file
...	...
1		-\documentclass[a4paper]{article}
	1	+\documentclass[a4paper, 12pt]{article}
	2	+
	3	+\usepackage[x11colors]{xcolor}
2	4	\usepackage{hyperref}
3	5	\hypersetup{
4	6	colorlinks=true,
5	7	linkcolor=blue,
6	8	}
	9	+
7	10	\usepackage{commath}
	11	+\usepackage{amsthm}
	12	+\newtheorem{assumption}{Assumption}
8	13	\usepackage{amssymb}
9	14	%
10	15	% Local commands
11	16	%
12		-\newcommand{\todo}[1]{{\color{orange}TODO #1}}
	17	+\newcommand{\note}[1]{\marginpar{\scriptsize #1}}
13	18	\newcommand{\naf}{\ensuremath{\sim\!}}
14	19	\newcommand{\larr}{\ensuremath{\leftarrow}}
15	20	\newcommand{\at}[1]{\ensuremath{\!\del{#1}}}
...	...	@@ -22,7 +27,7 @@
22	27	\newcommand{\langof}[1]{\ensuremath{\fml{L}\at{#1}}}
23	28	\newcommand{\uset}[1]{\ensuremath{\left\|{#1}\right>}}
24	29	\newcommand{\lset}[1]{\ensuremath{\left<{#1}\right\|}}
25		-\newcommand{\pr}[1]{\ensuremath{\mathrm{p}\at{#1}}}
	30	+\newcommand{\pr}[1]{\ensuremath{\mathrm{P}\at{#1}}}
26	31	\newcommand{\given}{\ensuremath{~\middle\|~}}
27	32
28	33	\title{Zugzwang\\\textit{Logic and Artificial Intelligence}}
...	...	@@ -42,84 +47,223 @@ Answer Set Programming (ASP) is a logic programming paradigm based on the Stable
42	47	The Distribution Semantics (DS) is a key approach to extend logical representations with probabilistic reasoning. Probabilistic Facts (PF) are the most basic stochastic DS primitive and they take the form of logical facts, $a$, labelled with a probability, such as $p::a$; Each probabilistic fact represents a boolean random variable that is true with probability $p$ and false with probability $1 - p$. A (consistent) combination of the PFs defines a \textit{total choice} $c = \set{p::a, \ldots}$ such that
43	48
44	49	\begin{equation}
45		- P(c) = \prod_{a\in c} p \prod_{a \not\in c} (1- p).
	50	+ \pr{C = x} = \prod_{a\in c} p \prod_{a \not\in c} (1- p).
	51	+ \label{eq:prob.total.choice}
46	52	\end{equation}
47	53
48	54	Our goal is to extend this probability, from total choices, to cover the specification domain. We can foresee two key applications of this extended probability:
49	55
50	56	\begin{enumerate}
51	57	\item Support any probabilistic reasoning/task on the specification domain.
52		- \item Also, given a dataset and a divergence measure, now the specification can be scored (by the divergence w.r.t. the \emph{empiric} distribution of the dataset), and sorted amongst other specifications. This is a key ingredient in algorithms searching, for example, an \textit{optimal specification} of the dataset.
	58	+ \item Also, given a dataset and a divergence measure, now the specification can be scored (by the divergence w.r.t.\ the \emph{empiric} distribution of the dataset), and sorted amongst other specifications. This is a key ingredient in algorithms searching, for example, an optimal specification of the dataset.
53	59	\end{enumerate}
54	60
55		-This goal faces a critical problem concerning situations where \textit{multiple} standard models result from a given total choice, illustrated by the following example. The specification
56		-$$
57		-\begin{aligned}
58		- 0.3::a&,\cr
59		- b \vee c& \leftarrow a.
60		-\end{aligned}
61		-$$
62		-has three stable models, $\set{\neg a}, \set{a, b}$ and $\set{a, c}$. While it is straightforward to set $P(\neg a)=0.7$, there is \textit{no further information} to assign values to $P(a,b)$ and $P(a,c)$. At best, we can use a parameter $\alpha$ such that
	61	+This goal faces a critical problem concerning situations where multiple standard models result from a given total choice, illustrated by the following example. The specification
	62	+\begin{equation}
	63	+ \begin{aligned}
	64	+ 0.3::a&,\cr
	65	+ b \vee c& \leftarrow a.
	66	+ \end{aligned}
	67	+ \label{eq:example.1}
	68	+\end{equation}
	69	+has three stable models, $\co{a}, ab$ and $ac$. While it is straightforward to set $P(\co{a})=0.7$, there is \textit{no further information} to assign values to $P(ab)$ and $P(ac)$. At best, we can use a parameter $x$ such that
63	70	$$
64	71	\begin{aligned}
65		-P(a,b) &= 0.3 \alpha,\cr
66		-P(a,c) &= 0.3 (1 - \alpha).
	72	+P(ab) &= 0.3 x,\cr
	73	+P(ac) &= 0.3 (1 - x).
67	74	\end{aligned}
68	75	$$
69	76
70		-This uncertainty in inherent to the specification, but can be mitigated with the help of a dataset: the parameter $\alpha$ can be estimated from the empirical distribution.
	77	+This uncertainty in inherent to the specification, but can be mitigated with the help of a dataset: the parameter $x$ can be estimated from the empirical distribution.
71	78
72		-In summary, if an ASP specification is intended to describe some system that can be observed then:
	79	+In summary, if an ASP specification is intended to describe some observable system then:
73	80
74	81	\begin{enumerate}
75		- \item The observations can be used to estimate the value of the parameters (such as $\alpha$ above and others entailed from further clauses).
	82	+ \item The observations can be used to estimate the value of the parameters (such as $x$ above and others entailed from further clauses).
76	83	\item With a probability set for the stable models, we want to extend it to all the samples (\textit{i.e.} consistent sets of literals) of the specification.
77	84	\item This extended probability can then be related to the \textit{empirical distribution}, using a probability divergence, such as Kullback-Leibler; and the divergence value used as a \textit{performance} measure of the specification with respect to the observations.
78	85	\item If that specification is only but one of many possible candidates then that performance measure can be used, \textit{e.g.} as fitness, by algorithms searching (optimal) specifications of a dataset of observations.
79	86	\end{enumerate}
80	87
81		-Currently, we are on the step two above: Extending a probability function (with parameters such as $\alpha$), defined on the stable sets of a specification, to all the samples of the specification. This extension must, of course, respect the axioms of probability so that probabilistic reasoning is consistent with the ASP specification.
	88	+Currently, we are on the step two above: Extending a probability function (with parameters such as $x$), defined on the stable sets of a specification, to all the events of the specification. This extension must, of course, respect the axioms of probability so that probabilistic reasoning is consistent with the ASP specification.
82	89
83	90	\section{Extending Probabilities}
84	91
85		-Given an ASP specification, we look at the \textit{total choices} $\fml{C}$, \textit{stable models} $\fml{S}$, \textit{atoms} and \textit{literals} $\fml{A}, \fml{L}$, \textit{samples} $z \in \fml{Z} \iff z \subseteq \fml{L}$ and \textit{events} $\fml{E}$ (consistent samples).
	92	+Given an ASP specification, we consider the \textit{atoms} $a \in \fml{A}$ and \textit{literals}, $z \in \fml{L}$, \textit{events} $e \in \fml{E} \iff e \subseteq \fml{L}$ and \textit{worlds} $w \in \fml{W}$ (consistent events), \textit{total choices} $c \in \fml{C} \iff c = a \vee \neg a$ and \textit{stable models} $s \in \fml{S}$.
	93	+
	94	+% In a statistical setting, the outcomes are the literals $x$, $\neg x$ for each atom $x$, the events express a set of possible outcomes (including $\emptyset$, $\set{a, b}$, $\set{a, \neg a, b}$, \textit{etc.}), and worlds are events with no contradictions.
	95	+
	96	+Our path, traced by equations (\ref{eq:prob.total.choice}) and (\ref{eq:prob.stablemodel} --- \ref{eq:prob.events}), starts with the probability of total choices, $\pr{C = c}$, expands it to stable models, $\pr{S = s}$, and then to worlds $\pr{W = w}$ and events $\pr{E = e}$.
86	97
87	98	\begin{enumerate}
88		- \item \textbf{Given} $\pr{C = c}$ from the specification. Each total choice $C = c$ (together with the specification facts and rules) entails some stable models, $S_c$.
89		- \item \textbf{Given} $\pr{S = s \given C = c}$ such that its value is zero if $s \not\in S_c$.
90		- \item Each event $E = e$ either:
	99	+ \item \textbf{Total Choices.} This case is given by $\pr{C = c}$, from equation \ref{eq:prob.total.choice}. Each total choice $C = c$ (together with the facts and rules) entails some stable models, $s \in S_c$, and each stable model $S = s$ contains a single total choice $c_s \subseteq s$.
	100	+ \item \textbf{Stable Models.} Given a stable model $s \in \fml{S}$, and variables/values $x_{s,c} \in \intcc{0, 1}$,
	101	+ \begin{equation}
	102	+ \pr{S = s \given C = c} = \begin{cases}
	103	+ x_{s,c} & \text{if~} s \in S_c,\cr
	104	+ 0&\text{otherwise}
	105	+ \end{cases}
	106	+ \label{eq:prob.stablemodel}
	107	+ \end{equation}
	108	+ such that $\sum_{s \in S_c} x_{s,c} = 1$.
	109	+ \item\label{item:world.cases} \textbf{Worlds.} Each world $W = w$ either:
91	110	\begin{enumerate}
92		- \item Is a stable model. Then
	111	+ \item Is a \textit{stable model}. Then
93	112	\begin{equation}
94		- \pr{E = e \given C = c} = \pr{S = e \given C = c}.
	113	+ \pr{W = w \given C = c} = \pr{S = s \given C = c}.
	114	+ \label{eq:world.fold.stablemodel}
95	115	\end{equation}
96		- \item \textit{Is contained} in some stable models. Then
	116	+ \item \textit{Contains} some stable models. Then
97	117	\begin{equation}
98		- \pr{E = e \given C = c} = \sum_{s \supseteq e}\pr{S = s \given C = c}.
	118	+ \pr{W = w \given C = c} = \prod_{s \subset w}\pr{S = s \given C = c}.
	119	+ \label{eq:world.fold.superset}
99	120	\end{equation}
100		- \item \textit{Contains} some stable models. Then
	121	+ \item \textit{Is contained} in some stable models. Then
101	122	\begin{equation}
102		- \pr{E = e \given C = c} = \prod_{s \subseteq e}\pr{S = s \given C = c}.
	123	+ \pr{W = w \given C = c} = \sum_{s \supset w}\pr{S = s \given C = c}.
	124	+ \label{eq:world.fold.subset}
103	125	\end{equation}
104	126	\item Neither contains nor is contained by a stable model. Then
105	127	\begin{equation}
106		- \pr{E = e \given C = c} = 0.
	128	+ \pr{W = w} = 0.
	129	+ \label{eq:world.fold.independent}
107	130	\end{equation}
108	131	\end{enumerate}
109		- \item For each sample $Z = z$
	132	+ \item \textbf{Events.} For each event $E = e$,
110	133	\begin{equation}
111		- \pr{Z = z \given C = c} = \begin{cases}
112		- \pr{E = z \given C = c} & z \in \fml{E}, \cr
	134	+ \pr{E = e \given C = c} = \begin{cases}
	135	+ \pr{W = e \given C = c} & e \in \fml{W}, \cr
113	136	0 & \text{otherwise}.
114	137	\end{cases}
	138	+ \label{eq:prob.events}
115	139	\end{equation}
116	140	\end{enumerate}
117	141
118		-\begin{quotation}\marginpar{Todo} Prove the four event cases, and support the sum/product options, with the independence assumptions.
	142	+Since stable model are minimal, there is no proper chain $s_1 \subset w \subset s_2$ so each world folds into exactly one ot the four cases of point \ref{item:world.cases} above.
	143	+
	144	+% PARAMETERS FOR UNCERTAINTY
	145	+
	146	+Equation (\ref{eq:prob.stablemodel}) expresses the lack of knowledge about the probability assignment when a single total choice entails more than one stable model. In this case, how to distribute the respective probability? Our answer to this problem consists in assigning an unknown probability, $x_{s,c}$, conditional on the total choice, $c$, to each stable model $s$. This approach allow the expression of an unknown quantity and future estimation, given observed data.
	147	+
	148	+% STABLE MODEL
	149	+The stable model case, in equation (\ref{eq:world.fold.stablemodel}), identifies the probability of a stable model \textit{as a world} with its probability as defined previously in equation (\ref{eq:prob.stablemodel}), as a stable model.
	150	+
	151	+% SUPERSET
	152	+Equation \ref{eq:world.fold.superset} results from conditional independence of the stable models $s \subset w$. Conditional independence of stable worlds asserts a least informed strategy that we make explicit:
	153	+
	154	+\begin{assumption}
	155	+ Stable models are conditionally independent, given their total choices.
	156	+\end{assumption}
	157	+
	158	+Consider the stable models $ab, ac$ from the example above. They result from the clause $b \vee c \leftarrow a$ and the total choice $a$. These formulas alone impose no relation between $b$ and $c$ (given $a$), so none should be assumed. Dependence relations are discussed in Subsection (\ref{subsec:dependence}).
	159	+
	160	+% SUBSET
	161	+
	162	+I'm not sure about what to say here.\marginpar{todo}
	163	+
	164	+\begin{equation*}
	165	+ \pr{W = w \given C = c} = \sum_{s \supset w}\pr{S = s \given C = c}.
	166	+\end{equation*}
	167	+
	168	+But! $\pr{W = w \given C = c}$ already separates $\pr{W}$ into \textbf{disjoint}
	169	+
	170	+% INDEPENDENCE
	171	+
	172	+A world that neither contains nor is contained in a stable model describes a case that, according to the specification, should never be observed. So the respective probability is set to zero, per equation (\ref{eq:world.fold.independent}).
	173	+%
	174	+% ================================================================
	175	+%
	176	+\subsection{Dependence}
	177	+\label{subsec:dependence}
	178	+
	179	+Dependence relations in the underlying system can be explicitly expressed in the specification.
	180	+
	181	+For example, $b \leftarrow c \wedge d$, where $d$ is an atomic choice, explicitly expressing this dependence between $b$ and $c$. One would get, for example, the specification
	182	+$$
	183	+0.3::a, b \vee c \leftarrow a, 0.2::d, b \leftarrow c \wedge d.
	184	+$$
	185	+with the stable models
	186	+$
	187	+\co{ad}, \co{a}d, a\co{d}b, a\co{d}c, adb
	188	+$.
	189	+
	190	+
	191	+The interesting case is the subtree of the total choice $ac$. Notice that no stable model $s$ contains $adc$ because (1) $adb$ is a stable model and (2) no stable model contains $adc$, because if $adc \subset s$ then $b \in s$ and $adb \subset s$.
	192	+
	193	+Following the case of equations (\ref{eq:world.fold.stablemodel}) and (\ref{eq:world.fold.independent}) this sets
	194	+\begin{equation*}
	195	+ \begin{cases}
	196	+ \pr{W = adc \given C = ad} = 0,\cr
	197	+ \pr{W = adb \given C = ad} = 1
	198	+ \end{cases}
	199	+\end{equation*}
	200	+which concentrates all probability mass from the total choice $ad$ in the $adb$ branch, including the node $W = adbc$. This leads to the following cases:
	201	+$$
	202	+\begin{array}{l\|r}
	203	+ x & \pr{x \given C = ad}\\
	204	+ \hline
	205	+ ad & 1 \\
	206	+ adb & 1\\
	207	+ adc & 0\\
	208	+ adbc & 1
	209	+\end{array}
	210	+$$
	211	+so, for $C = ad$, $\pr{b} = 2/3, \pr{c} = 1/3, \pr{b,c} = 1/3 \not= \pr{b}\pr{c}$.
	212	+
	213	+\begin{quotation}\note{Todo}
	214	+
	215	+ Prove the four world cases (done), support the product (done) and sum (tbd) options, with the independence assumptions.
119	216	\end{quotation}
120	217
	218	+\section{Developed Example}
	219	+
	220	+We continue with the specification from equation \ref{eq:example.1}.
	221	+
	222	+\textbf{Step 1: Total Choices.} The total choices, and respective stable models, are
	223	+\begin{center}
	224	+ \begin{tabular}{l\|r\|r}
	225	+ Total Choice ($c$) & $\pr{C = c}$ & Stable Models ($s$)\\
	226	+ \hline
	227	+ $a$ & $0.3$ & $ab$ and $ac$.\\
	228	+ $\co{a} = \neg a$ & $\co{0.3} = 0.7$ & $\co{a}$.
	229	+ \end{tabular}
	230	+\end{center}
	231	+
	232	+\textbf{Step 2: Stable Models.} Suppose now that
	233	+\begin{center}
	234	+ \begin{tabular}{l\|c\|r}
	235	+ Stable Models ($s$) & Total Choice ($c$) & $\pr{S = c \given C = c}$\\
	236	+ \hline
	237	+ $\co{a}$ & $1.0$ & $\co{a}$. \\
	238	+ $ab$ & $0.8$ & $a$. \\
	239	+ $ac$ & $0.2 = \co{0.8}$ & $a$.
	240	+ \end{tabular}
	241	+\end{center}
	242	+
	243	+\textbf{Step 3: Worlds.} Following equations \ref{eq:world.fold.stablemodel} --- \ref{eq:world.fold.independent} we get:
	244	+\begin{center}
	245	+ \begin{tabular}{l\|c\|l\|c\|r}
	246	+ Occ. ($o$) & S.M. ($s$) & Relation & T.C. ($c$) & $\pr{W = w}$\\
	247	+ \hline
	248	+ $\emptyset$ & all & contained & $a$, $\co{a}$ & $1.0$ \\
	249	+ $a$ & $ab$, $ac$ & contained & $a$ & $0.8\times 0.3 + 0.2\times 0.3 = 0.3$ \\
	250	+ $b$ & $ab$ & contained & $a$ & $0.8\times 0.3 = 0.24$ \\
	251	+ $c$ & $ac$ & contained & $a$ & $0.2\times 0.3 = 0.06$ \\
	252	+ $\co{a}$ & $\co{a}$ & stable model & $\co{a}$ & $1.0\times 0.3 = 0.3$ \\
	253	+ $\co{b}$ & none & independent & none & $0.0$ \\
	254	+ $\co{c}$ & none & \ldots & & \\
	255	+ $ab$ & $ab$ & stable model & $a$ & $0.24$ \\
	256	+ $ac$ & $ac$ & stable model & $a$ & $0.06$ \\
	257	+ $a\co{b}$ & none & \ldots & & \\
	258	+ $a\co{c}$ & none & \ldots & & \\
	259	+ $\co{a}b$ & $\co{a}$ & contains & $\co{a}$ & $1.0$ \\
	260	+ $\co{a}c$ & $\co{a}$ & \ldots & & \\
	261	+ $\co{a}\co{b}$ & $\co{a}$ & \ldots & & \\
	262	+ $\co{a}\co{c}$ & $\co{a}$ & \ldots & & \\
	263	+ $abc$ & $ab$, $ac$ & contains & $a$ & $0.8\times 0.2 = 0.016$ \\
	264	+ \end{tabular}
	265	+\end{center}
121	266
122		-
123	267	\section*{References}
124	268
125	269
...	...