On Universal Prediction and Bayesian Confirmation

Introduction

“… in spite of it’s incomputability, Algorithmic Probability can serve as a kind of ‘Gold Standard’ for induction systems”

Given the weather in the past, what is the probability of rain tomorrow? What is the correct answer in an IQ test asking to continue the sequence 1,4,9,16,? Given historic stock-charts, can one predict the quotes of tomorrow? Assuming the sun rose 5000 years every day, how likely is doomsday (that the sun does not rise) tomorrow? These are instances of the important problem of induction or time-series forecasting or sequence prediction. Finding prediction rules for every particular (new) problem is possible but cumbersome and prone to disagreement or contradiction. What is desirable is a formal general theory for prediction.

The Bayesian framework is the most consistent and successful framework developed thus far [Ear93, Jay03]. A Bayesian considers a set of environments=hypotheses=models ${\cal M}$ which includes the true data generating probability distribution $\mu$. From one’s prior belief $w_{\nu}$ in environment $\nu\in{\cal M}$ and the observed data sequence $x=x_{1}...x_{n}$, Bayes’ rule yields one’s posterior confidence in $\nu$. In a prequential [Daw84] or transductive [Vap99, Sec.9.1] setting, one directly determines the predictive probability of the next symbol $x_{n+1}$ without the intermediate step of identifying a (true or good or causal or useful) model. With the exception of Section 4, this paper concentrates on prediction rather than model identification. The ultimate goal is to make “good” predictions in the sense of maximizing one’s profit or minimizing one’s loss. Note that classification and regression can be regarded as special sequence prediction problems, where the sequence $x_{1}y_{1}...x_{n}y_{n}x_{n+1}$ of $(x,y)$-pairs is given and the class label or function value $y_{n+1}$ shall be predicted.

The Bayesian framework leaves open how to choose the model class ${\cal M}$ and prior $w_{\nu}$. General guidelines are that ${\cal M}$ should be small but large enough to contain the true environment $\mu$, and $w_{\nu}$ should reflect one’s prior (subjective) belief in $\nu$ or should be non-informative or neutral or objective if no prior knowledge is available. But these are informal and ambiguous considerations outside the formal Bayesian framework. Solomonoff’s [Sol64] rigorous, essentially unique, formal, and universal solution to this problem is to consider a single large universal class ${\cal M}_{U}$ suitable for all induction problems. The corresponding universal prior $w_{\nu}^{U}$ is biased towards simple environments in such a way that it dominates (=superior to) all other priors. This leads to an a priori probability $M(x)$ which is equivalent to the probability that a universal Turing machine with random input tape outputs $x$, and the shortest program computing $x$ produces the most likely continuation (prediction) of $x$.

Many interesting, important, and deep results have been proven for Solomonoff’s universal distribution $M$ [ZL70, Sol78, Gác83, LV97, Hut01, Hut04]. The motivation and goal of this paper is to provide a broad discussion of how and in which sense universal sequence prediction solves all kinds of (philosophical) problems of Bayesian sequence prediction, and to present some recent results. Many arguments and ideas could be further developed. I hope that the exposition stimulates such a future, more detailed, investigation.

In Section 2, I review the excellent predictive and decision-theoretic performance results of Bayesian sequence prediction for generic (non-i.i.d.) countable and continuous model classes. Section 3 critically reviews the classical principles (indifference, symmetry, minimax) for obtaining objective priors, introduces the universal prior inspired by Occam’s razor and quantified in terms of Kolmogorov complexity. In Section 4 (for i.i.d. ${\cal M}$) and Section 5 (for universal ${\cal M}_{U}$) I show various desirable properties of the universal prior and class (non-zero p(oste)rior, confirmation of universal hypotheses, reparametrization and regrouping invariance, no old-evidence and updating problem) in contrast to (most) classical continuous prior densities. I also complement the general total bounds of Section 2 with some universal and some i.i.d.-specific instantaneous and future bounds. Finally, I show that the universal mixture performs better than classical continuous mixtures, even in uncomputable environments. Section 6 contains critique, summary, and conclusions.

The reparametrization and regrouping invariance, the (weak) instantaneous bounds, the good performance of $M$ in non-computable environments, and most of the discussion (zero prior and universal hypotheses, old evidence) are new or new in the light of universal sequence prediction. Technical and mathematical non-trivial new results are the Hellinger-like loss bound (8) and the instantaneous bounds (14) and (17).

Bayesian Sequence Prediction

I now formally introduce the Bayesian sequence prediction setup and describe the most important results. I consider sequences over a finite alphabet, assume that the true environment is unknown but known to belong to a countable or continuous class of environments (no i.i.d. or Markov or stationarity assumption), and consider general prior. I show that the predictive distribution converges rapidly to the true sampling distribution and that the Bayes-optimal predictor performs excellent for any bounded loss function.

I assume that sequence $\omega=\omega_{1:\infty}\in{\cal X}^{\infty}$ is sampled from the “true” probability measure $\mu$, i.e. $\mu(x_{1:n}):={\rm P}[\omega_{1:n}=x_{1:n}|\mu]$ is the $\mu$-probability that $\omega$ starts with $x_{1:n}$. I denote expectations w.r.t. $\mu$ by ${\bf E}$. In particular for a function $f:{\cal X}^{n}\to I\mskip-3.0mu\mskip-3.0muR$, we have ${\bf E}[f]={\bf E}[f(\omega_{1:n})]=\sum_{x_{1:n}}\mu(x_{1:n})f(x_{1:n})$. Note that in Bayesian learning, measures, environments, and models coincide, and are the same objects; let ${\cal M}=\{\nu_{1},\nu_{2},...\}$ denote a countable class of these measures. Assume that (a) $\mu$ is unknown but known to be a member of ${\cal M}$, (b) $\{H_{\nu}:\nu\in{\cal M}\}$ forms a mutually exclusive and complete class of hypotheses, and (c) $w_{\nu}:={\rm P}[H_{\nu}]$ is the given prior belief in $H_{\nu}$. Then $\xi(x_{1:n}):={\rm P}[\omega_{1:n}=x_{1:n}]=\sum_{\nu\in{\cal M}}{\rm P}[\omega_{1:n}=x_{1:n}|H_{\nu}]{\rm P}[H_{\nu}]$ must be our (prior) belief in $x_{1:n}$, and $w_{\nu}(x_{1:n}):={\rm P}[H_{\nu}|\omega_{1:n}=x_{1:n}]={{\rm P}[\omega_{1:n}=x_{1:n}|H_{\nu}]{\rm P}[H_{\nu}]\over{\rm P}[\omega_{1:n}=x_{1:n}]}$ be our posterior belief in $\nu$ by Bayes’ rule.

For a sequence $a_{1},a_{2},...$ of random variables, $\sum_{t=1}^{\infty}{\bf E}[a_{t}^{2}]\leq c<\infty$ implies $a_{t}\smash{\stackrel{{\scriptstyle t\to\infty}}{{\longrightarrow}}}0$ with $\mu$-probability 1 (w.p.1). Convergence is rapid in the sense that the probability that $a_{t}^{2}$ exceeds $\varepsilon>0$ at more than ${c\over\varepsilon\delta}$ times $t$ is bounded by $\delta$. I sometimes loosely call this the number of errors.

Sequence prediction. Given a sequence $x_{1}x_{2}...x_{t-1}$, we want to predict its likely continuation $x_{t}$. I assume that the strings which have to be continued are drawn from a “true” probability distribution $\mu$. The maximal prior information a prediction algorithm can possess is the exact knowledge of $\mu$, but often the true distribution is unknown. Instead, prediction is based on a guess $\rho$ of $\mu$. While I require $\mu$ to be a measure, I allow $\rho$ to be a semimeasure [LV97, Hut04]: Readers unfamiliar or uneasy with semimeasures can without loss ignore this technicality. Formally, $\rho:{\cal X}^{*}\to$ is a semimeasure if $\rho(x)\geq\sum_{a\in{\cal X}}\rho(xa)\mskip 3.0mu\forall x\in{\cal X}^{*}$, and a (probability) measure if equality holds and $\rho(\epsilon)=1$, where $\epsilon$ is the empty string. $\rho(x)$ denotes the $\rho$-probability that a sequence starts with string $x$. Further, $\rho(a|x):=\rho(xa)/\rho(x)$ is the “posterior” or “predictive” $\rho$-probability that the next symbol is $a\in{\cal X}$, given sequence $x\in{\cal X}^{*}$.

Bayes mixture. We may know or assume that $\mu$ belongs to some countable class ${\cal M}:=\{\nu_{1},\nu_{2},...\}\ni\mu$ of semimeasures. Then we can use the weighted average on $\cal M$ (Bayes-mixture, data evidence, marginal)

for prediction. One may interpret $w_{\nu}={\rm P}[H_{\nu}]$ as prior belief in $\nu$ and $\xi(x)={\rm P}[x]$ as the subjective probability of $x$, and $\mu(x)={\rm P}[x|\mu]$ is the sampling distribution or likelihood. The most important property of semimeasure $\xi$ is its dominance

which is a strong form of absolute continuity.

Convergence for deterministic environments. In the predictive setting we are not interested in identifying the true environment, but to predict the next symbol well. Let us consider deterministic $\mu$ first. An environment is called deterministic if $\mu(\alpha_{1:n})=1\forall n$ for some sequence $\alpha$, and $\mu=0$ elsewhere (off-sequence). In this case we identify $\mu$ with $\alpha$ and the following holds:

where $w_{\alpha}>0$ is the weight of $\alpha\widehat{=}\mu\in{\cal M}$. This shows that $\xi(\alpha_{t}|\alpha_{<t})$ rapidly converges to 1 and hence also $\xi(\bar{\alpha}_{t}|\alpha_{<t})\to 0$ for $\bar{\alpha}_{t}\neq\alpha_{t}$, and that $\xi$ is also a good multi-step lookahead predictor. Proof: $\xi(\alpha_{1:n})\to c>0$, since $\xi(\alpha_{1:n})$ is monotone decreasing in $n$ and $\xi(\alpha_{1:n})\geq w_{\mu}\mu(\alpha_{1:n})=w_{\mu}>0$. Hence $\xi(\alpha_{1:n})/\xi(\alpha_{1:t})\to c/c=1$ for any limit sequence $t,n\to\infty$. The bound follows from $\sum_{t=1}^{n}1-\xi(x_{t}|x_{<t})\leq-\sum_{t=1}^{n}\ln\xi(x_{t}|x_{<t})=-\ln\xi(x_{1:n})$ and $\xi(\alpha_{1:n})\geq w_{\alpha}$.

Convergence in probabilistic environments. In the general probabilistic case we want to know how close $\xi(x_{t}|x_{<t})$ is to the true probability $\mu(x_{t}|x_{<t})$. One convenient distance measure is the (squared) Hellinger distance

The first two inequalities actually hold for any two (semi)measures, and the last inequality follows from (2). These bounds (with $n=\infty$) imply $h_{t}\to 0$ and hence

An improved bound ${\bf E}[\exp({\textstyle{1\over 2}}\sum_{t}h_{t})]\leq w_{\mu}^{-1/2}$ [HM04] even shows that the probability that $\sum_{t}h_{t}$ additively exceeds $\ln w_{\mu}^{-1}$ by $c$ (e.g. $c>10$) is tiny ${\rm e}^{-c/2}$. One can also show multi-step lookahead convergence $\xi(x_{t:n_{t}}|\omega_{<t})-\mu(x_{t:n_{t}}|\omega_{<t})\to 0$ (even for unbounded horizon $1\leq n_{t}-t+1\to\infty$), which is interesting for delayed sequence prediction and in reactive environments [Hut04]. Since $\xi$ rapidly converges to $\mu$, one can anticipate that also decisions based on $\xi$ are good.

Let $\Lambda$ be any prediction scheme (deterministic or probabilistic) with no constraint at all, taking any action $y_{t}^{\Lambda}\in\cal Y$ with total expected loss $L_{n}^{\Lambda}$. If $\mu$ is known, $\Lambda_{\mu}$ is obviously the best prediction scheme in the sense of achieving minimal expected loss $L_{n}^{\smash{\Lambda_{\mu}}}\leq L_{n}^{\Lambda}$ for any $\Lambda$. For the predictor $\Lambda_{\xi}$ based on the Bayes mixture $\xi$, one can show (proof in Appendix A; see also [MF98, Hut03a] for related bounds)

which actually holds for any two (semi)measures. Chaining with (5) implies, for instance, $l_{t}^{\smash{\Lambda_{\xi}}}\to l_{t}^{\smash{\Lambda_{\mu}}}$ rapid w.p.1, $\sqrt{L_{n}^{\smash{\Lambda_{\xi}}}}$ exceeds $\sqrt{L_{n}^{\smash{\Lambda_{\mu}}}}$ by at most $\sqrt{2\ln w_{\mu}^{-1}}$, $L_{n}^{\smash{\Lambda_{\xi}}}/L_{n}^{\smash{\Lambda_{\mu}}}\to 1$ for $L_{n}^{\smash{\Lambda_{\mu}}}\to\infty$, or if $L_{\infty}^{\smash{\Lambda_{\mu}}}$ is finite, then also $L_{\infty}^{\smash{\Lambda_{\xi}}}$. This shows that $\xi$ (via $\Lambda_{\xi}$) performs also excellent from a decision-theoretic perspective, i.e. suffers loss only slightly larger than the optimal $\Lambda_{\mu}$ predictor.

One can also show that $\Lambda_{\xi}$ is Pareto-optimal (admissible) in the sense that every other predictor with smaller loss than $\Lambda_{\xi}$ in some environment $\nu\in{\cal M}$ must be worse in another environment [Hut03c].

Continuous environmental classes. I will argue later that countable $\cal M$ are sufficiently large from a philosophical and computational perspective. On the other hand, countable ${\cal M}$ exclude all continuously parameterized families (like the class of all i.i.d. or Markov processes), common in statistical practice. I show that the bounds above remain approximately valid for most parametric model classes. Let

be a family of probability distributions parameterized by a $d$-dimensional continuous parameter $\theta$, and $\mu\equiv\nu_{\theta_{0}}\in{\cal M}$ the true generating distribution. For a continuous weight density $w()$ will always denote densities, and $w_{()}$ probabilities. $w(\theta)>0$ the sums (1) are naturally replaced by integrals:

The most important property of $\xi$ was the dominance (2) achieved by dropping the sum over $\nu$. The analogous construction here is to restrict the integral over $\theta$ to a small vicinity of $\theta_{0}$. Since a continuous parameter can typically be estimated to accuracy $\propto n^{-1/2}$ after $n$ observations, the largest volume in which $\nu_{\theta}$ as a function of $\theta$ is approximately flat is $\propto(n^{-1/2})^{d}$, hence $\xi(x_{1:n})\gtrsim n^{-d/2}w(\theta_{0})\mu(x_{1:n})$. Under some weak regularity conditions one can prove [CB90, Hut03c]

where $w(\theta_{0})$ is the weight density (9) of $\mu$ in $\xi$, and $o(1)$ tends to zero for $n\to\infty$, and the average Fisher information matrix $\bar{\jmath}_{n}(\theta)=-{1\over n}{\bf E}[\nabla_{\theta}\nabla^{T}_{\theta}\ln\nu_{\theta}(\omega_{1:n})]$ measures the local smoothness of $\nu_{\theta}$ and is bounded for many reasonable classes, including all stationary ($k^{th}$-order) finite-state Markov processes. See Section 4 for an application to the i.i.d. ($k=0$) case. We see that in the continuous case, $D_{n}$ is no longer bounded by a constant, but grows very slowly (logarithmically) with $n$, which still implies that $\varepsilon$-deviations are exponentially seldom. Hence, (10) allows to bound (5) and (8) even in case of continuous ${\cal M}$.

How to Choose the Prior

I showed in the last section how to predict if the true environment $\mu$ is unknown, but known to belong some class ${\cal M}$ of environments. In this section, I assume ${\cal M}$ to be given, and discuss how to (universally) choose the prior $w_{\nu}$. After reviewing various classical principles (indifference, symmetry, minimax) for obtaining objective priors for “small” ${\cal M}$, I discuss large ${\cal M}$. Occam’s razor in conjunction with Epicurus’ principle of multiple explanations, quantified by Kolmogorov complexity, leads us to a universal prior, which results in a better predictor than any other prior over countable ${\cal M}$.

Classical principles. The probability axioms (implying Bayes’ rule) allow to compute posteriors and predictive distributions from prior ones, but are mute about how to choose the prior. Much has been written on the choice of priors (see [KW96] for a survey and references). A main classification is between objective and subjective priors. An objective prior $w_{\nu}$ is a prior constructed based on some rational principles, which ideally everyone without (relevant) extra prior knowledge should adopt. In contrast, a subjective prior aims at modelling the agents personal (subjective) belief in environment $\nu$ prior to observation of $x$, but based on his past personal experience or knowledge (e.g. of related phenomena). In Section 6, I show that one way to arrive at a subjective prior is to start with an objective prior, make all past personal experience explicit, determine a “posterior” and use it as subjective prior. So I concentrate in the following on the more important objective priors.

Consider a very simple case of two environments, e.g. a biased coin with head or tail probability $1/3$. In absence of any extra knowledge (which I henceforth assume) there is no reason to prefer head probability $\theta=1/3$ over $\theta=2/3$ and vice versa, leaving $w_{1/3}=w_{2/3}={\textstyle{1\over 2}}$ as the only rational choice. More generally, for finite ${\cal M}$, the symmetry or indifference argument [Lap12] suggests to set $w_{\nu}={1\over|{\cal M}|}$ $\forall\nu\in{\cal M}$. Similarly for a compact measurable parameter space $\Theta$ we may choose a uniform density $w(\theta)=[\mbox{Vol}(\Theta)]^{-1}$. But there is a problem: If we go to a different parametrization (e.g. $\theta\leadsto\theta^{\prime}:=\sqrt{\theta}$ in the Bernoulli case), the prior $w(\theta)\leadsto w^{\prime}(\theta^{\prime})$ becomes non-uniform. Jeffreys’ [Jef46] solution is to find a symmetry group of the problem (like permutations for finite ${\cal M}$) and require the prior to be invariant under group transformations. For instance, if $\theta\in I\mskip-3.0mu\mskip-3.0muR$ is a location parameter (e.g. the mean) it is natural to require a translation-invariant prior. Problems are that there may be no obvious symmetry, the resulting prior may be improper (like for the translation group), and the result can depend on which parameters are treated as nuisance parameters.

The maximum entropy principle extends the symmetry principle by allowing certain types of constraints on the parameters. Conjugate priors are classes of priors such that the posteriors are themselves again in the class. While this can lead to interesting classes, the principle itself is not selective, since e.g. the class of all priors forms a conjugate class.

Another minimax approach by Bernardo [Ber79, CB90] is to consider bound (10), which can actually be improved within $o(1)$ to an equality. Since we want $D_{n}$ to be small, we minimize the r.h.s. for the worst $\mu\in{\cal M}$. Choice $w(\theta)\propto\sqrt{\det\bar{\jmath}_{n}(\theta)}$ equalizes and hence minimizes (10). The problems are the same as for Jeffrey’s prior (actually often both priors coincide), and also the dependence on the model class and potentially on $n$.

The principles above, although not unproblematic, can provide good objective priors in many cases of small discrete or compact spaces, but we will meet some more problems later. For “large” model classes I am interested in, i.e. countably infinite, non-compact, or non-parametric spaces, the principles typically do not apply or break down.

Occam’s razor et al. Machine learning, the computer science branch of statistics, often deals with very large model classes. Naturally, machine learning has (re)discovered and exploited quite different principles for choosing priors, appropriate for this situation. The overarching principles put together by Solomonoff [Sol64] are: Occam’s razor (choose the simplest model consistent with the data), Epicurus’ principle of multiple explanations (keep all explanations consistent with the data), (Universal) Turing machines (to compute, quantify and assign codes to all quantities of interest), and Kolmogorov complexity (to define what simplicity/complexity means).

I will first “derive” the so called universal prior, and subsequently justify it by presenting various welcome theoretical properties and by examples. The idea is that a priori, i.e. before seeing the data, all models are “consistent,’’ so a-priori Epicurus would regard all models (in ${\cal M}$) possible, i.e. choose $w_{\nu}>0$ $\forall\nu\in{\cal M}$. In order to also do (some) justice to Occam’s razor we should prefer simple hypotheses, i.e. assign high prior (low) prior $w_{\nu}$ to simple (complex) hypotheses $H_{\nu}$. Before I can define this prior, I need to quantify the notion of complexity.

Notation. A function $f:{\cal S}\to I\mskip-3.0mu\mskip-3.0muR\cup\{\pm\infty\}$ is said to be lower semi-computable (or enumerable) if the set $\{(x,y)\mskip 3.0mu:\mskip 3.0muy<f(x),\mskip 3.0mux\in{\cal S},\mskip 3.0muy\in I\mskip-3.0mu\mskip-3.0mu\mskip-3.0muQ\}$ is recursively enumerable. $f$ is upper semi-computable (or co-enumerable) if $-f$ is enumerable. $f$ is computable (or recursive) if $f$ and $-f$ are enumerable. The set of (co)enumerable functions is recursively enumerable. I write $O(1)$ for a constant of reasonable size: For instance, a sequence of length $100$ is reasonable, maybe even $2^{30}$, but $2^{500}$ is not. I write f(x)\mskip 3.0mu\smash{\stackrel{{\scriptstyle\raisebox{3.01389pt}{\scriptstyle\mskip-3.0mu\mskip-3.0mu\mskip 5.0mu plus 5.0mu+}}}{{\smash{\leq}}}}\mskip 3.0mug(x) for $f(x)\leq g(x)+O(1)$ and f(x)\mskip 3.0mu\smash{\stackrel{{\scriptstyle\raisebox{3.01389pt}{\scriptstyle\mskip-3.0mu\mskip-3.0mu\mskip 5.0mu plus 5.0mu\times}}}{{\smash{\leq}}}}\mskip 3.0mug(x) for $f(x)\leq 2^{O(1)}\cdot g(x)$. Corresponding equalities hold if the inequalities hold in both directions. I will ignore these additive and multiplicative fudges in the discussion till Section 6. We say that a property $A(n)\in\{true,false\}$ holds for most $n$, if $\#\{t\leq n:A(t)\}/n\smash{\stackrel{{\scriptstyle n\to\infty}}{{\longrightarrow}}}1$.

Simple strings like 000…0 can be generated by short programs, and, hence have low Kolmogorov complexity, but irregular (e.g. random) strings are their own shortest description, and hence have high Kolmogorov complexity. For non-string objects $o$ (like numbers and functions) we define $K(o):=K(\langle o\rangle)$, where $\langle o\rangle\in{\cal X}^{*}$ is some standard code for $o$. In particular, if $(f_{i})_{i=1}^{\infty}$ is an enumeration of all (co)enumerable functions, we define $K(f_{i})=K(i)$.

An important property of $K$ is that it is nearly independent of the choice of $U$. More precisely, if we switch from one universal TM to another, $K(x)$ changes at most by an additive constant independent of $x$. For natural universal TMs, the compiler constant is of reasonable size $O(1)$. A defining property of $K:{\cal X}^{*}\to I\mskip-3.0mu\mskip-3.0muN$ is that it additively dominates all co-enumerable functions $f:{\cal X}^{*}\to I\mskip-3.0mu\mskip-3.0muN$ that satisfy Kraft’s inequality $\sum_{x}2^{-f(x)}\mskip 5.0mu plus 5.0mu\leq\mskip 5.0mu plus 5.0mu1$, i.e. K(x)\mskip 3.0mu\smash{\stackrel{{\scriptstyle\raisebox{3.01389pt}{\scriptstyle\mskip-3.0mu\mskip-3.0mu\mskip 5.0mu plus 5.0mu+}}}{{\smash{\leq}}}}\mskip 3.0muf(x) for $K(f)=O(1)$. The universal TM provides a shorter prefix code than any other effective prefix code. $K$ shares many properties with Shannon’s entropy (information measure) $S$, but $K$ is superior to $S$ in many respects. To be brief, $K$ is an excellent universal complexity measure, suitable for quantifying Occam’s razor. We need the following properties of $K$:

$K$ is not computable, but only upper semi-computable,

the upper bound K(n)\mskip 3.0mu\smash{\stackrel{{\scriptstyle\raisebox{3.01389pt}{\scriptstyle\mskip-3.0mu\mskip-3.0mu\mskip 5.0mu plus 5.0mu+}}}{{\smash{\leq}}}}\mskip 3.0mu{\log_{2}}n+2{\log_{2}}\log n, (11)

Kraft’s inequality $\sum_{x}2^{-K(x)}\leq 1$, which implies $2^{-K(n)}\leq{\textstyle{1\over n}}$ for most $n$,

information non-increase K(f(x))\mskip 3.0mu\smash{\stackrel{{\scriptstyle\raisebox{3.01389pt}{\scriptstyle\mskip-3.0mu\mskip-3.0mu\mskip 5.0mu plus 5.0mu+}}}{{\smash{\leq}}}}\mskip 3.0muK(x)+K(f) for recursive $f:{\cal X}^{*}\to{\cal X}^{*}$,

K(x)\mskip 3.0mu\smash{\stackrel{{\scriptstyle\raisebox{3.01389pt}{\scriptstyle\mskip-3.0mu\mskip-3.0mu\mskip 5.0mu plus 5.0mu+}}}{{\smash{\leq}}}}\mskip 3.0mu-{\log_{2}}P(x)+K(P) if $P:{\cal X}^{*}\to$ is enumerable and $\sum_{x}P(x)\leq 1$,

\sum_{x:f(x)=y}2^{-K(x)}\mskip 3.0mu\smash{\stackrel{{\scriptstyle\raisebox{1.80835pt}{\scriptstyle\times}}}{{\smash{=}}}}\mskip 3.0mu2^{-K(y)} if $f$ is recursive and $K(f)=O(1)$.

The proof of $(f)$ can be found in Appendix A and the proofs of $(a)-(e)$ in [LV97].

The universal prior. We can now quantify a prior biased towards simple models. First, we quantify the complexity of an environment $\nu$ or hypothesis $H_{\nu}$ by its Kolmogorov complexity $K(\nu)$. The universal prior should be a decreasing function in the model’s complexity, and of course sum to (less than) one. Since $K$ satisfies Kraft’s inequality ( b ) $c$), this suggests the following choice:

For this choice, the bound (5) on $D_{n}$ (which bounds (5) and (8)) reads

i.e. the number of times, $\xi$ deviates from $\mu$ or $l^{\Lambda_{\xi}}$ deviates from $l^{\Lambda_{\mu}}$ by more than $\varepsilon>0$ is bounded by $O(K(\mu))$, i.e. is proportional to the complexity of the environment. Could other choices for $w_{\nu}$ lead to better bounds? The answer is essentially no [Hut04]: Consider any other reasonable prior $w^{\prime}_{\nu}$, where reasonable means (lower semi)computable with a program of size $O(1)$. Then, MDL bound ( b ) $e$) with $P()\leadsto w^{\prime}_{()}$ and $x\leadsto\langle\mu\rangle$ shows K(\mu)\mskip 3.0mu\smash{\stackrel{{\scriptstyle\raisebox{3.01389pt}{\scriptstyle\mskip-3.0mu\mskip-3.0mu\mskip 5.0mu plus 5.0mu+}}}{{\smash{\leq}}}}\mskip 3.0mu-{\log_{2}}w^{\prime}_{\mu}+K(w^{\prime}_{()}), hence \ln w_{\mu}^{\prime}\mskip-3.0mu{}^{-1}\mskip 3.0mu\smash{\stackrel{{\scriptstyle\raisebox{3.01389pt}{\scriptstyle+\mskip-3.0mu\mskip-3.0mu\mskip 5.0mu plus 5.0mu}}}{{\smash{\geq}}}}\mskip 3.0muK(\mu)\ln 2 leads (within an additive constant) to a weaker bound. A counting argument also shows that $O(K(\mu))$ errors for most $\mu$ are unavoidable. So this choice of prior leads to very good prediction.

Even for continuous classes ${\cal M}$, we can assign a (proper) universal prior (not density) $w_{\theta}^{U}=2^{-K(\theta)}>0$ for computable $\theta$, and 0 for uncomputable ones. This effectively reduces ${\cal M}$ to a discrete class $\{\nu_{\theta}\in{\cal M}:w_{\theta}^{U}>0\}$ which is typically dense in ${\cal M}$. We will see that this prior has many advantages over the classical prior densities.

Independent Identically Distributed Data

I now compare the classical continuous prior densities to the universal prior on classes of i.i.d. environments. I present some standard critiques to the former, illustrated on Bayes-Laplace’s classical Bernoulli class with uniform prior: the problem of zero p(oste)rior, non-confirmation of universal hypotheses, and reparametrization and regrouping non-invariance. I show that the universal prior does not suffer from these problems. Finally I complement the general total bounds of Section 2 with some i.i.d.-specific instantaneous bounds.

Laplace’s rule for Bernoulli sequences. Let $x=x_{1}x_{2}...x_{n}\in{\cal X}^{n}=\{0,1\}^{n}$ be generated by a biased coin with head=1 probability $\theta\in$, i.e. the likelihood of $x$ under hypothesis $H_{\theta}$ is $\nu_{\theta}(x)={\rm P}[x|H_{\theta}]=\theta^{n_{1}}(1-\theta)^{n_{0}}$, where $n_{1}=x_{1}+...+x_{n}=n-n_{0}$. Bayes [Bay63] assumed a uniform prior density $w(\theta)=1$. The evidence is $\xi(x)=\int_{0}^{1}\nu_{\theta}(x)w(\theta)\mskip 3.0mud\theta={n_{1}!n_{0}!\over(n+1)!}$ and the posterior probability weight density $w(\theta|x)=\nu_{\theta}(x)w(\theta)/\xi(x)={(n+1)!\over n_{1}!n_{0}!}\theta^{n_{1}}(1-\theta)^{n_{0}}$ of $\theta$ after seeing $x$ is strongly peaked around the frequency estimate $\hat{\theta}={n_{1}\over n}$ for large $n$. Laplace [Lap12] asked for the predictive probability $\xi(1|x)$ of observing $x_{n+1}=1$ after having seen $x=x_{1}...x_{n}$, which is $\xi(1|x)={\xi(x1)\over\xi(x)}={n_{1}+1\over n+2}$. (Laplace believed that the sun had risen for $5\mskip 3.0mu000$ years = $1\mskip 3.0mu826\mskip 3.0mu213$ days since creation, so he concluded that the probability of doom, i.e. that the sun won’t rise tomorrow is ${1\over 1826215}$.) This looks like a reasonable estimate, since it is close to the relative frequency, asymptotically consistent, symmetric, even defined for $n=0$, and not overconfident (never assigns probability 1).

The problem of zero prior. But also Laplace’s rule is not without problems. The appropriateness of the uniform prior has been questioned in Section 3 and will be detailed below. Here I discuss a version of the zero prior problem. If the prior is zero, then the posterior is necessarily also zero. The above example seems unproblematic, since the prior and posterior densities $w(\theta)$ and $w(\theta|x)$ are non-zero. Nevertheless it is problematic e.g. in the context of scientific confirmation theory [Ear93].

Consider the hypothesis $H$ that all balls in some urn, or all ravens, are black (=1). A natural model is to assume that balls (or ravens) are drawn randomly from an infinite population with fraction $\theta$ of black balls (or ravens) and to assume a uniform prior over $\theta$, i.e. just the Bayes-Laplace model. Now we draw $n$ objects and observe that they are all black.

We may formalize $H$ as the hypothesis $H^{\prime}:=\{\theta=1\}$. Although the posterior probability of the relaxed hypothesis $H_{\varepsilon}:=\{\theta\geq 1-\varepsilon\}$, ${\rm P}[H_{\varepsilon}|1^{n}]=\int_{1-\varepsilon}^{1}w(\theta|1^{n})\mskip 3.0mud\theta=\int_{1-\varepsilon}^{1}(n+1)\theta^{n}d\theta=1-(1-\varepsilon)^{n+1}$ tends to 1 for $n\to\infty$ for every fixed $\varepsilon>0$, ${\rm P}[H^{\prime}|1^{n}]={\rm P}[H_{0}|1^{n}]$ remains identically zero, i.e. no amount of evidence can confirm $H^{\prime}$. The reason is simply that zero prior ${\rm P}[H^{\prime}]=0$ implies zero posterior.

Note that $H^{\prime}$ refers to the unobservable quantity $\theta$ and only demands blackness with probability 1. So maybe a better formalization of $H$ is purely in terms of observational quantities: $H^{\prime\prime}:=\{\omega_{1:\infty}=1^{\infty}\}$. Since $\xi(1^{n})={1\over n+1}$, the predictive probability of observing $k$ further black objects is $\xi(1^{k}|1^{n})={\smash{\xi(1^{n+k})\over\xi(1^{n})}}={n+1\over n+k+1}$. While for fixed $k$ this tends to 1, ${\rm P}[H^{\prime\prime}|1^{n}]=\lim_{k\to\infty}\xi(1^{k}|1^{n})\equiv 0$ $\forall n$, as for $H^{\prime}$.

One may speculate that the crux is the infinite population. But for a finite population of size $N$ and sampling with (similarly without) repetition, ${\rm P}[H^{\prime\prime}|1^{n}]=\xi(1^{N-n}|1^{n})={n+1\over N+1}$ is close to one only if a large fraction of objects has been observed. This contradicts scientific practice: Although only a tiny fraction of all existing ravens have been observed, we regard this as sufficient evidence for believing strongly in $H$. This quantifies [Mah04, Thm.11] and shows that Maher does not solve the problem of confirmation of universal hypotheses.

There are two solutions of this problem: We may abandon strict/logical/all-quantified/universal hypotheses altogether in favor of soft hypotheses like $H_{\varepsilon}$. Although not unreasonable, this approach is unattractive for several reasons. The other solution is to assign a non-zero prior to $\theta=1$. Consider, for instance, the improper density $w(\theta)={\textstyle{1\over 2}}[1+\delta(1-\theta)]$, where $\delta$ is the Dirac-delta ($\int f(\theta)\delta(\theta-a)\mskip 3.0mud\theta=f(a)$), or equivalently ${\rm P}[\theta\geq a]=1-{\textstyle{1\over 2}}a$. We get $\xi(x_{1:n})={\textstyle{1\over 2}}[{n_{1}!n_{0}!\over(n+1)!}+\delta_{0n_{0}}]$, where $\delta_{ij}=\{{1\text{ if }i=j\atop 0\text{ else}\mskip 5.0mu plus 5.0mu\mskip 5.0mu plus 5.0mu\mskip 5.0mu plus 5.0mu}\}$ is Kronecker’s $\delta$. In particular $\xi(1^{n})={\textstyle{1\over 2}}{n+2\over n+1}$ is much larger than for uniform prior. Since $\xi(1^{k}|1^{n})={n+k+2\over n+k+1}\cdot{n+1\over n+2}$, we get ${\rm P}[H^{\prime\prime}|1^{n}]=\lim_{k\to\infty}\xi(1^{k}|1^{n})={n+1\over n+2}\to 1$, i.e. $H^{\prime\prime}$ gets strongly confirmed by observing a reasonable number of black objects. This correct asymptotics also follows from the general result (3). Confirmation of $H^{\prime\prime}$ is also reflected in the fact that $\xi(0|1^{n})={1\over(n+2)^{2}}$ tends much faster to zero than for uniform prior, i.e. the confidence that the next object is black is much higher. The power actually depends on the shape of $w(\theta)$ around $\theta=1$. Similarly $H^{\prime}$ gets confirmed: ${\rm P}[H^{\prime}|1^{n}]=\mu_{1}(1^{n}){\rm P}[\theta=1]/\xi(1^{n})={n+1\over n+2}\to 1$. On the other hand, if a single (or more) 0 are observed ($n_{0}>0$), then the predictive distribution $\xi(\cdot|x)$ and posterior $w(\theta|x)$ are the same as for uniform prior.

The findings above remain qualitatively valid for i.i.d. processes over finite non-binary alphabet $|{\cal X}|>2$ and for non-uniform prior.

Surely to get a generally working setup, we should also assign a non-zero prior to $\theta=0$ and to all other “special” $\theta$, like ${\textstyle{1\over 2}}$ and ${1\over 6}$, which may naturally appear in a hypothesis, like “is the coin or die fair”. The natural continuation of this thought is to assign non-zero prior to all computable $\theta$. This is another motivation for the universal prior $w_{\theta}^{U}=2^{-K(\theta)}$ (12) constructed in Section 3. It is difficult but not impossible to operate with such a prior [PH04, PH06]. One may want to mix the discrete prior $w_{\nu}^{U}$ with a continuous (e.g. uniform) prior density, so that the set of non-computable $\theta$ keeps a non-zero density. Although possible, we will see that this is actually not necessary.

Reparametrization invariance. Naively, the uniform prior is justified by the indifference principle, but as discussed in Section 3, uniformity is not reparametrization invariant. For instance if in our Bernoulli example we introduce a new parametrization $\theta^{\prime}=\sqrt{\theta}$, then the $\theta^{\prime}$-density $w^{\prime}(\theta^{\prime})=2\sqrt{\theta}w(\theta)$ is no longer uniform if $w(\theta)=1$ is uniform.

Regrouping invariance. There are important transformations $f$ which are not bijections, which we consider in the following. A simple non-bijection is $\theta=f(\theta^{\prime})=\theta^{\prime 2}$ if we consider $\theta^{\prime}\in$. More interesting is the following example: Assume we had decided not to record blackness versus non-blackness of objects, but their “color”. For simplicity of exposition assume we record only whether an object is black or white or colored, i.e. ${\cal X}^{\prime}=\{B,W,C\}$. In analogy to the binary case we use the indifference principle to assign a uniform prior on $\boldsymbol{\theta^{\prime}}\in\Theta^{\prime}:=\Delta_{3}$, where $\Delta_{d}:=\{\boldsymbol{\theta^{\prime}}\in^{d}:\sum_{i=1}^{d}\theta^{\prime}_{i}=1\}$, and $\nu_{\theta^{\prime}}(x^{\prime}_{1:n})=\prod_{i}{\theta^{\prime}_{i}}^{n_{i}}$. All inferences regarding blackness (predictive and posterior) are identical to the binomial model $\nu_{\theta}(x_{1:n})=\theta^{n_{1}}(1-\theta)^{n_{0}}$ with $x^{\prime}_{t}=B$ $\leadsto$ $x_{t}=1$ and $x^{\prime}_{t}=W\mskip 3.0mu$or$\mskip 3.0muC$ $\leadsto$ $x_{t}=0$ and $\theta=f(\boldsymbol{\theta^{\prime}})=\theta^{\prime}_{B}$ and $w(\theta)=\int_{\Delta_{3}}w^{\prime}(\boldsymbol{\theta^{\prime}})\delta(\theta^{\prime}_{B}-\theta)d\boldsymbol{\theta^{\prime}}$. Unfortunately, for uniform prior $w^{\prime}(\boldsymbol{\theta^{\prime}})\propto 1$, $w(\theta)\propto 1-\theta$ is not uniform, i.e. the indifference principle is not invariant under splitting/grouping, or general regrouping. Regrouping invariance is regarded as a very important and desirable property [Wal96].

I now consider general i.i.d. processes $\nu_{\theta}(x)=\prod_{i=1}^{d}\theta_{i}^{n_{i}}$. Dirichlet priors $w(\theta)\propto\prod_{i=1}^{d}\theta_{i}^{\alpha_{i}-1}$ form a natural conjugate class ($w(\theta|x)\propto\prod_{i=1}^{d}\theta_{i}^{n_{i}+\alpha_{i}-1}$) and are the default priors for multinomial (i.i.d.) processes over finite alphabet ${\cal X}$ of size $d$. Note that $\xi(a|x)={n_{a}+\alpha_{a}\over n+\alpha_{1}+...+\alpha_{d}}$ generalizes Laplace’s rule and coincides with Carnap’s [Car52] confirmation function. Symmetry demands $\alpha_{1}=...=\alpha_{d}$; for instance $\alpha\equiv 1$ for uniform and $\alpha\equiv{\textstyle{1\over 2}}$ for Bernard-Jeffrey’s prior. Grouping two “colors” $i$ and $j$ results in a Dirichlet prior with $\alpha_{i\&j}=\alpha_{i}+\alpha_{j}$ for the group. The only way to respect symmetry under all possible groupings is to set $\alpha\equiv 0$. This is Haldane’s improper prior, which results in unacceptably overconfident predictions $\xi(1|1^{n})=1$. Walley [Wal96] solves the problem that there is no single acceptable prior density by considering sets of priors.

i.e. the universal prior is general transformation and hence regrouping invariant (within a multiplicative constant) w.r.t. simple computable transformations $f$.

Note that reparametrization and regrouping invariance hold for arbitrary classes ${\cal M}$ and are not limited to the i.i.d. case.

Instantaneous bounds. The cumulative bounds (5) and (10) stay valid for i.i.d. processes, but instantaneous bounds are now also possible. For i.i.d. ${\cal M}$ with continuous, discrete, and universal prior, respectively, one can show (in preparation; see [Kri98, PH04, PH06] for related bounds)

Note that, if summed up over $n$, they lead to weaker cumulative bounds.

Universal Sequence Prediction

Section 3 derived the universal prior and Section 4 discussed i.i.d. classes. What remains and will be done in this section is to find a universal class of environments, namely Solomonoff-Levin’s class of all (lower semi)computable (semi)measures. The resulting universal mixture is equivalent to the output distribution of a universal Turing machine with uniform input distribution. The universal prior avoids the problem of old evidence and the universal class avoids the necessity of updating ${\cal M}$. I discuss the general total bounds of Section 2 for the specific universal mixture, and supplement them with some weak instantaneous bounds. Finally, I show that the universal mixture performs better than classical continuous mixtures, even in uncomputable environments.

Universal choice of $\cal M$. The bounds of Section 2 apply if ${\cal M}$ contains the true environment $\mu$. The larger ${\cal M}$ the less restrictive is this assumption. The class of all computable distributions, although only countable, is pretty large from a practical point of view. (Finding a non-computable physical system would overturn the Church-Turing thesis.) It is the largest class, relevant from a computational point of view. Solomonoff [Sol64, Eq.(13)] defined and studied the mixture over this class.

One problem is that this class is not enumerable, since the class of computable functions $f:{\cal X}^{*}\to I\mskip-3.0mu\mskip-3.0muR$ is not enumerable (halting problem), nor is it decidable whether a function is a measure. Hence $\xi$ is completely incomputable. Levin [ZL70] had the idea to “slightly” extend the class and include also lower semi-computable semimeasures. One can show that this class ${\cal M}_{U}=\{\nu_{1},\nu_{2},...\}$ is enumerable, hence

is itself lower semi-computable, i.e. $\xi_{U}\in{\cal M}_{U}$, which is a convenient property in itself. Note that since {1\over n\log^{2}n}\mskip 3.0mu\smash{\stackrel{{\scriptstyle\raisebox{3.01389pt}{\scriptstyle\mskip-3.0mu\mskip-3.0mu\mskip 5.0mu plus 5.0mu\times}}}{{\smash{\leq}}}}\mskip 3.0muw_{\nu_{n}}^{U}\leq{\textstyle{1\over n}} for most $n$ by ( b ) $b$) and ( b ) $c$), most $\nu_{n}$ have prior approximately reciprocal to their index $n$, as advocated by Jeffreys [Jef61, p238] and Rissanen [Ris83].

In some sense ${\cal M}_{U}$ is the largest class of environments for which $\xi$ is in some sense computable [Hut03b, Hut06], but see [Sch02a] for even larger classes. Note that including non-semi-computable $\nu$ would not affect $\xi_{U}$, since $w_{\nu}^{U}=0$ on such environments.

The problem of old evidence. An important problem in Bayesian inference in general and (Bayesian) confirmation theory [Ear93] in particular is how to deal with ‘old evidence’ or equivalently with ‘new theories’. How shall a Bayesian treat the case when some evidence $E\widehat{=}x$ (e.g. Mercury’s perihelion advance) is known well-before the correct hypothesis/theory/model $H\widehat{=}\mu$ (Einstein’s general relativity theory) is found? How shall $H$ be added to the Bayesian machinery a posteriori? What is the prior of $H$? Should it be the belief in $H$ in a hypothetical counterfactual world in which $E$ is not known? Can old evidence $E$ confirm $H$? After all, $H$ could simply be constructed/biased/fitted towards “explaining” $E$.

The universal class ${\cal M}_{U}$ and universal prior $w_{\nu}^{U}$ formally solve this problem: The universal prior of $H$ is $2^{-K(H)}$. This is independent of ${\cal M}$ and of whether $E$ is known or not. If we use $E$ to construct $H$ or fit $H$ to explain $E$, this will lead to a theory which is more complex (K(H)\mskip 3.0mu\smash{\stackrel{{\scriptstyle\raisebox{3.01389pt}{\scriptstyle+\mskip-3.0mu\mskip-3.0mu\mskip 5.0mu plus 5.0mu}}}{{\smash{\geq}}}}\mskip 3.0muK(E)) than a theory from scratch ($K(H)=O(1)$), so cheats are automatically penalized. There is no problem of adding hypotheses to ${\cal M}$ a posteriori. Priors of old hypotheses are not affected. Finally, ${\cal M}_{U}$ includes all hypotheses (including yet unknown or unnamed ones) a priori. So at least theoretically, updating ${\cal M}$ is unnecessary.

If $x_{1:\infty}$ is a computable sequence, then $K\mskip-3.0mum(x_{1:\infty})$ is finite, which implies $M(x_{t}|x_{<t})\to 1$ on every computable sequence. This means that if the environment is a computable sequence (whichsoever, e.g. $1^{\infty}$ or the digits of $\pi$ or ${\rm e}$), after having seen the first few digits, $M$ correctly predicts the next digit with high probability, i.e. it recognizes the structure of the sequence. In particular, observing an increasing number of black balls or black ravens or sunrises, $M(1|1^{n})\to 1$ ($K\mskip-3.0mum(1^{\infty})=O(1)$) becomes rapidly confident that future balls and ravens are black and that the sun will rise tomorrow.

Total bounds (3) and (13) are suitable in an online setting, but given a fixed number of $n$ observations, they give no guarantee on the next instance.

Weak instantaneous bounds. In Section 4, I derived good instantaneous bounds for i.i.d. classes. For coin or die flips or balls drawn from an urn this model is appropriate. But ornithologists do not really sample ravens independently at random. Although not strictly valid, the i.i.d. model may in this case still serve as a useful proxy for the true process. But to model the rise of the sun as an i.i.d. process is more than questionable. On the other hand it is plausible that these examples (and other processes like weather or stock-market) are governed by some (probabilistic) computable process. So model class ${\cal M}_{U}$ and predictor $M$ seem appropriate. While excellent total bounds (3) and (13) exist, the essentially only instantaneous bound I was able to derive (proof in Appendix A) is

valid for all $n$ and $x_{1:n}$ and $\bar{x}_{n}\neq x_{n}$. I discuss the bound for the sequence $x_{1:\infty}=1^{\infty}$, but most of what I say remains valid for any other computable sequence. Since $K\mskip-3.0mum(1^{n})=O(1)$, we get

Since $2^{-K(n)}\leq{\textstyle{1\over n}}$ for most $n$, this shows that $M$ quickly disbelieves in non-black objects and doomsday, similarly as in the i.i.d. model, but now only for most $n$.

Magic numbers. This ‘most’ qualification has interesting consequences: $M(0|1^{n})$ spikes up for simple $n$. So $M$ is cautious at magic instance numbers, e.g. fears doom on day $2^{20}$ more than on a comparable random day. While this looks odd and pours water on the mills of prophets, it is not completely absurd. For instance, major software problems have been anticipated for the magic date, 1st of January 2000. There are many other occasions, where something happens at “magic” dates or instances; for instance solar eclipses.

Also, certain processes in nature follow fast growing sequences like those of the powers of two (e.g. the number of cells in an early human embryo) or the Fibonacci numbers (e.g. the number of petals or the arrangement of seeds in some flowers). Finally, that numbers with low (Kolmogorov) complexity cause high probability in real data bases can readily be verified by counting their frequency in the world wide web with Google [CV06].

On the other hand, (returning to sequence prediction) on most simple dates, nothing exceptional happens. Due to the total bound $\sum_{n=0}^{\infty}M(0|1^{n})\leq O(1)$, $M$ cannot spike up too much too often. $M$ tells us to be more prepared but not to expect the unexpected on those days. Another issue is that often we do not know the exact start of the sequence. How many ravens exactly have ornithologists observed, and how many days exactly did the sun rise so far? In absence of this knowledge we need to Bayes-average over the sequence length which will wash out the spikes.

Universal is better than continuous ${\cal M}$. Although I argued that incomputable environments $\mu$ can safely be ignored, one may be nevertheless uneasy using Solomonoff’s M\mskip 3.0mu\smash{\stackrel{{\scriptstyle\raisebox{1.80835pt}{\scriptstyle\times}}}{{\smash{=}}}}\mskip 3.0mu\xi_{U} (16) if outperformed by a continuous mixture $\xi$ (9) on such $\mu\in{\cal M}\setminus{\cal M}_{U}$, for instance if $M$ would fail to predict a Bernoulli($\theta$) sequence for incomputable $\theta$. Luckily this is not the case: Although $\nu_{\theta}()$ and $w_{\theta}$ can be incomputable, the studied classes ${\cal M}$ themselves, i.e. the two-argument function $\nu_{()}()$, and the weight function $w_{()}$, and hence $\xi()$, are typically computable (the integral can be approximated to arbitrary precision). Hence M(x)\mskip 3.0mu\smash{\stackrel{{\scriptstyle\raisebox{1.80835pt}{\scriptstyle\times}}}{{\smash{=}}}}\mskip 3.0mu\xi_{U}(x)\geq 2^{-K(\xi)}\xi(x) by (15) and $K(\xi)$ is often quite small. This implies for all $\mu$

So any bound (10) for $D_{n}(\mu||\xi)$ is directly valid also for $D_{n}(\mu||M)$, save an additive constant. That is, $M$ is superior (or equal) to all computable mixture predictors $\xi$ based on any (continuous or discrete) model class ${\cal M}$ and weight $w(\theta)$, even if environment $\mu$ is not computable. Furthermore, while for essentially all parametric classes, $D_{n}(\mu||\xi)\sim{d\over 2}\ln n$ grows logarithmically in $n$ for all (incl. computable) $\mu\in{\cal M}$, $D_{n}(\mu||M)\leq K(\mu)\ln 2$ is finite for computable $\mu$. Bernardo’s prior even implies a bound for $M$ that is uniform (minimax) in $\theta\in\Theta$. Many other priors based on reasonable principles are argued for (see Section 3 and [KW96]). The above shows that $M$ is superior to all of them. Actually the existence of any computable probabilistic predictor $\rho$ with $D_{n}(\mu||\rho)=o(n)$ is sufficient for $M$ to predict $\mu$ equally well.

Future bounds. Another important question is how many errors are still to come after some grace or learning period. Formally, given $x_{1:n}$, how large is the future expected error $r_{n}:=\sum_{t=n+1}^{\infty}{\bf E}[h_{t}|\omega_{1:n}=x_{1:n}]$? The total bound (5)+(13) only implies that $r_{n}$ asymptotically tends to zero w.p.1, and the instantaneous bounds (14) and (17) are weak and do not sum up finitely. Since the complexity of $\mu$ bounds the total loss, a natural guess is that something like the conditional complexity of $\mu$ given $x$ (on an extra input tape) bounds the future loss. Indeed one can show [Hut04, CH05]

i.e. if our past observations $\omega_{1:n}$ contain a lot of information about $\mu$, we make few errors in future. For instance, consider the large space ${\cal X}$ of pixel images, and all observations are identical $\mu\mskip 3.0mu\widehat{=}\mskip 3.0mu\omega=x_{1}x_{1}x_{1}...$, where $x_{1}$ is a “typical” image of complexity, say, K(x_{1})\mskip 3.0mu\smash{\stackrel{{\scriptstyle\raisebox{2.41112pt}{\scriptstyle+}}}{{\smash{=}}}}\mskip 3.0mu10^{6}\mskip 3.0mu\smash{\stackrel{{\scriptstyle\raisebox{2.41112pt}{\scriptstyle+}}}{{\smash{=}}}}\mskip 3.0muK\mskip-3.0mum(\omega). Obviously, after seeing a couple of identical images we expect the next one to be the same again. While total bound (13) quite uselessly tells us that $M$ makes less than $10^{6}$ errors, future bound (18) with $n=1$ shows that $M$ makes only $K(\mu|x_{1})=O(1)$ errors. The $K(n)$ term can be improved to the complexity of the randomness deficiency of $\omega_{1:n}$ if a more suitable variant of algorithmic complexity that is monotone in its condition is used [CH05, CHS07]. No future bounds analogous to (18) for general prior or class are known.

Discussion

Critique and problems. In practice we often have extra information about the problem at hand, which could and should be used to guide the forecasting. One way is to explicate all our prior knowledge $y$ and place it on an extra input tape of our universal Turing machine $U$, which leads to the conditional complexity $K(\cdot|y)$. We now assign “subjective” prior $w_{\nu|y}^{U}=2^{-K(\nu|y)}$ to environment $\nu$, which is large for those $\nu$ that are simple (have short description) relative to our background knowledge $y$. Since K(\mu|y)\mskip 3.0mu\smash{\stackrel{{\scriptstyle\raisebox{3.01389pt}{\scriptstyle\mskip-3.0mu\mskip-3.0mu\mskip 5.0mu plus 5.0mu+}}}{{\smash{\leq}}}}\mskip 3.0muK(\mu), extra knowledge never misguides (see (13)). Alternatively we could prefix our observation sequence $x$ by $y$ and use $M(yx)$ for prediction [Hut04].

Another critique concerns the dependence of $K$ and $M$ on $U$. Predictions for short sequences $x$ (shorter than typical compiler lengths) can be arbitrary. But taking into account our (whole) scientific prior knowledge $y$, and predicting the now long string $yx$ leads to good (less sensitive to “reasonable” $U$) predictions [Hut04]. For an interesting attempt to make $M$ unique see [Mül06].

Finally, $K$ and $M$ can serve as “gold standards” which practitioners should aim at, but since they are only semi-computable, they have to be (crudely) approximated in practice. Levin complexity [LV97], the speed prior [Sch02b], the minimal message and description length principles [Ris89, Wal05], and off-the-shelf compressors like Lempel-Ziv [LZ76] are such approximations, which have been successfully applied to a plethora of problems [CV05, Sch04].

Summary. I compared traditional Bayesian sequence prediction based on continuous classes and prior densities to Solomonoff’s universal predictor $M$, prior $w_{\nu}^{U}$, and class ${\cal M}_{U}$. I discussed the following advantages (+) and problems ($-$) of Solomonoff’s approach:

general total bounds for generic class, prior, and loss,

universal and i.i.d.-specific instantaneous and future bounds,

the $D_{n}$ bound for continuous classes,

the problem of zero p(oste)rior and confirmation of universal hypotheses,

reparametrization and regrouping invariance,

the problem of old evidence and updating,

that $M$ works even in non-computable environments,

the constant fudges in all results and the $U$-dependence,

$M$’s incomputability and crude practical approximations.

In short, universal prediction solves or avoids or meliorates many foundational and philosophical problems, but has to be compromised in practice.

Conclusion. The goal of the paper was to establish a single, universal theory for (sequence) prediction and (hypothesis) confirmation, applicable to all inductive inference problems. I started by showing that Bayesian prediction is consistent for any countable model class, provided it contains the true distribution. The major (agonizing) problem Bayesian statistics leaves open is how to choose the model class and the prior. Solomonoff’s theory fills this gap by choosing the class of all computable (stochastic) models, and a universal prior inspired by Ockham and Epicurus, and quantified by Kolmogorov complexity. I discussed in breadth how and in which sense this theory solves the inductive inference problem, by studying a plethora of problems other approaches suffer from. In one line: All you need for universal prediction is Ockham, Epicurus, Bayes, Solomonoff, Kolmogorov, and Turing. By including Bellman, one can extend this theory to universal decisions in reactive environments [Hut04].

Acknowledgements. I would like to thank Frank Stephan for his detailed feedback on earlier drafts.

References

Appendix A Proofs of (8), ( b ) ff), and (17)

Proof of loss bound (8). Let $X$ and $Y$ be real-valued random variables. Taking the square root of the well-known Schwarz inequality $({\bf E}[XY])^{2}\leq{\bf E}[X^{2}]{\bf E}[Y^{2}]$ we get

Substituting $X\leadsto\sqrt{a_{i}}$, $Y\leadsto\sqrt{b_{i}}$, ${\bf E}[...]\leadsto{1\over v_{\Sigma}}\sum_{i}v_{i}...$ with $v_{\Sigma}:=\sum_{i}v_{i}$, we get, after multiplying with $v_{\Sigma}$, the “Hellinger” bound

for real $a_{i},b_{i},v_{i}\geq 0$ (also valid for $v_{\Sigma}=0$). I will use (19) three times in proving (8). With the abbreviations $m=y_{t}^{\smash{\Lambda_{\mu}}}$ and $s=y_{t}^{\smash{\Lambda_{\xi}}}$ and

Actually, I need the first constraint only for $j=s$ and the second for $j=m$. From (20) we get

Proof of equation ( b ) $f$). Function $P(y):=\sum_{x:f(x)=y}2^{-K(x)}$ is lower semi-computable, since $K(x)$ is upper semi-computable, all $x\in{\cal X}^{*}$ can be enumerated, and $f(x)=y$ is decidable. Further, $\sum_{y}P(y)=\sum_{x}2^{-K(x)}\leq 1$, hence MDL bound ( b ) $e$) implies K(y)\mskip 3.0mu\smash{\stackrel{{\scriptstyle\raisebox{3.01389pt}{\scriptstyle\mskip-3.0mu\mskip-3.0mu\mskip 5.0mu plus 5.0mu+}}}{{\smash{\leq}}}}\mskip 3.0mu-{\log_{2}}P(y)+K(P). Let $g(y)=\min\{x:f(y)=x\}$ be the lexicographically first inverse of $f$. With K(P)\mskip 3.0mu\smash{\stackrel{{\scriptstyle\raisebox{3.01389pt}{\scriptstyle\mskip-3.0mu\mskip-3.0mu\mskip 5.0mu plus 5.0mu+}}}{{\smash{\leq}}}}\mskip 3.0muK(f)=O(1), also function $g$ has complexity $O(1)$. Hence

where I dropped all but the contribution from $g(y)$ in the sum, and used ( b ) $d$) for $g$.

Proof of bound (17) M(\bar{x}_{n}|x_{<n})\mskip 3.0mu\smash{\stackrel{{\scriptstyle\raisebox{3.1111pt}{\scriptstyle\times\mskip-3.0mu\mskip-3.0mu\mskip 5.0mu plus 5.0mu}}}{{\smash{\geq}}}}\mskip 3.0mu2^{-K(n)}. For $x=x_{<n}\in{\cal X}^{n-1}$ and $a=\bar{x}_{n}\in{\cal X}$ we have

Proof of bound (17) M(\bar{x}_{n}|x_{<n})\mskip 3.0mu\smash{\stackrel{{\scriptstyle\raisebox{3.1111pt}{\scriptstyle\mskip-3.0mu\mskip-3.0mu\mskip 5.0mu plus 5.0mu\times}}}{{\smash{\leq}}}}\mskip 3.0mu2^{2K\mskip-3.0mum(x_{1:n})-K(n)}. Assume $x_{1:\infty}$ is a computable sequence, ${\cal X}$ is binary, and $\bar{x}_{n}\neq x_{n}$, and define $P(n):=M(x_{<n}\bar{x}_{n})$. Given $x_{1:\infty}$, $P$ can be semi-computed from below, hence K(P)\mskip 3.0mu\smash{\stackrel{{\scriptstyle\raisebox{3.01389pt}{\scriptstyle\mskip-3.0mu\mskip-3.0mu\mskip 5.0mu plus 5.0mu+}}}{{\smash{\leq}}}}\mskip 3.0muK\mskip-3.0mum(x_{1:\infty}). Also $\sum_{n}P(n)\leq 1$, since $\{x_{<n}\bar{x}_{n}:n\in I\mskip-3.0mu\mskip-3.0muN\}$ forms a prefix-free set. Hence K(n)\mskip 3.0mu\smash{\stackrel{{\scriptstyle\raisebox{3.01389pt}{\scriptstyle\mskip-3.0mu\mskip-3.0mu\mskip 5.0mu plus 5.0mu+}}}{{\smash{\leq}}}}\mskip 3.0mu-{\log_{2}}P(n)+K(P) by ( b ) $e$), which implies M(x_{<n}\bar{x}_{n})\mskip 3.0mu\smash{\stackrel{{\scriptstyle\raisebox{3.01389pt}{\scriptstyle\mskip-3.0mu\mskip-3.0mu\mskip 5.0mu plus 5.0mu\times}}}{{\smash{\leq}}}}\mskip 3.0mu2^{K\mskip-3.0mum(x_{1:\infty})-K(n)}. Since $M(x_{<n})\geq 2^{-K\mskip-3.0mum(x_{<n})}\geq 2^{-K\mskip-3.0mum(x_{1:\infty})}$, we get M(\bar{x}_{n}|x_{<n})\mskip 3.0mu\smash{\stackrel{{\scriptstyle\raisebox{3.01389pt}{\scriptstyle\mskip-3.0mu\mskip-3.0mu\mskip 5.0mu plus 5.0mu\times}}}{{\smash{\leq}}}}\mskip 3.0mu2^{2K\mskip-3.0mum(x_{1:\infty})-K(n)}, which nearly is (17). Since the l.h.s. is independent of $x_{n+1:\infty}$, a bound independent of it should be (and is) possible, as we will now show.