From f-divergence to quantum quasi-entropies and their use

ff-divergence and its use

Let ${\cal A}$ be a partition of ${\cal X}$. If $p$ is a probability distribution on ${\cal X}$, then $p_{\cal A}(A):=\sum_{x\in A}p(x)$ becomes a probability distribution on ${\cal A}$

Let ${\cal A}$ be a partition of ${\cal X}$ and $p,q$ be probability distributions on ${\cal X}$. If $f\in{\cal F}$, then

The inequality in the theorem is the monotonicity of the $f$-divergence. A particular case is

Since the divergence is a kind of informational distance, we want $D_{f}(p||p)=0$ and require $f(1)=0$. This is nothing else but a normalization,

A bit more generally, we can say that if $f(x)-g(x)$ is a linear function, then $D_{f}$ and $D_{g}$ are essentially the same quantities.

It is interesting to remark that $qf(p/q)$ can be considered also as a mean of $p$ and $q$. In that case the mean of $p$ and $p$ should be $p$, so in the theory of means $f(1)=1$ is a different natural requirement.

Set $f^{*}(x)=xf(x^{-1})$. Then $D_{f}(p||q)=D_{f^{*}}(q||p)$. The equality $f^{*}=f$ is the symmetry condition.

is the variational distance of $p$ and $q$. $\square$

is the squared Hellinger distance of $p$ and $q$. $\square$

The limit $\alpha\to 0$ gives the relative entropy. $\square$

Several other functions appeared in the literature, we list a few of them:

The following result of Csiszár is a characterization (or axiomatization) of the $f$-divergence.

$C(p,q)$ is invariant under the permutations of the basic set ${\cal X}$.

if ${\cal A}$ is a partition of ${\cal X}$, then $C(p_{\cal A},q_{\cal A})\leq C(p,q)$ and the equality holds if and only if

Quantum quasi-entropy

In the mathematical formalism of quantum mechanics, instead of $n$-tuples of numbers one works with $n\times n$ complex matrices. They form an algebra and this allows an algebraic approach. In this approach, a probability density is replaced by a positive semidefinite matrix of trace 1 which is called density matrix. The eigenvalues of a density matrix give a probability density. However, this is not the only probability density provided by a density matrix. If we rewrite the matrix in a certain orthonormal basis, then the diagonal element $p_{1},p_{2},\dots,p_{n}$ form a probability density.

This concept was introduced in , see also Chapter 7 in and it is the quantum generalization of the $f$-entropy of Csiszár used in classical information theory (and statistics) .

for every number $0<\lambda<1$ and for positive definite square matrices $A$ and $B$ (of the same size). In the other condition the number $\lambda$ is (heuristically) replaced by a matrix:

Let $\alpha:{\cal M}_{0}\to{\cal M}$ be a mapping between two matrix algebras. The dual $\alpha^{*}:{\cal M}\to{\cal M}_{0}$ with respect to the Hilbert-Schmidt inner product is positive if and only if $\alpha$ is positive. Moreover, $\alpha$ is unital if and only if $\alpha^{*}$ is trace preserving. $\alpha:{\cal M}_{0}\to{\cal M}$ is called a Schwarz mapping if

The quasi-entropies are monotone and jointly convex .

holds for $A\in{\cal M}_{0}$ and for invertible density matrices $\rho_{1}$ and $\rho_{2}$ from the matrix algebra ${\cal M}$.

Proof: The proof is based on inequalities for operator monotone and operator concave functions. First note that

for a positive constant $c$. Due to the Schwarz inequality (9), we may assume that $f(0)=0$.

Let $\Delta:=\Delta(\rho_{1}/\rho_{2})$ and $\Delta_{0}:=\Delta(\alpha^{*}(\rho_{1})/\alpha^{*}(\rho_{2}))$. The operator

since the Schwarz inequality is applicable to $\alpha$. A similar simple computation gives that

Since $f$ is operator monotone, we have $f(\Delta_{0})\geq f(V^{*}\Delta V)$. Recall that $f$ is operator concave, therefore $f(V^{*}\Delta V)\geq V^{*}f(\Delta)V$ and we conclude

Application to the vector $A\alpha^{*}(\rho_{2})^{1/2}$ gives the statement. $\square$

It is remarkable that for a multiplicative $\alpha$ we do not need the condition $f(0)\geq 0$. Moreover, $V^{*}\Delta V=\Delta_{0}$ and we do not need the matrix monotonicity of the function $f$. In this case the only condition is the matrix concavity, analogously to Theorem 1.

If we apply the monotonicity (10) to the embedding $\alpha(X)=X\oplus X$ of ${\cal M}$ into ${\cal M}\oplus{\cal M}$ and to the densities $\rho_{1}=\lambda E_{1}\oplus(1-\lambda)F_{1}$, $\rho_{2}=\lambda E_{2}\oplus(1-\lambda)F_{2}$, then we obtain the joint concavity of the quasi-entropy:

Let $\rho_{1}$ and $\rho_{2}$ be density matrices in ${\cal M}$. If in certain basis they have diagonal $p=(p_{1}.p_{2},\dots,p_{n})$ and $q=(q_{1},q_{2},\dots,q_{n})$, then the monotonicity theorem gives the inequality

for a matrix convex function $f$. If $\rho_{1}$ and $\rho_{2}$ commute, them we can take the common eigenbasis and in (14) the equality appears. It is not trivial that otherwise the inequality is strict.

If $\rho_{1}$ and $\rho_{2}$ are different, then there is a choice for $p$ and $q$ such that they are different as well. Then

Conversely, if $S_{f}(\rho_{1}\|\rho_{2})=0$, then $p=q$ for every basis and this implies $\rho_{1}=\rho_{2}$. For the relative entropy, a deeper result is known. The Pinsker-Csiszár inequality says that

It would be interesting to extend Theorem 3 of Csiszár to the quantum case. If we require monotonicity and specify the condition for equality, then a function $f$ is provided by Theorem 3, but for non-commuting densities the conclusion is not clear.

is matrix monotone decreasing for $\alpha\in(-1,1)$. (For $\alpha=0$, the limit is taken and it is $-\log x$.) Then the relative entropies of degree $\alpha$ are produced:

These quantities are essential in the quantum case. $\square$

If $\rho_{2}=\rho_{1}=\rho$ and $A,B\in{\cal M}$ are arbitrary, then one can approach to the generalized covariance .

The usual symmetrized covariance corresponds to the function $f(t)=(t+1)/2$:

The interpretation of the covariances is not at all clear. In the next section they will be called quadratic cost functions. It turns out that there is a one-to-one correspondence between quadratic cost functions and Fisher informations.

Fisher information

The Cramér-Rao inequality belongs to the basics of estimation theory in mathematical statistics. Its quantum analog was discovered immediately after the foundation of mathematical quantum estimation theory in the 1960’s, see the book of Helstrom, or the book of Holevo for a rigorous summary of the subject. Although both the classical Cramér-Rao inequality and its quantum analog are as trivial as the Schwarz inequality, the subject takes a lot of attention because it is located on the highly exciting boundary of statistics, information and quantum theory.

This condition holds if $A$ is an unbiased estimator for $\theta$, that is

To require this equality for all values of the parameter is a serious restriction on the observable $A$ and we prefer to use the weaker condition (20).

Let $\varphi_{0}[K,L]$ be an inner product (or quadratic cost function) on the linear space of self-adjoint matrices. When $\rho(\theta)$ is smooth in $\theta$, as already was assumed above, then

with some $L=L^{*}$. From (20) and (22), we have $\varphi_{0}[A,L]=1$ and the Schwarz inequality yields

This is the celebrated inequality of Cramér-Rao type for the locally unbiased estimator.

The right-hand-side of (23) is independent of the estimator and provides a lower bound for the quadratic cost. The denominator $\varphi_{0}[L,L]$ appears to be in the role of Fisher information here. We call it quantum Fisher information with respect to the cost function $\varphi_{0}[{\,\cdot\,},{\,\cdot\,}]$. This quantity depends on the tangent of the curve $\rho(\theta)$. If the densities $\rho(\theta)$ and the estimator $A$ commute, then

We want to conclude from the above argument that whatever Fisher information and generalized variance are in the quantum mechanical setting, they are very strongly related. In an earlier work we used a monotonicity condition to make a limitation on the class of Riemannian metrics on the state space of a quantum system. The monotone metrics are called Fisher information quantities in this paper.

Since the sufficient and necessary condition for the equality in the Schwarz inequality is well-known, we are able to analyze the case of equality in (23). The condition for equality is

2 Coarse-graining and monotonicity

In the simple setting in which the state is described by a density matrix, a coarse-graining is an affine mapping sending density matrices into density matrices. Such a mapping extends to all matrices and provides a positivity and trace preserving linear transformation. A common example of coarse-graining sends the density matrix $\rho_{12}$ of a composite system $1+2$ into the (reduced) density matrix $\rho_{1}$ of component 1. There are several reasons to assume completely positivity about a coarse graining and we do so.

Assume that $\rho(\theta)$ is a smooth curve of density matrices with tangent $A:=\dot{\rho}$ at $\rho$. The quantum Fisher information $F_{\rho}(A)$ is an information quantity associated with the pair $(\rho,A)$, it appeared in the Cramér-Rao inequality above and the classical Fisher information gives a bound for the variance of a locally unbiased estimator. Let now $\beta$ be a coarse-graining. Then $\beta(\rho(\theta))$ is another curve in the state space. Due to the linearity of $\beta$, the tangent at $\beta(\rho_{0})$ is $\beta(A)$. As it is usual in statistics, information cannot be gained by coarse graining, therefore we expect that the Fisher information at the density matrix $\rho_{0}$ in the direction $A$ must be larger than the Fisher information at $\beta(\rho_{0})$ in the direction $\beta(A)$. This is the monotonicity property of the Fisher information under coarse-graining:

Although we do not want to have a concrete formula for the quantum Fisher information, we require that this monotonicity condition must hold. Another requirement is that $F_{\rho}(A)$ should be quadratic in $A$, in other words there exists a non-degenerate real bilinear form $\gamma_{\rho}(A,B)$ on the self-adjoint matrices such that

The requirements (26) and (27) are strong enough to obtain a reasonable but still wide class of possible quantum Fisher informations.

for every coarse graining $\beta$. ($\beta^{*}$ stand for the adjoint of $\beta$ with respect to the Hilbert-Schmidt product. Recall that $\beta$ is completely positive and trace preserving if and only if $\beta^{*}$ is completely positive and unital.) On the other hand the latter condition is equivalent to

$\gamma_{\rho}(A,A)$ is continuous in $\rho$ for every fixed $A$,

$\gamma_{\rho}(A,A)=\gamma_{\rho}(A^{*},A^{*})$,

The above $\gamma_{\rho}(A,A)$ is formally a quasi-entropy, $S_{1/f}^{A\rho^{-1}}(\rho,\rho)$, however this form is not suitable to show the monotonicity. Assume that $\rho=\mbox{Diag}\,(\lambda_{1},\lambda_{2},\dots,\lambda_{n})$. Then

It is clear from this formula that the Fisher information is affine in the function $1/f$. Therefore, Hansen’s canonical representation of the reciprocal of a standard operator monotone function can be used .

where $\mu$ is a probability measure on $$.

The theorem implies that the set $\{1/f:f\mbox{\ is\ standard\ operator \ monotone}\}$ is convex and gives the extremal points

Hence $g_{\lambda}$ is decreasing in the parameter $\lambda$. For $\lambda=0$ we have the largest function $g_{0}(t)=(t+1)/(2t)$ and for $\lambda=1$ the smallest is $g_{1}(t)=2/(t+1)$. (Note that this was also obtained in the setting of positive operator means , harmonic and arithmetic means.)

which is a quadratic cost functional. According to (30) (or Theorem 4) this possesses the monotonicity property

Since (29) and (30) are equivalent we observe a one-to-one correspondence between monotone Fisher informations and monotone quadratic cost functions.

$\varphi_{\rho}[A,A]$ is continuous in $\rho$ for every fixed $A$,

$\varphi_{\rho}[A,A]=\varphi_{\rho}[A^{*},A^{*}]$,

Among the standard operator monotone functions, $f_{a}(t)=(1+t)/2$ is maximal. This leads to the fact that among all monotone quantum Fisher informations there is a smallest one which corresponds to the function $f_{a}(t)$. In this case

For the purpose of a quantum Cramér-Rao inequality the minimal quantity seems to be the best, since the inverse gives the largest lower bound. In fact, the matrix $L$ has been used for a long time under the name of symmetric logarithmic derivative, see and . In this example the quadratic cost function is

To see the second formula of (37), set $A(t):=e^{-t\rho/2}Ae^{-t\rho/2}$. Then

Let $T=T^{*}$ and $\rho_{0}$ be a density matrix. Then $D(\theta):=\exp(\theta T/2)\rho_{0}\exp(\theta T/2)$ satisfies the differential equation

and $T$ is a locally unbiased estimator (of the parameter $\theta$ at $\theta=0$). Since

we have equality in the Cramér-Rao inequality, see (25). $\square$

Apart from a constant factor this expression is the skew information proposed by Wigner and Yanase some time ago (). In the limiting cases $\beta\to 0$ or $1$ we have

is named after Kubo, Mori, Bogoliubov etc. The Kubo-Mori inner product plays a role in quantum statistical mechanics (see , for example). In this case

Therefore the corresponding quadratic cost functional is

Note that (45) is again an exponential family, the differential equation for

It would be interesting to find more exponential families. This means solution of the differential equation

If the self-adjoint $T$ and the positive $\rho$ commute, then the solution is $D(\theta)=\exp(\theta T)\rho_{0}$. A concrete example is

3 Manifolds of density matrices

Let $\mathcal{M}:=\{\rho(\theta):\theta\in G\}$ be a smooth $m$-dimensional manifold of invertible density matrices. When a quadratic cost function $\varphi_{0}$ is fixed, the corresponding Fisher information is a Riemannian metric on the manifold. This gives a possibility for geometric interpretation of statistical statements .

Fisher information appears not only as a Riemannian metric but as an information matrix as well. The quantum score operators (or logarithmic derivatives) are defined as

is the quantum Fisher information matrix.

The next result is the monotonicity of Fisher information matrix.

Let $\beta$ be a coarse-graining sending density matrices on the Hilbert space $\mathcal{H}_{1}$ into those acting on the Hilbert space $\mathcal{H}_{2}$ and let $\mathcal{M}:=\{\rho(\theta):\theta\in G\}$ be a smooth $m$-dimensional manifold of invertible density matrices on $\mathcal{H}_{1}$. For the Fisher information matrix $I^{1Q}(\theta)$ of $\mathcal{M}$ and for Fisher information matrix $I^{2Q}(\theta)$ of $\beta(\mathcal{M}):=\{\beta(\rho(\theta)):\theta\in G\}$ we have the monotonicity relation

Assume that $F_{j}$ are positive operators acting on a Hilbert space $\mathcal{H}_{1}$ on which the family $\mathcal{M}:=\{\rho(\theta):\theta\in G\}$ is given. When $\sum_{j=1}^{n}F_{j}=I$, these operators determine a measurement. For any $\rho(\theta)$ the formula

gives a diagonal density matrix. Since this family is commutative, all quantum Fisher informations coincide with the classical (24) and the classical Fisher information stand on the left-hand-side of (48). The right-hand-side can be arbitrary quantum quantity but it is minimal if it based on the symmetric logarithmic derivative, see Example 6. This particular case of the Theorem is in the paper .

Assume that a manifold ${\cal M}:=\{\rho(\theta):\theta\in G\}$ of density matrices is given together a statistically relevant Riemannian metric $\gamma$. Given two points on the manifold their geodesic distance is interpreted as the statistical distinguish-ability of the two density matrices in some statistical procedure.

Let $\rho_{0}\in{\cal M}$ be a point on our statistical manifold. The geodesic ball

contains all density matrices which can be distinguished by an effort smaller than $\varepsilon$ from the fixed density $\rho_{0}$. The size of the inference region $B_{\varepsilon}(\rho_{0})$ measures the statistical uncertainty at the density $\rho_{0}$. Following Jeffrey’s rule the size is the volume measure determined by the statistical (or information) metric. More precisely, it is better to consider the asymptotics of the volume of $B_{\varepsilon}(\rho_{0})$ as $\varepsilon\to 0$. It is known in differential geometry that

where $m$ is the dimension of our manifold, $C_{m}$ is a constant (equals to the volume of the unit ball in the Euclidean $m$-space) and $Scal$ means the scalar curvature, see [13, 3.98 Theorem]. In this way, the scalar curvature of a statistically relevant Riemannian metric might be interpreted as the average statistical uncertainty of the density matrix (in the given statistical manifold). This interpretation becomes particularly interesting for the full state space endowed by the Kubo-Mori inner product as a statistically relevant Riemannian metric.

The Kubo-Mori (or Bogoliubov) inner product is given by

or (42) in the affine parametrization. On the basis of numerical evidences it was conjectured in that the scalar curvature which is a statistical uncertainty is monotone in the following sense. For any coarse graining $\alpha$ the scalar curvature at a density $\rho$ is smaller than at $\alpha(\rho)$. The average statistical uncertainty is increasing under coarse graining. Up to now this conjecture has not been proven mathematically. Another form of the conjecture is the statement that along a curve of Gibbs states

the scalar curvature changes monotonly with the inverse temperature $\beta\geq 0$, that is, the scalar curvature is monotone decreasing function of $\beta$. (Some partial results are in .)

Let ${\cal M}$ be the manifold of all invertible $n\times n$ density matrices. If we use the affine parametrization, then the tangent space $T_{\rho}$ consists of the traceless self-adjoint matrices and has ab orthogonal decomposition

We denote the two subspaces by $T_{\rho}^{q}$ and $T_{\rho}^{c}$, respectively. If $A_{2}\in T_{\rho}^{c}$, then

independently of the function $f$. Moreover, if $A_{1}\in T_{\rho}^{q}$, then

Therefore, the decomposition (51) is orthogonal with respect to any Fisher information and any quadratic cost functional. Moreover, the effect of the function $f$ and the really quantum situation are provided by the components from $T_{\rho}^{q}$.

4 Skew information

Let $f$ be a standard function and $X=X^{*}\in M_{n}$. The quantity

was called skew information in in this general setting. The skew information is nothing else but the Fisher information restricted to $T_{\rho}^{q}$, but it is parametrized by the commutator.

If $\rho=\mbox{Diag}\,(\lambda_{1},\dots,\lambda_{n})$ is diagonal, then

The proof of the lemma is elementary. From the lemma and Theorem 6, Theorem 9 follows straightforwardly .

The skew information is the Hessian of a quasi-entropy:

We compute the Hessian of the relative entropy of degree $\alpha$ in an exponential parametrization:

for $\alpha\leq 1/2$ and for $\alpha\geq 1/2$ $g_{\alpha}=g_{1-\alpha}$.

where $g_{\alpha}$ is as above. $\square$

If $\alpha=0$, then we have the Kubo-Mori inner product. $\square$

Von Neumann algebras

Let ${\cal M}$ be a von Neumann algebra. Assume that it is in standard form, it acts on a Hilbert space ${\cal H}$, ${\cal P}\subset{\cal H}$ is the positive cone and $J:{\cal H}\to{\cal H}$ is the modular conjugation. Let $\varphi$ and $\omega$ be normal states with representing vectors $\Phi$ and $\Omega$ in the positive cone. For the sake of simplicity, assume that $\varphi$ and $\omega$ are faithful. This means that $\Phi$ and $\Omega$ are cyclic and separating vectors. The closure of the unbounded operator $A\Phi\mapsto A^{*}\Omega$ has a polar decomposition $J\Delta(\omega/\varphi)^{1/2}$ and $\Delta(\omega/\varphi)$ is called relative modular operator. $A\Phi$ is in the domain of $\Delta(\omega/\varphi)^{1/2}$ for every $A\in{\cal M}$.

was introduced in , see also Chapter 7 in . Of course, (5) is a particular case.

holds for $A\in{\cal M}_{0}$ and for normal states $\omega$ and $\varphi$ of the von Neumann algebra ${\cal M}$.

The relative entropies are jointly convex in this setting similarly to the finite dimensional case. Now we shall concentrate on the generalized variance.

where $\Delta(\omega/\omega)$ is actually the modular operator. Although $\Delta(\omega/\omega)$ is unbounded, the definition works. For the function $f$, the inequality

holds. Therefore $A\Omega$ is in the domain of $\sqrt{f(\Delta(\omega/\omega))}$.

is a particular case of Theorem 11 and it is the monotonicity of the generalized covariance under coarse-graining. The common symmetrized covariance

is recovered by the particular case $f(t)=(1+t)/2$.

it is enough to consider these sesquilinear forms on the subspace $T_{\omega}:=\{A\in{\cal M}:\omega(A)=0\}$.

2 The Cramér-Rao Inequality

Let $\{\omega_{\theta}:\theta\in G\}$ be a smooth $m$-dimensional manifold in the set of normal states of the von Neumann algebra ${\cal M}$ and assume that a collection $A=(A_{1},\dots,A_{m})$ of self-adjoint operators is used to estimate the true value of $\theta$. The subspace spanned by $A_{1},A_{2},\dots,A_{m}$ is denoted by $V$.

Given a standard matrix monotone function $f$, we have the corresponding cost function

for every $\theta$ and the cost matrix of the estimator $A$ is a positive semidefinite matrix, defined by

For an unbiased estimator we have $b(\theta)=0$. From the bias vector we form a bias matrix

For a locally unbiased estimator at $\theta_{0}$, we have $B(\theta_{0})=0$.

determines the logarithmic derivatives $L_{i}(\theta)$. The Fisher information matrix is

Let $A=(A_{1},\dots,A_{m})$ be an estimator of $\theta$. Then for the above defined quantities the inequality

holds in the sense of the order on positive semidefinite matrices.

3 Uncertainty relation

In the von Neumann algebra setting the skew information (as a sesquilinear form) can be defined as

if $\omega(X)=\omega(Y)=0$. (Then $I_{\omega}^{f}(X)=I_{\omega}^{f}(X,X)$.)

Let ${\cal K}$ be a Hilbert space with inner product $\langle\!\langle{\,\cdot\,},{\,\cdot\,}\rangle\!\rangle$ and let $\langle{\,\cdot\,},{\,\cdot\,}\rangle$ be a sesquilinear form on ${\cal K}$ such that

holds for every $f_{1},f_{2},\dots,f_{m}\in{\cal K}$.

by assumption. This says that $G-H$ is positive semidefinite, hence it is clear that $G\geq H$. $\square$

Proof: Let $E({\,\cdot\,})$ be the spectral measure of $\Delta(\omega,\omega)$. Then for $m=1$ the inequality is

where $d\mu(\lambda)=d\langle A\Omega,E(\lambda)A\Omega\rangle$. Since the inequality

and this implies the integral inequality.

Consider the finite dimensional subspace ${\cal N}$ generated by the operators $A_{1},A_{2},\dots,A_{m}$. On ${\cal N}$ we have the inner products

Since $\langle A,A\rangle\leq\langle\!\langle A,A\rangle\!\rangle$, the determinant inequality holds, see Lemma 2. $\square$

This theorem is interpreted as quantum uncertainty principle . In the earlier works the function $g$ from the left-hand-side was $(x+1)/2$ and the proofs were more complicated. The general $g$ appeared in .

References