How is the quantum relative entropy $S(\rho\|\sigma)$ defined when $\sigma$ is a pure state?

Question

I want to evalualte the quantum relative entropy $S(\rho|| \sigma)=-{\rm tr}(\rho {\rm log}(\sigma))-S(\rho)$, where $\sigma=|\Psi\rangle\langle\Psi|$ is a density matrix corresponding to a pure state and $\rho$ is a density matrix corresponding to an arbitrary mixed state. Here, $S(\rho)$ simply denotes the Von Neumann entropy of $\rho$. Given that $\sigma$ is diagonal, with eigenvalues $0$ and $1$ it seems that the first term in the quantum relative entropy will in general be infinite. As $S(\rho)\leq {\rm log}(d)$, where $\rho \in L({\mathcal H}^{d})$, the first term dominates and the quantum relative entropy is also infinite. Is this correct? And if so, what's the intuition behind this fact?

Answer 1

If $\sigma$ is not full rank, then the correct way to interpret the quantum relative entropy formula you wrote is to assign it the value of $+\infty$ when the support of $\rho$ is not included in the support of $\sigma$. Wikipedia has a nice explanation of how to interpret this, but you can think that the reason for which the quantum relative entropy is finite in that case is that $\lim_{x\to 0} x \log(x) = 0$.

In your case, the support of $\sigma$ is simply the one-dimensional subspace spanned by $|\Psi\rangle$. For $\rho$ to be supported on this subspace, it must hold that $\rho$ is also pure, so it must be also equal to $|\Psi\rangle\!\langle\Psi|$, and so $\rho=\sigma$. In this case, the quantum relative entropy vanishes.

So in summary, if $\sigma$ is pure, than $S(\rho||\sigma)$ is either $0$ (when $\rho=\sigma$) or $+\infty$ (in the other cases).

Answer 2

I'd like to add to Angelo Lucia's answer slightly. It is not very surprising that $S(\rho \| \sigma)$ can take the value $+\infty$ once we realise that the relative entropy is a generalization of the Kullback-Liebler divergence $D(p \| q)$ between probability distributions $p$ and $q$. Formally, given two distributions $p,q$ over some finite set $\mathcal{X}$ the KL-divergence is defined as $$ D(p\| q) = \begin{cases} \sum_{x \in \mathcal{x}} p(x) \log \frac{p(x)}{q(x)} \quad & \text{if } \mathrm{supp}(p) \subseteq \mathrm{supp}(q) \\ + \infty & \text{otherwise} \end{cases} $$ where $\mathrm{supp}(p) = \{x \in \mathcal{X} : p(x)> 0\}$. Note that if we fix a basis and consider only diagonal states in that basis, i.e. $\rho = \sum_{x} p(x) |x\rangle \langle x |$ and $\sigma = \sum_x q(x) |x\rangle \langle x |$, then computing $S(\rho \| \sigma)$ we recover exactly the KL-divergence $D(p \| q)$. The analogous situation to taking $\sigma$ to be a pure state is taking $q$ to be some point distribution (delta-distribution). In this case we see that $D(p\| q)$ is finite iff $p=q$, which is exactly what we observe in the quantum case for pure states.