How can the cosmological constant be $1/R^2$ and still be constant in time?

Question

The cosmological constant $\Lambda$ is roughly the inverse square of the size $R$ of the observable universe, or horizon radius (with proper constants to get the right units). $\Lambda$ has a solid basis in observations. Observations also suggest that $\Lambda$ is the same for distant galaxies. Various research proposals even explain the value of $\Lambda$ with the radius of the observable universe. (If my reading of the literature is correct, no explanation of the value of $\Lambda$ unrelated to $R$ has ever been proposed, but I might be wrong.)

But if $\Lambda$ is related to the size of the observable universe $R$, its value should have been higher in the past, when the observable universe was smaller. This would imply that $\Lambda$ is not a constant, and that its value should be higher for distant galaxies. This last conclusion contradicts observations.

How can one reconcile these two contrasting arguments?

Edit: Note that I might not have distinguished $\Lambda$ and $\Omega_\Lambda$ clearly in this question. My question is about energy density: Measurements show that the energy density due to $\Lambda$ is constant over time. But then, why is $\Lambda \approx 1/R^2$ ($R$ being the present radius)? This coincidence that two numbers have the same value, one constant, the other time dependent, is really astonishing.

(Even Milgrom's https://arxiv.org/abs/2001.09729 does not attempt an answer.)

Edit: Many recent papers speculate that dark energy density (assuming this is Lambda) is related to the temperature and entropy of the horizon (like the Hawing radiation for a black hole). If one adds numbers, this come out about right. However, this would also mean that the temperature and dark energy density must have been higher in the past. And again, this last conclusion is not backed up by data.

Edit: As Koshi writes, the question is one of the great unsolved problems of physics. So there is no reason to expect a simple answer.

Answer 1

If I understand correctly, I think what you are referring is the following, and I will go trough all of the steps to get there: The cosmological constant pops up as a possible constant in the Einstein equations, as $\Lambda$, and this why it pops up in the Friedman equations: $$ H^2+\frac{k}{a^2}=\frac{8\pi G}{3}(\rho_m+\rho_r)+\Lambda.$$

To get everything to the same footing, let us introduce the constant (!) energy density in 'dark energy' by setting $\rho_{\Lambda}=3\Lambda/(8\pi G)$. We can then write the first equation as: $$ H^2+\frac{k}{a^2}=\frac{8\pi G}{3}(\rho_m+\rho_r+\rho_{\Lambda}).$$ We now introduce density parameters $\Omega_{i,0}$, where $i$ can relate to matter, radiation, the cosmological constant (i.e. dark energy) and curvature via $$\Omega_{i,0}\equiv\frac{\rho_{i}(t_0)}{\rho_{\mathrm{crit}}(t_0)},\quad i\in\{m,r,\Lambda\},\quad\Omega_{k,0}\equiv\frac{k}{a(t_0)^2H^2(t_0)},$$ where $\rho_{\mathrm{crit},0}\equiv 3/(8\pi G) H_0^2$ is the critical density of the universe that would lead to a flat universe TODAY and $H_0=H(t_0)$. Inserting the known scaling of the different energy densities with the scale factor for matter, radiation and dark energy, and the obvious scaling of $\Omega_k$ we can read of from the definition, and set $a_0=1$, we find the Friedman equation to be $$H^2(t)=H_0^2\left(\Omega_{r,0}a^{-4}+\Omega_{m,0}a^{-3}+\Omega_{k,0}a^{-2}+\Omega_{\Lambda,0}\right).$$ This equation tells us that to find $H$ at a certain time, we have to insert the density parameters we find today and scale them with their behavior with $a$. We do not have to scale $\Omega_{\Lambda}$, since $\rho_{\Lambda}$ is constant, but that does not mean that $\Omega_{\Lambda,t}$ is constant.

I think this what you are referring to is the following: You can find $\Lambda$ by measuring $\Omega_{\Lambda,0}$ and $H_0$ (where both measurements are far from trivial) and use our definition for the density parameter to get $\Lambda$ from that by $$\Lambda=\Omega_{\Lambda,0}H_0^2$$ This is where the $H_0^2=1/R_0^2$ dependence pops up that you mentioned. For some this raises the question, since $\Omega_{\Lambda}$ is roughly 1, why the value for dark energy is so related to today's horizon size i.e. to the miniscule value of the 'Hubble constant'. BUT if one looks back at the definition of $\Omega_{\Lambda,0}$ (and of $\rho_{\Lambda}$), one sees that the factors of $H_0^2$ cancel and there is no time dependence in $\Lambda$. (The interesting question 'why is $\Lambda\approx H_0^2$?' should therefore maybe be rephrased to 'How come we live in the short time where $\Omega_{\Lambda}$ and $\Omega_m$ are roughly equal??' or maybe just 'Why is the cosmological constant so small?') If you would evaluate the equation for $\Lambda$ at a different time, then also $\Omega_{\Lambda,t}$ would be different, since the critical density that was used to define this parameter is time dependent, with $H^2$! There can sometimes be confusion about the density parameters, since some people only use today's values and skip the 0 index for that matter.

Answer 2

Are you talking about the Hubble sphere/horizon rather than the observable universe (which are not the same)? If so, Hubble horizon is defined as $R=1/(aH)$ (Planck units), and for a cosmological constant driving the expansion (present era) we have $H\sim \sqrt{\Lambda}$, so yes, $\Lambda\sim 1/R^2$ (at the present time fixing $a=1$). But this relation doesn't hold if other sources contribute to the expansion, like in the earlier universe where radiation and matter were the main contributors, or even earlier when inflation took place.

Edit: Hubble horizon in general is defined as $1/(aH)$ with the scale factor $a$ and the Hubble parameter both time-dependent of course. So if $H$ is constant, Hubble radius is still time dependent due to $a(t)$. The inverse $H$ is called Hubble time.

Answer 3

The OP's question is a variation of this question, with a different start point. The answer, therefore, is very similar:

Its often conjectured that the observable universe will asymptote to a de Sitter state (dark energy only, no matter, flat universe).

It is well known that the de Sitter characteristic length $l_\Lambda$ (i.e. future cosmic event horizon radius) is related to the cosmological constant $\Lambda$ (dimensions $L^{-2}$):

\begin{equation} {\Lambda} = \frac{3}{l_\Lambda^2} \end{equation}
Also, in a dS state, the future Hubble Horizon equals the cosmic event horizon, so: \begin{equation} {l_\Lambda} = \frac{c}{H(\infty)} \end{equation} That is the OP's $1/R^2$ relationship (also pointed out via the Hubble parameter by the other answers); the $R$ is the future de Sitter cosmic event horizon radius, which is a fixed number. At the present time, the radius of the observable Universe is around 2.9 times the future dS CEH radius.

TLDR: $\Lambda$ is related to the future cosmic event horizon radius, which is fixed. As an energy density (i.e. vacuum energy) the cosmological constant remains also fixed over time. The radius of the observable universe, on the other hand, is not fixed.