It is a common argument in the theory of kinematic groups (the groups of motions for a spacetime) that the subgroups generated by boosts must be non-compact...
... and it's common for the assumption to not be made. For instance, in the comprehensive classification of the Kinematical Lie algebras via deformation theory here in dimensions $3 + 1$, $D + 1 (D > 3)$ and $D + 1 (D < 3)$, which is a superset of the Bacry / Lévi-Leblond (BLL) classification (your [1], Possible Kinematics), all the cases are included, including those where the boosts are compact.
The BLL classification included the assumption to keep the signature $3+1$. The excluded groups are those associated with a $4+0$ signature: the 4D Euclidean group, the symmetry groups for the positive (hyperspherical) and negative (hyperbolic) curvature 4D spaces.
However, the assumption mars the uniformity of the larger BLL classification that includes the extra groups. In the extra groups, boosts are spatial rotations and wrap around. Avoiding this, and the wrap-around associated with it, is what the intent stated in the Possible Kinematics paper was, in as many words.
It's actually cleaner, instead, to keep the groups in, and excluding them may exclude applications, such as Euclideanization. It may even clarify that issue, since there are three different routes to Euclideanization under this classification - a distinction that's not normally otherwise made. In the summary below, the transition is from $γ > 0$ to $γ = 0$ to $γ < 0$. One route goes through the Galilei group, with $β ≠ 0$, the second goes through the Carroll group with $α ≠ 0$, and the third goes through the static group with $(α,β) = (0,0)$.
You can see everything more clearly by laying out the groups in the BLL classification explicitly as a 3-paramter family. The best way to do this is to combine all the basis elements into generic Lie vectors
$$
Λ = · + · + · - τ H[ + ψ μ],\quad Λ' = '· + '· + '· - τ' H[ + ψ' μ]$$
and write
$$\begin{align}
[Λ,Λ']
&= (×' - γ ×' + λ ×')· + (×' + ×')· + (×' + ×')·\\
&+ (·' - ·') M + β (τ' - τ')· + κ (τ' - τ')·
\end{align}$$
where the central charge $μ$ is also added in, and $M = μ + α H$. (The central extension is "trivial" when $α ≠ 0$, though that won't be of any concern here or below.) The larger deformation-theoretic transformation, cited above, has several other branches. The three-parameter BLL family is just one branch.
The cases $γ < 0$ are the ones where the $4 + 0$ signature arises. Otherwise, when $γ ≥ 0$, the case $β → 0$ corresponds to $c → 0$, while $α → 0$ corresponds to $c → ∞$, with $c = \sqrt{β/α}$.
This way of organizing the Lie brackets helps make the picture more transparent: $$, $$, $$ and $τ$ correspond to infinitesimal forms, respectively, of rotation, boost, spatial translation and time translation. The extra parameter $ψ$, associated with the central charge, corresponds to a translation generator in a fifth dimension for a 5-D geometric representation that I won't get into here. (When all the Lie algebras are centrally extended, there is a uniform 5-D geometric representation for the whole BLL family, though it is non-linear when $κ ≠ 0$.)
Their respective actions on $(,,,H,M,μ)$ can be seen by applying the infinitesimal transform $δ = \{\_,Λ\}$ to each one, with the results being:
$$
δ = × + × + ×,\quad δ = × - γ × + M - β τ ,\\
δ = × - M + λ × - κ τ ,\quad δH = - β · - κ ·,\\
δM = - γ · - λ ·,\quad δμ = 0.
$$
(Technically, this is the co-adjoint action, where $(,,,H,M,μ)$ are treated as coordinates in the dual of the Lie algebras, rather than as basis elements of the Lie algebras.)
Now, apply each of the infinitesimal transforms separately to see what happens. Along the way, we'll also see what happens with the boosts. Depending on how the parameters are set, they will either go hyperbolic, parabolic or elliptical.
For infinitesimal rotations, the process is easy: $δ = ×$ for vectors $ ∈ \{,,\}$ and $δS = 0$, for scalars $S ∈ \{H,M,μ\}$. Upon exponentiation, we obtain the finite forms of the respectively transforms $x → e^{sδ} x$. For vectors and scalars, they are respectively:
$$ → + \frac{\sin θ}θ × + \frac{1 - \cos θ}{θ^2} ×(×),\quad S → S,$$
where
$$θ = s ω,\quad = s .$$
For infinitesimal time translations,
$$δ(,,,H,M,μ) = (,-βτ,-κτ,0,0,0),$$
you have the characteristic equations $δ^2 = ζ τ^2$ for $\{,\}$, where $ζ = β κ$, and $δ = 0$ for $\{,H,M,μ\}$. This leads to trivial transforms, in finite form of a translation by $b = s τ$, for the latter set
$$ → ,\quad H → H,\quad M → M,\quad μ → μ.$$
For the former set, it leads to the following finite forms
$$ → e_0(b,ζ) - β e_1(b,ζ) ,\quad → e_0(b,ζ) - κ e_1(b,ζ) .$$
The auxiliary functions are defined by
$$e_i(x,z) = \sum_{n≥0} z^n \frac{x^{2n+i}}{(2n+i)!},$$
with $e_0$, $e_1$ and $e_2$ (which we'll use below), branching into three cases
$$\begin{align}
z = A^2 > 0 &→ \left(e_0(x,z),e_1(x,z),e_2(x,z)\right) = \left(\cosh Ax, \frac{\sinh Ax}A,\frac{\cosh A x - 1}{A^2}\right),\\
z = 0 &→ \left(e_0(x,z),e_1(x,z),e_2(x,z)\right) = (1,x,\frac{x^2}2),\\
z = -B^2 < 0 &→ \left(e_0(x,z),e_1(x,z),e_2(x,z)\right) = \left(\cos Bx, \frac{\sin Bx}{B},\frac{1 - \cos B x}{B^2}\right).
\end{align}$$
Time translation is hyperbolic if $ζ > 0$, circular if $ζ < 0$ and linear if $ζ = 0$.
Spatial translations have the following infinitesimal form:
$$δ = ×,\quad δ = M,\quad δ = λ ×,\quad δH = - κ ·,\quad δM = - λ ·,\quad δμ = 0.$$
This satisfies the characteristic equation $δ = 0$ for $μ$, $δ^2 = -λ ε^2$ for $M$, $δ^3 = -λ ε^2 δ$ for $\{,,,H\}$. Correspondingly, the finite forms of the spatial translation, with $ = s $, are
$$\begin{align}
&→ + \frac{e_1(a,-λ)}a × + λ \frac{e_2(a,-λ)}{a^2} × ( × ),\\
&→ + \frac{e_1(a,-λ)}a M - λ \frac{e_2(a,-λ)}{a^2} ·,\\
&→ + λ \frac{e_1(a,-λ)}a × + λ \frac{e_2(a,-λ)}{a^2} × ( × ),\\
H &→ H - κ \frac{e_1(a,-λ)}a · - κ e_2(a,-λ) M,\\
M &→ e_0(a,-λ) M - λ \frac{e_1(a,-λ)}a ·,\\
μ &→ μ.
\end{align}$$
Spatial translation is hyperspherical if $λ > 0$, hyperbolic if $λ < 0$ and flat if $λ = 0$.
Finally, we get to the boosts, which have the following infinitesimal forms:
$$
δ = ×,\quad δ = -γ ×,\quad δ = - M,\quad δH = -β ·,\quad δM = -γ ·,\quad δμ = 0.
$$
The situation is analogous to that for the spatial translations, with $(λ,κ)$ replaced by $(γ,β)$ and $(,)$ swapped with $(,)$, up to sign. The characteristic equations are $δ = 0$ for $μ$, $δ^2 = γ υ^2$ for $M$ and $δ^3 = γ υ^2$ for $(,,,H)$. The finite form of the boost, with $ = s $ are:
$$\begin{align}
&→ + \frac{e_1(v,γ)}v × - γ \frac{e_2(v,γ)}{v^2} × ( × ),\\
&→ - \frac{e_1(v,γ)}v M + γ \frac{e_2(v,γ)}{v^2} ·,\\
&→ - γ \frac{e_1(v,γ)}v × - γ \frac{e_2(v,γ)}{v^2} × ( × ),\\
H &→ H - β \frac{e_1(v,γ)}v · + β e_2(v,γ) M,\\
M &→ e_0(v,γ) M - γ \frac{e_1(v,γ)}v ·,\\
μ &→ μ.
\end{align}$$
When $γ > 0$, the boosts use hyperbolic functions; when $γ < 0$, they use circular functions and wrap around, while the case $γ = 0$ is the parabolic case, analogous to the Galilean transform. The $γ < 0$ cases are the ones where the boosts are compact. Depending on the sign of $λ$, as already noted, they are hyperspherical, hyperbolic or flat and Euclidean. Since $ζ = α^2 γ λ$ and $α ≠ 0$ when $γ < 0$, then $ζ$ and $λ$ have opposite signs in the case of compact boosts. The time translations are hyperbolic if and only if the spatial translations are; and they are linear / flat, if and only if the spatial translations are.
