According to the Pauli objection (see for example here or the answer to this question) there can be no time operator $\hat{T}$ canonically conjugate to the Hamiltonian $\hat{H}$ of a physical system since this would imply that the spectrum of $\hat{H}$ would be continuous and unbounded. However, although the spectrum of the momentum operator is unbounded, it is very often quantised, so why does the Pauli objection not also disqualify the existence of the position operator $\hat{x}$?
The argument would go along similar lines. We start by assuming that $\hat{x}$ is in fact canonically conjugate to the momentum operator $\hat{p}$. That is, the two have the commutation relations
\begin{equation} [\hat{x}, \hat{p}] = i\hbar \end{equation}
(which is generally accepted). We then define
\begin{equation} U_{q} = e^{-iq\hat{x}/\hbar} \end{equation}
where $q$ is a real number with dimensions of momentum. It can then be shown that
\begin{equation} U_{q}^{\dagger}\hat{p}U_{q} = \hat{p} - q. \end{equation}
Since $U_{q}$ is unitary, multiplying on the left with $U_{q}$, we have
\begin{equation} \hat{p}U_{q} = U_{q}\hat{p} - qU_{q}. \end{equation}
If this then acts on an eigenvector of $\hat{p}$, $\left|p_{i}\right>$ with eigenvalue $p_{i}$, we get
\begin{equation} \hat{p}U_{q}\left|p_{i}\right> = \left(p_{i} - q\right)U_{q}\left|p_{i}\right>. \end{equation}
This then implies that $U_{q}\left|p_{i}\right>$ is an eigenvector of $\hat{p}$ with eigenvalue $p_{i} - q$. Since $q$ is a real number, this implies that the spectrum of $\hat{p}$ is unbounded (which is not a problem for momentum) and continuous, which, in general, is a problem. For example, imposing fixed or periodic boundary conditions on an otherwise free particle leads to solutions exhibiting discrete momentum eigenvalues.
How does one argue around this to retain the canonically conjugate relation between $\hat{x}$ and $\hat{p}$?
Edit Please note that this is not a question about whether or not time is an observable, so a very different question to is there an observable of time?. It is a question about the nature of the Pauli objection itself and whether, on the face of it, it should not also disqualify the position operator. Please also note, I do not think that there shouldn't be a position operator!