Consider a vector bundle $E \overset{\pi}{\rightarrow} M$. In your question, you construct a diffeomorphism $\phi=x\circ y^{-1}:M\rightarrow M$ out of two global charts $x, y:M\rightarrow \mathbb{R}^4$. To me, it seems that you say that $\phi$ defines a transformation of the space of sections:
\begin{align}
\Phi:\rm{Diff}(M)\times\Gamma(M,E)&\rightarrow \Gamma(M,E) \\
(\phi, X) &\mapsto (p \mapsto X_{\phi(p)})
\end{align}
asuming that we are identifying of all the fibers $\pi^{-1}(p)$ for $p\in M$ using the flat connection associated to the flat metric $g$. Of course this defines some action of the Poincaré group.
However, for some suitable bundle a non-equivalent action of the Poincaré group can be defined. Consider a representation of the Lorentz group $R:SO(1,3)\rightarrow \rm{End}(V)$ and suppose that the fibers of our bundle are isomorphic to (and identified with) $V$. The action over $\Gamma(M,E)$ of the Poincaré group that we want is
\begin{align}
\Psi_R: \rm{Iso}(M)\times\Gamma(M,E )&\rightarrow \Gamma(M,E) \\
(\phi, X) &\mapsto (p \mapsto R(d\phi) X_{\phi(p)})
\end{align}
This is the action that is used in physics. It includes as a particular case the restriction of $\Phi$ from $\rm{Diff}(M)$ to $\rm{Iso}(M)$. This can be seen substituting $R$ by the tensor product of the trivial representation with itself. $\Psi_R$ is the action that involves the Lorentz matrices, but only when $R$ is the fundamental representation.
Now, it is true that for every Poincaré transformation $\phi$ there's an associated coordinate transformation $C^\infty(M,\mathbb{R}^4)\rightarrow
C^\infty(M,\mathbb{R}^4):x\mapsto \phi\circ x$. There's also a "trivial" action of every Poincaré transformation over the space of sections: $\Phi$, but it's not the one used in physics and it doesn't involve Lorentz matrices. For the action used in physics we need some extra information, namely the representation $R$.
As for the reason why such a non-trivial action of the Poincaré group is used in physics, in special relativity the space-time symmetry group is $\rm{Iso}(M)\simeq ISO(1,3)$ (the reason for this is, if you want, experimental evidence) and we want to know how the relevant objects in the theory transform under it. In relativistic field theory, those objects are the fields: sections of the vector bundles equipped with some $\Psi_R$.
In general, the statement that the action $S:\Gamma(M,E)\rightarrow \mathbb{R}$ is invariant under some transformation $T:\Gamma(M,E)\rightarrow\Gamma(M,E)$ means that $S[X]=S[T(X)]$ for every section $X$.
The integral that defines the action can be written in coordinates. Thus the action may be seen as a functional $S_{\mathbb{R}^4}$ over sections of a bundle over $\mathbb{R}^4$ instead of the original over $M$. Then the trivial statement that a coordinate transformation $\phi:\mathbb{R}^4\rightarrow \mathbb{R}^4$ induces another action $S'_{\mathbb{R}^4}$ (a different functional) given by $S'_{\mathbb{R}^4}[X]=S_{\mathbb{R}^4}[X\circ \phi]$ is not equivalent to the invariance of the action under the transformation $\phi$ (it would be so only if $S'=S$). Every coordinate transformation induces a new action, but not all of them leave the action invariant (only the Poincaré transformations do).
