I am trying to work out Non-Abelian gauge theories but I couldn't get my head around the fact that gauge fields transform with an extra inhomogeneous term under the adjoint action of a group $G$, that is
$A_{\mu} \rightarrow gA_{\mu}g^{-1} - \partial_{\mu}g g^{-1} $
where $g \in G$. As far as I understand, the adjoint action of a group on a Lie Algebra valued object is given as
$Ad(g) T = gTg^{-1}$
where $T \in Lie(G)$. So how do (can) gauge fields transform differently even though the gauge fields themselves are Lie Algebra valued ? I understand we want to keep covariant derivatives gauge covariant but that doesn't make anything clear about this behavior of gauge fields.