My comments have confused many things (apologies!). Let me clear everything up in this answer.
The Maxwell action is
$$
S_M[g,A] = - \frac{1}{4} \int d^d x \sqrt{-g} g^{\mu\alpha} g^{\nu\beta} F_{\mu\nu} F_{\alpha\beta} \tag{1}
$$
Suppose that under a Weyl transformation $g \to g'=\Omega^2 g$ and $A \to A'$ (which takes $F \to F'$). Then,
$$
S_M[g',A'] = - \frac{1}{4} \int d^d x \sqrt{-g} g^{\mu\alpha} g^{\nu\beta} \Omega^{d-4} F'_{\mu\nu} F'_{\alpha\beta}
$$
Thus in order for the action to be Weyl invariant, we need $F' = \Omega^{\frac{4-d}{2}} F$. What is the transformation of the gauge field that will lead to this transformation of $F$? A naive guess is
$$
A \to A' = \Omega^{\frac{4-d}{2}} A \tag{2}
$$
However, under this, we find
$$
F \to F' = \Omega^{\frac{4-d}{2}} F + A \wedge d \Omega^{\frac{4-d}{2}}
$$
Then, if the gauge field transforms as (2), then
$$
S_M[g',A'] = S_M[g,A] + {\cal O}(\partial \Omega)
$$
It follows that $S[g,A]$ is not Weyl invariant (unless $d=4$).
To make the action Weyl invariant in $d>4$, we can try to modify the Maxwell action $S_M[g,A] \to S[g,A]$ or the Weyl transformation $A' \to A'_{new}$. The modifications must be such that $(a)$ $S[\eta,A]=S_M[\eta,A]$ and $(b)$ when $\Omega$ is a constant, $A'_{new}=A'$. You can try this as an exercise, and you'll find that there is no way to do this in $d>4$. (E.g. you can try to add a term like $R^{\mu\nu\alpha\beta} F_{\mu\nu} F_{\alpha\beta}$ to the action).
Everything I said above also applies to scalar fields. In that case, the free scalar action $S[g,\phi] = \frac{1}{2} \int d^d x \sqrt{-g} g^{\mu\nu} \partial_\mu \phi \partial_\nu \phi$ is NOT invariant under $g \to \Omega^2 g$ and $\phi \to \phi' = \Omega^{\frac{2-d}{2}} \phi$. To make the theory Weyl invariant, we have to add an extra term to the action of the form $R \phi^2$.
