In "QFT for the gifted amateur" page 37, the author defines the field operators:
$$ \hat{\psi}^\dagger(x)=\frac1 {\sqrt V} \sum_p{\hat a ^\dagger _p e^{-ipx}} \tag{4.7}$$ $$ \hat{\psi}(x)=\frac1 {\sqrt V} \sum_p{\hat a _p e^{ipx}}\tag{4.8}$$
Why does the creation operator has a minus sign and the annihilation has a plus sign? Is it merely a convention?
Quantum Mechanic's answer is a nice heuristic, but I think there's a better answer. The mode expansion you listed is an example of a change of basis in second quantization. Specifically, suppose I have a basis of states $\{ | \alpha \rangle \}$ and another set of basis states $\{ | \nu \rangle \}$. In first quantization, I would express the former basis in terms of the latter via a resolution of the identity: $$ | \alpha \rangle = \left( \sum_{\nu} | \nu \rangle \langle \nu | \right) | \alpha \rangle = \sum_{\nu} \langle \nu | \alpha \rangle | \nu \rangle $$ In second quantization, a similar formula holds. Suppose the operators $a_{\alpha}$ and $a^{\dagger}_{\alpha}$ respectively annihilate and create operators in the basis states $| \alpha \rangle$, while $b_{\nu}$ and $b^{\dagger}_{\nu}$ do the same for the basis states $| \nu \rangle$. Then it must be possible to express $a_{\alpha}$ and $b_{\nu}$ in terms of each other. An easy way to derive the relation is as follows: $$ a^{\dagger}_{\alpha} | 0 \rangle = | \alpha \rangle = \sum_{\nu} \langle \nu | \alpha \rangle | \nu \rangle = \sum_{\nu} \langle \nu | \alpha \rangle b^{\dagger}_{\nu} | 0 \rangle $$ We therefore must have $$ a^{\dagger}_{\alpha} = \sum_{\nu} \langle \nu | \alpha \rangle b^{\dagger}_{\nu}, \quad a_{\alpha} = \sum_{\nu} \langle \alpha | \nu \rangle b_{\nu} $$ where the latter equation follows from Hermitian conjugating the former. Note that this is not technically a complete proof, since I've only shown that these operators have the same action on the vacuum state $| 0 \rangle$. But with a little more work, you can show that they have the same action on any occupation number state. It is sometimes said that creation operators transform like kets, while annihilation operators transform like bras.
Now, go to your example. In first quantization, we are (presumably) using periodic boundary conditions to obtain normalized momentum eigenstates of the form $$ \langle \vec{x} | \vec{p} \rangle = \frac{1}{\sqrt{V}} e^{i \vec{p} \cdot \vec{x}} $$ Then, the change of basis formula for annihilation operators reads $$ \hat{\psi}(\vec{x}) = \sum_{\vec{p}} \langle \vec{x} | \vec{p} \rangle a_{\vec{p}} = \frac{1}{\sqrt{V}} \sum_{\vec{p}} e^{i \vec{p} \cdot \vec{x}} a_{\vec{p}} $$ One easy way to remember this: if you invert the above expression to obtain $a^{\dagger}_{\alpha}$ for any state $| \alpha \rangle$ in terms of $\hat{\psi}$, you get $$ a^{\dagger}_{\alpha} = \int d^d \vec{x} \, \phi_{\alpha}(\vec{x}) \hat{\psi}^{\dagger}(\vec{x}) $$ where $\phi_{\alpha}(\vec{x}) = \langle \vec{x} | \alpha \rangle$ is the position-space wavefunction for $| \alpha \rangle$. Compare this with the familiar formula from first quantization: $$ | \alpha\rangle = \int d^d \vec{x} \, \phi_{\alpha}(\vec{x}) | \vec{x} \rangle $$ Indeed, acting both sides of the former equation on the vacuum, you obtain the latter equation.
A travelling wave moving with velocity $v$ takes some functional form $f(x-vt)$. If $v$ is positive the wave is moving to the right and if $v$ is negative the wave is moving to the left. Since the time evolution for a free field has $da/dt=-i\omega a$, your operators will evolve as $$\psi(x,t)\propto\sum_p a_p e^{ipx-i\omega t}=\sum_p a_p e^{ip(x-\omega t/p)},$$ which nicely matches the description of a travelling wave. The Hermitian adjoint likewise matches. Some authors refer to $\psi$ and $\psi^\dagger$ as the positive and negative frequency parts of the field (e.g. in quantum optics).
Here's a simple explanation. We can understand the destruction field operator $\hat{\psi}(x)=\frac1 {\sqrt V} \sum_p{\hat a _p e^{ipx}}\tag{1}$ as the expansion in the basis {$e^{ipx}$}, now changing the sign simply means we expand it in another basis {$e^{-ipx}$}. Since the summation is over all negative and positive values of $p$, the two basis are indeed identical. The only difference is we just need to redefine the "coefficients" $\hat{a}_p$. That is in the {$e^{-ipx}$} basis we have $\hat{\psi}(x)=\frac1 {\sqrt V} \sum_p{\hat a' _p e^{-ipx}}\tag{2}$ and we must have $\hat{a'}_p=\hat{a}_{-p}$. To check it we can substitute it back to (2) and get $\hat{\psi}(x)=\frac1 {\sqrt V} \sum_p{\hat a _{-p} e^{-ipx}}\tag{3}=\frac1 {\sqrt V} \sum_{-p}{\hat a _{-p} e^{-ipx}}=\frac1 {\sqrt V} \sum_p{\hat a _p e^{ipx}}$. So you can expand the destruction operator with a minus sign in the exponential and just need to redefine the $\hat{a}_p$.
