Yes, a simple passive low pass filter will likely help. The problem is it will be somewhat dependent on the source impedance. The filter can be made more complicated to deal with this in various ways, but here is a simple design that is likely good enough:

R2 and C1 form a simple low pass filter. The -3dB rolloff frequency of this filter is
f = 1 / 2 * Pi * R * C = 34 kHz
However, that assumes the input voltage has zero impedance. Any impedance of the input source will be effectively added to the R in the forumula above and lower the filter rolloff frequency. If you know the guitar pickup impedance, then you can adjust R2 or C1 accordingly. You want to make the rolloff at least 20 kHz, but still low enough to usefully attenuate the unwanted high frequencies.
Many audio pickups and the like are designed for 600 Ω load. R1 is there to present a reasonable load to the input signal and also to make its impedance somewhat predictable to the R2/C1 filter. You have to check what load your pickups really want. As I said, 600 Ω wouldn't be surprising but it could easily be rather different too. The values in the circuit above are rough examples if indeed the source is around 600 Ω and the input impedance of the amplifier is "high". I'm trying to show you how this filter works so you can adjust it to whatever your conditions really are.
For the circuit above, the signal impedance as seen by the R2/C1 filter can only be from zero to 620 Ω. That gives a -3dB rolloff range of 21-34 kHz, which should be good enough. Note that if the actual source impedance is really much higher than 620 Ω, then R1 will attenuate the signal a lot and you won't get much out. Again, you need to adjust the values to your actual case.