I'm going to raise some of the same points the other answers have, but in slightly more detail.
There are two main types of visible light coming from a pulsar: thermal black body emission from the surface of the neutron star, and optical synchrotron radiation originating in the magnetosphere. The first problem with observing either of these lies in the fact that neither contributes significantly strong emission. The Crab pulsar appeared as a magnitude $+17.7$ V-band source in the initial optical observations (see Cocke et al. 1969), pushing the limits of what could be observed at the time.
The vast majority of neutron stars do not display substantial thermal emission ($\lesssim1\%$ of total optical radiation), meaning that optical synchrotron radiation is a much better thing to look for at visible wavelengths. However, given the dimness of the source, you need to understand the radio/x-ray pulsations of the neutron star. The confirmation of the first optical detection involved a complicated setup to synchronize with the target and time-average the signal (Nather et al. 1969), and this would not have been possible had the Crab not already been detected at radio wavelengths. The technology existed, but one wouldn't casually scan the sky at optical wavelengths with it; that would be fairly pointless unless you had a pulsating target.
To expand on a point made in Thomas's answer, systems for detecting optical pulsars don't naively use CCDs, but instead use a principle called photon counting, which, well, counts photons.