Most real-world versions of the experiment that I've seen don't actually use two slits, but rather an optical circuit with separate paths using beam-splitters. In "Induced coherence and indistinguishability in optical interference" by Zou et. al., they discuss an experiment with two coherently pumped optical down-converters. The down-converters generate both signal and idler photons. The path of the signal photons is measured using a pair of photon counters configured to detect the idler photons. Non-classical interference patterns are observed that depend on the configuration of a beam stop placed in the idler path. As they say in the paper, this is strange from a classical point of view and only makes sense in the context of quantum effects occurring.
(This kind of setup is a Mach-Zehnder interferometer and has a long history).
As to why people were so 'confident' that a which-way detector destroys the interference pattern before the experiment was conducted in the literal sense, it's because this is a straightforward prediction of quantum mechanics, and quantum mechanics itself has a lot of experimental verification. As far as I know, the actual experiment with a which-way detector wasn't carried out even in Feynman's time and remained purely a thought experiment. He says so himself (The Feynman Lectures on Physics, 1965, vol. 3, 1.4):
We should say right away that you should not try to set up this experiment (as you could have done with the two we have already described). This experiment has never been done in just this way. The trouble is that the apparatus would have to be made on an impossibly small scale to show the effects we are interested in. We are doing a “thought experiment,” which we have chosen because it is easy to think about. We know the results that would be obtained because there are many experiments that have been done, in which the scale and the proportions have been chosen to show the effects we shall describe.
He doesn't make specific the 'many experiments' he's referring to, however it's not unlikely that he's referring to Compton scattering. The core of the argument that interference disappears when you use a which-way detector is based on wavefunction collapse; the earliest experiment that I'm aware of that demonstrates wavefunction collapse (of a single photon) is Compton's 1925 experiment. In the years 1925-1927 there were a lot of further experiments which culminated in the 1927 Solvay conference and subsequent debates on collapse and various interpretations. A lot of the details of how this understanding evolved have been lost in the re-telling.
More modern perspectives on this experiment have been given, with some more history and discussion here.
