My assumption is that stars have significantly larger volume than black holes. And if they spiraled they would would merge before they had the chance to reach high enough velocities to generate gravitational waves detectable by the current LIGO.
Is this the main reason why we know LIGO detected black holes?
Another idea I had was mass. But if black holes can have masses like 30 solar mass then the stars by which they formed would be at least so massive if not more. Right?
Have a look at Is it possible to produce gravitational waves artificially? for some info on how the power radiated from merging masses is calculated.
The power is proportional to the quadrupole moment squared and the angular velocity to the sixth power, and the angular velocity is the most important factor because it's raised to such a high power. In the detections so far the period of the orbit was in the tens of milliseconds. Tens of milliseconds! There is simply no way to get anything bigger than black holes to orbit at such a high frequency. Stars, even neutron stars, would merge long before the orbital frequency got that high.
An other important point is the distance of the two objects before the coalescence. From the form of the signal you can calculate that the final merging distance was really small (only a few Schwarzschild radii of the involved masses). Star systems and even BH-star systems would not be able to come so close without merging, only BHs can do that.
See LIGO explenation.
When you say that stars have significantly larger volume that the black holes you must be careful. There are black holes which have volumes bigger than most of the stars (e.g. supermassive black holes, like the one which "lives" in the center of our galaxy).
The key thing of generating gravitational waves is not the volume, but the mass. So, as you already assumed, the answer is indeed the reason of huge masses involved in the merging process.
Also, the stars which created a bh with, say 30 S.M. (solar masses) don't need to have also 30 S.M. The star can have 15, then when it collapses and creates the bh, the bh could start absorbing matter from the cosmic neighborhood, so it will generally become bigger and heavier.
I hope this helps :)
