Obviously electrons annihilate with positrons, but can a muon annihilate with an positron, or can an anti-taon cancel with a muon? similarly for quarks of different species, e.g. u and anti-strange.
I think this is possible as long as quantum numbers like charge and spin are conserved, with the excess energy being given off in kinetic energy, but has it ever been observed?
There are more quantum numbers to be conserve than charge and spin. There is also lepton number and baryon number and strangeness and charmness and color for strong interactions.
A muon hitting a positron can interact exchanging a photon and keep going, or a W and turn into a muon-neutrino and an electron-antineutrino through the week interaction .
Quarks are worse because they also have color, three extra quantum numbers plus charge number (1/3 or 2/3) plus baryon number(1/3).
Take the pion: pi+ is a linear combination of up anti-down and will eventually turn into a (muon+ anti-muneutrino) via the weak decay.
You may call it annihilation, as you may call all interactions where the incoming particles and the outgoing differ. The customary use of the term is for particle antiparticle annihilation.
It is more complicated than you think.
A few theoretical samples of mixed annihilation (at low energy):
$e^- \mu^+ \to \gamma \; \nu_e \bar{\nu}_\mu$
$\mu^- \tau^+ \to \pi^0 \nu_\mu \bar{\nu}_\tau$
$\mu^- \pi^+ \to \gamma \; \nu_\mu$
