Functionally, $\mathbf{E}$ and $\mathbf{B}$ are analagous because they both represent the total field, taking into account both free and bound sources.
However, mathematically, we have $\mathbf{E}\sim\mathbf{H}$ and $\mathbf{D}\sim\mathbf{B}$ because when there is no free charge or current, $\nabla\times\mathbf{E} = \nabla\times\mathbf{H} = \mathbf{0}$ while $\nabla\cdot\mathbf{D} = \nabla\cdot\mathbf{B} = 0$. Therefore, both $\mathbf{E}$ and $\mathbf{H}$ admit scalar potentials. This is rooted in the fact that electrostatic fields have zero curl while magnetostatic fields have zero divergence. Sources contribute to the divergence of $\mathbf{E}$ but the curl of $\mathbf{B}$. In addition, from an experimental point of view, $\mathbf{E}$ and $\mathbf{H}$ are the more useful quantities. From Griffiths' Introduction to Electrodynamics, section 6.3:
$\mathbf{H}$ plays a role in magnetostatics analogous to $\mathbf{D}$ in electrostatics: Just as $\mathbf{D}$ allowed us to write Gauss's law in terms of the free charge alone, $\mathbf{H}$ permits us to express Ampere's law in terms of the free current alone-and free current is what we control directly. Bound current, like bound charge, comes along for the ride - the material gets magnetized, and this results in bound currents; we cannot turn them on or off independently, as we can free currents.
As it turns out, $\mathbf{H}$ is a more useful quantity than $\mathbf{D}$. In the laboratory, you will frequently hear people talking about $\mathbf{H}$ (more often even than $\mathbf{B}$), but you will never hear anyone speak of $\mathbf{D}$ (only $\mathbf{E}$). The reason is this: To build an electromagnet you run a certain (free) current through a coil. The current is the thing you read on the dial, and this determines $\mathbf{H}$ (or at any rate, the line integral
of $\mathbf{H}$); $\mathbf{B}$ depends on the specific materials you used and even, if iron is present, on the history of your magnet. On the other hand, if you want to set up an electric field, you do not plaster a known free charge on the plates of a parallel plate capacitor; rather, you connect them to a battery of known voltage. It's the potential difference you read on your dial, and that determines $\mathbf{E}$ (or rather, the line integral of $\mathbf{E}$); $\mathbf{D}$ depends on the details of the dielectric you're using. If it were easy to measure charge, and hard to measure potential, then you'd find experimentalists talking about $\mathbf{D}$ instead of $\mathbf{E}$. So the relative familiarity of $\mathbf{H}$, as contrasted with $\mathbf{D}$, derives from purely practical considerations; theoretically, they're on an equal footing. Many authors call $\mathbf{H}$, not $\mathbf{B}$, the "magnetic field". Then they have to invent a new word for $\mathbf{B}$: the "flux density," or magnetic "induction" (an absurd choice, since that term already has at least two other meanings in electrodynamics). Anyway, $\mathbf{B}$ is indisputably the fundamental quantity, so I shall continue to call it the "magnetic field," as everyone does in the spoken language. $\mathbf{H}$ has no sensible name: just call it "H".
The other difference between the two expressions is due to the convention of the Coulomb and Biot-Savart laws:
$$\mathbf{E}(\mathbf{r}) = \frac{1}{4\pi\varepsilon_0}\int\ \rho(\mathbf{r}') \frac{\mathbf{r}-\mathbf{r}'}{\left|\mathbf{r}-\mathbf{r}'\right|^3}\mathrm{d}^3\mathbf{r}' \\ \mathbf{B}(\mathbf{r}) = \frac{\mu_0}{4\pi}\int\frac{\mathbf{J}(\mathbf{r}')\times\left(\mathbf{r}-\mathbf{r}'\right)}{\left|\mathbf{r}-\mathbf{r}'\right|^3}\mathrm{d}^3\mathbf{r}'$$
Notice how the constants are on opposite sides of the fraction. The magnetic field can be rewritten as
$$\mathbf{H} = \frac{1}{\mu_0}\mathbf{B} - \mathbf{M}$$
which will be completely analogous to the electric case except for the sign. The convention was chosen such that the greater the permittivity, the smaller the electric field (because more polarization is "permitted" which opposes the electric field, reducing it) while the greater the permeability, the greater the magnetic field. So the permeability was moved to the other side to match. As for the difference in sign, I suspect that it is because a polarization $\mathbf{P}$ creates an $\mathbf{E}$ that points opposite to it while, on the other hand, a magnetization $\mathbf{M}$ creates a $\mathbf{B}$ that points in the same direction. This is again a matter of convention.
More discussion can be found here.
