You've received a lot of incorrect answers so far. Let's start with some definitions. As you walk up or down stairs, let the change in your gravitational potential energy be $U$, and let the amount of food energy you burn be $E$. Define $C_\text{vert}=E/U$, the vertical cost of walking. Conservation of energy requires only that $E\ge U$. So when you go up stairs, with $U>0$, we must have $C_\text{vert}\ge 1$, and when you go down, $C_\text{vert}\le 1$.
Pretty much everything I've said so far is very generic, so it would apply equally well to any machine. This is the problem with the answers by JalfredP, Math_Whiz, and Claudio Saspinski, which attempt to explain what's going on in the human body without using any facts about how the human body differs from other machines. This is doomed to failure. An electric car going downhill has $C_\text{vert}>0$, because it recharges the battery with regenerative braking. A human body going downhill has $C_\text{vert}<0$, because you still burn calories even when walking downhill. There is no generic explanation of these facts that can be based simply on the principles of mechanics, without saying something about the nature of the machine involved. There is nothing in the principles of newtonian mechanics or conservation of energy that prevents me from building a battery-powered anthropomorphic robot that uses regenerative braking when it walks down stairs.
Let's look at some real data on the human body. Minetti (2002) measured this sort of thing for elite mountain runners running on a treadmill inclined at a gradient $i$ ($i$ is the tangent of the angle). Their energy consumption $E$ was determined by measuring the amount of oxygen they consumed. The quantity $C_\text{vert}$, considered as a function of $i$, blows up at $i=0$, where $U=0$. Therefore it's nicer to look at the quantity $C=E/m\ell$, where $m$ is the person's body mass and $\ell$ is the distance traveled. If you're climbing straight up or down a ladder, then $C=gC_\text{vert}$. $C$ is always positive because $\ell$ is positive by definition, and the human body can't have $E<0$ like a Prius.

On steep uphill slopes, the observed value of $C_\text{vert}$, in these elite runners, is very close to what you would get just from the efficiency of muscle fibers. For sedentary people, it's worse by almost a factor of two.
In the limit, does walking slowly downstairs cost me as much energy as walking upstairs?
No, as you can see, the cost is far less.
How does the speed of my descent affect the energy I expend by braking with my muscles? Is the relationship linear?
Approximately, but not exactly. For running, Minetti found that this was true to a good approximation. This is what we would expect if the efficiency was independent of speed. However, there is no fundamental physics principle that dictates that this should be so. It's just an approximate observation about the human body. In general, if you take a frog's leg muscle and stimulate it, both the force it can generate and its efficiency do depend on the speed.
If you want a physics explanation of why the human body has the behavior described above, then this seems to have been partly figured out in recent years, but it's pretty complicated and there are a lot of unknowns. In general, during exercise, a lot of the food energy is being transformed into heat, and some of it is also going into endothermic chemical reactions. Only what's left over is available to do mechanical work. At the microscopic level, the smallest structural units of a muscle are proteins called sarcomeres. These include myosin, actin, and titin. There is something called the sliding filament theory, which was advanced in 1954. When you go downstairs, your muscles are doing negative work, which physiologists refer to as an eccentric contraction. In recent years, people have been working on explaining why the sarcomeres have the energy efficiency they do in eccentric contractions, and there are a couple of hypotheses available, both of which may be true. There is something called the winding filament hypothesis, illustrated below with a figure from Hessel.

I don't know enough about the biophysics to be able to understand all the details. This is just meant to point you in the direction of the general type of explanation that is required here, which is an explanation involving the details of the machine, not general principles about newtonian mechanics. The classical observations, going back about a century, are that muscles in eccentric contractions are capable of generating large forces at low energy cost, where low energy cost means that $|C_\text{vert}|$ is small. If I'm understanding Hessel in a vaguely correct way, then I think the idea is that titin gives the muscle some mechanism that during a concentric (shortening) phase allows the fiber to stiffen, and then during the eccentric phase this stiffness lets the muscle supply a force at a low energy cost.
References
Minetti et al., http://jap.physiology.org/content/93/3/1039.full
Hessel et al., Front Physiol. 2017; 8: 70, https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5299520/ , doi: 10.3389/fphys.2017.00070
Related
Why does holding something up cost energy while no work is being done?