"friction does not have anything to do with the area an object takes up"
To say that it has nothing to do with the area might a bit to harsh a wording - but indeed, friction does not grow with growing contact area. And that is because - as you touch upon shortly in the question - that the whole load (weight) is carried by that area, and if the area is smaller, the pressure is antiproportionally larger. So the effect of the area change cancels out.
When a surface meets another surface, actually the roughness peaks touch the other surface. So the real contact area $A_r$ is only at the peak tops and is very small (much smaller than the apparent area $A$).
Each peak binds to the other surface. So to slide, we must break this bond; this means that we must overcome the material shear strength $k$ (force per area needed to shear it) at each peak.
Friction $F$ is therefore this strength per area times the actual area, so $$F=A_rk$$
We then see that if the load $n$ (or weight) increases, the peaks flatten a bit. The real contact area thus increases.
It has then experimentally been shown that the real contact area $A_r$ increases proportionally with increasing load $n$. If you push double as hard down, the peaks flatten double as much. This gives the expression $$A_r=cn$$ where $c$ is the proportionality constant for this setup (for these materials).
Plugging this in removes $A_r$ from the formula,
$$F=ckn=\mu n$$
and we can collect all the constant parameters in one and call them the coefficient of friction $\mu$.
When the loads become large, the roughness peaks flatten so much that this $A_r$ is not proportional to $n$ any more. This is when the deformation zones of the peaks begin to overlap and prevent each other from further deformation. See more in this answer. And then the contact area starts to have an influence.
