If you 'carry' an object 10 meters in one direction then return it back 10 meters from where you started the work done on the object is not the force you expended times distance walked.
The formula you write is often misunderstood and misused. In your example, when you lift the object in a gravitational field, the work being done on the object is its weight (force) times the vertical distance it's lifted. When you walk ten feet in one direction, then back there is zero work done on the object assuming you held the object at the same vertical position. Once you return to your point of origin and lower the object the object is now doing work on your muscles.
The process of carrying the object horizontally is much more complicated when trying to figure out how much work is being expended. The expended work is what's being spent in maintaining the isometric tension in your muscles. It's not force times distance.
But returning to your original question I believe you are having difficulty differentiating the terms distance and displacement. Distance is an absolute measure between two points in space. Displacement is a relative measure of distance which can turn out to be zero if you start and end at the same place.
When you raise a weight 10 ft high and lower it to it's original height the net work done ON the object is zero since the displacement is zero. That's because when you return the object to the original position its gained potential energy (that was stored 'in' the object) is being spent. The object is now doing work on whatever opposing force is doing the lowering.
The formula can be used when the restoring force is other than a gravitational field. A spring for example allows you to store then release energy when it's compressed and expanded respectively. Work is done on the spring, and the spring subsequently does work on the object.