Pure mechanical inaccuracy is due to elasticity and deformation of the materials used to construct the device, and backlash in the joined moving members.
Assuming a stepper is used without position sensing and correction, there must be some backlash and play between moving components, or the friction would be so high it could not move without severe abrasion of contacting surfaces.
Bearings can be slightly compressible due to spring action of the ball/roller cage, and the rolling elements moving slightly as the bearing is compressed under load. Bearings also need a small amount of backlash or they would be so tight the rolling elements would not move without very high driving force. Sleeve bearings directly ride a hydrodynamic wedge of oil which is itself mobile, and introduces play and elasticity.
And then there is the elasticity of the design materials themselves, which may deform slightly under compression or tension, storing potential energy that is later released.
All of this will also affect actively driven infinite positioning servos, as all of these effects can affect the position sensing accuracy of whatever sensor is providing feedback to the actively driven infinite positioner motor.
Ideally the position sensor for an infinite servo would a separate component that can be mounted as close to the final sensed load and its first mounting pivot, rather than built directly into the servo unit.
There may be a long chain of compressible, flexing components between a combined servo motor/position sensor and the final output lever or wheel that is being moved by the servo, which will introduce positioning inaccuracy under high load, no matter how precise a combined servo motor/sensor unit is.