Let’s consider a pull-up resistor. The job of a pull-up resistor is to pull a particular pin to the HIGH state. However, the pin won't always be in the HIGH state because some circuitry can pull it down to ground. Consider I²C lines. They are pulled up via pull-up resistors and the microcontroller pulls them down as and when needed. Had these lines been permanently pulled up AKA "hard pulled up", I²C communication wouldn't have happened. The SDA line will see a permanent HIGH state.
Your scenario
In your case, if there's a resistor between the pin and GND/Vcc, don't remove it. If the datasheet says to put a resistor, do it. However if you want to delve deeper and understand the functionality of the pull-up /pull-down, look for the IC block diagram in the datasheet. Sometimes you may even find a circuit diagram of the internal blocks. Try to understand the function of the particular resistor (if you find it difficult to understand the circuit, you can post the schematic here). Like Huisman said, if a pin is permanently pulled up or pulled down, the pin's state isn't called pulled up or pulled down. Instead it is HIGH or LOW.
Some info about pull-up /pull down
Pull-up (and pull-down) resistors are generally high, about 10 kΩ generally and they keep the pin in a defined state - HIGH (or LOW) . When an external circuit pulls down a pulled up pin, it provides a path of lower resistance to the ground for that pin. Hence, the pull-up resistor value depends on the resistance offered by the external circuitry to the pin, to GND. The pull-up value must be significantly greater than the resistance of pull-down path. Some circuit designs with ASICs may use even a higher value of pull-up or pull-down.
So to summarise the answer, a pull-up or pull-down resistor is used when the state of the pin has to be changed via some circuitry. If the pin state won't be changed anytime in future, you can hardwire it to either VCC or GND.