Yes, all of quantum mechanics is essentially linear algebra, so you can indeed take any set of basis vectors or basis states that you want. For the purposes of making proofs in quantum information, the choice of the vectors is completely arbitrary, but of course problems are often much easier to solve in one basis than another.
Where the physics appears is at least twofold. First, a generic quantum information protocol requires the initialization of data onto a quantum system and the manipulation of that data via unitary operations. To initialize data, one must always do it with respect to some fixed basis (otherwise, there would be no meaning), while unitary operations enact basis transformations that again must be understood in reference to a fixed basis (otherwise, if you just take a vector and rotate it and ask where it ends up, you'll have no way of describing the result). Some particular encodings and transformations are easy to do on specific physical systems than others, so one often uses the physical system to define the computational basis states. Just like in classical transistors there is an on and an off state, so too for quantum computers there are two native physical states that can be easily distinguished an accessed, like a ground and an excited state of a molecule. And then one prefers a physical system in which the unitary transformations are straightforward to implement, so that also dictates something about what basis one chooses.
The other physical consideration is measurement. This physically allows one to access one and only one component of the state vector within a given basis. Of course, choosing a different basis allows one to access a wholly new parameter, so the accessible measurement basis is important for dictating the properties of a state that can be accessed. Different bases can be connected with unitary transformations, so different measurements can be obtained by preceding some fixed measurement by different unitaries, but the fixed measurement at the end is another way to distinguish an important basis against which everything else can be referred. So one can choose the computational basis as "the basis states that are most easily distinguished to measurements."
In all: any basis can be chosen, computational basis is an arbitrary reference from which everything can be defined. There are certain bases for which states are easier to initialize or measure in the basis states, so one often uses those as the computational basis.