The question is delicate from a mathematicaly rigorous point of view. What Physicists do is the following:
States are described by elements of a Hilbert space called state space $\mathcal{E}$. One denotes the elements of $\mathcal{E}$ as $|\varphi\rangle$. On the space $\mathcal{E}$ there acts hermitian operators which represent physical quantities associated to the system, they are called observables. Given the particular system, we assume $\mathcal{E}$ has the necessary observables acting on it.
The spectrum of one observable comprises the set of all possible valued to be measured. Now here comes the things: one assumes that every observable has a basis of eigenstates. When the observable is bounded the basis really exists as a discrete orthonormal basis $|\varphi_i\rangle$, which allows the decomposition of any state as
$$|\varphi\rangle=\sum_{i=1}^\infty a_i |\varphi_i\rangle.$$
On the other hand, even when the obserable is unbounded, physicists assume a generalized continuous basis exists, i.e., that there is a continuous set of states $|x\rangle$ such that one can decompose
$$|\varphi\rangle=\int\varphi(x)|x\rangle dx.$$
One also assumes here the completeness relation holds in this generalized sense
$$\int|x\rangle\langle x| dx = \mathbf{1}.$$
In the case of a particle which has position, for example, one assumes there is $\mathcal{E}$ with the position operator $X$ and the corresponding position basis $|x\rangle$ such that $X|x\rangle = x|x\rangle$ and such that the above decompositions can be done. Now the momentum operator $P$ should act as the generator of spatial translations and hence it should satisfy the equation you post.
It turns out that it is known that the generalized eigenvectors do not exist. The generalized eigenvectors exists only when you go into the so-called Gel'fand tripled (or Rigged Hilbert Space) formalism.
Nonetheless physicists assumes this can be done. So for your two points: (1) the basis $|x\rangle$ exists by assumption, while $\langle x|$ is the dual of $|x\rangle$, in the sense that $\langle x | (|\varphi\rangle) = (|x\rangle,\varphi\rangle)$. The wave function is the projection of the state on the position basis, so that $\varphi(x)=\langle x|\varphi\rangle$. (2) The identity is true, because one assumes there is a momentum operator satisfying it.
This is the traditional approach. But let's elaborate on some things. First of all, notice from (1) I wrote, that one assumes there is a Hilbert space, but the important thing is to assume there are the operators. On the algebraic approach, one can define the algebra of operators and looks for representations of the algebra on Hilbert spaces, i.e., how to realize the algebra as operators in a Hilbert space.
It turns out that for a particle with position, you still want momentum to be the generator of spatial translations. Fortunately this can be expressed as a commutation relation: this is equivalent to have operators $X,P$ with $[X,P]=i\hbar$ - this is the Canonical Commutation Relation (CCR).
Thus you look for an algebra of operators generated by $X,P$ with the relation $[X,P]$ imposed. It turns out that in the case of Quantum Mechanics (this fails just in Quantum Field Theory) the Stone-Von Neumann theorem states that up to unitary equivalence there is a single representation satisfying the CCR. They are all isomorphic to the $L^2$ representation where $X\varphi(x) = x\varphi(x)$.
So indeed you can assume there exists the Hilbert space with the operators. This justifies the abstract state space approach and justifies the existence of $P$ that acts as $-i\hbar \nabla$. Just the basis $|x\rangle$ that is non-rigorous and can only be justified in the Gel'fand triplet approach.