First, let me try to answer your second question.
And my second doubt is, I do not understand or unable to imagine, how symmetry cancels the contribution of end caps of the Gaussian cylinder and electric field is only due to the curved part of the cylinder. If someone can help me show diagrammatically would be easy to understand it.
By cylindrical symmetry, we have:
$$
\mathbf{E}(\mathbf{r}) = E(\rho) \, \hat{\mathbf{\rho}}
$$
Now, the Gauss law states that
$$
\oint_S \mathbf{E} \cdot d\mathbf{a} = \frac{Q_{enc}}{\varepsilon_0}
$$
where $S$ is the gaussian surface of our choice. If we choose a cylinder of radius $\rho$ and length $L$, then we have (writing $d\mathbf{a}$ in cylindrical coordinates)
$$
\oint_S \mathbf{E} \cdot d\mathbf{a} = \int_0^{2\pi} \int_0^L (E(\rho) \, \hat{\mathbf{\rho}}) \cdot (\rho \, d\phi \, dz \, \hat{\mathbf{\rho}}) \; + \; 2 \int_0^r \int_0^{2\pi} (E(\rho) \, \hat{\mathbf{\rho}}) \cdot (\rho \, d\rho \, d\phi \, \hat{\mathbf{z}})
$$
$\hat{\mathbf{\rho}}$ and $\hat{\mathbf{z}}$ make up an orthonormal basis, thus $\hat{\mathbf{\rho}} \cdot \hat{\mathbf{z}} = 0$, and we have
$$
\int_0^{2\pi} \int_0^L (E(\rho) \, \hat{\mathbf{\rho}}) \cdot (\rho \, d\phi \, dz \, \hat{\mathbf{\rho}}) \; + \; 2 \int_0^r \int_0^{2\pi} (E(\rho) \, \hat{\mathbf{\rho}}) \cdot (\rho \, d\rho \, d\phi \, \hat{\mathbf{z}}) = \int_0^{2\pi} \int_0^L (E(\rho) \, \hat{\mathbf{\rho}}) \cdot (\rho \, d\phi \, dz \, \hat{\mathbf{\rho}}) = E(\rho) \rho \int_0^{2\pi} \int_0^L d\phi \, dz = 2 \pi \rho L \, E(\rho)
$$
The last equality holds because we do not integrate over $\rho$ and $E$ only depends on $\rho$.
In terms of visual intuition, you can think of it this way: the infinite cylinder is a "sum" of infinite number of point charges positioned in a line. The line is infinitely long, thus each field component in $z$ direction is cancelled by "neighbouring" point charges. As a result, the field depends only on the distance from the axis, which is just another way of saying that the field is cylindrically symmetric.
My doubt is, why we are finding electric field at a distance z from the wire and not along the axis of a the wire.
As you finish the calculation, you'll notice that the field is proportional to $\frac{1}{\rho}$. Thus, it is not sensible to ask about the field on the axis, because as $\rho \rightarrow 0$, $E \rightarrow \infty$.
