TL;DR: POVM $M$ is not an operator. It's more like a probability distribution over the set of all possible measurement outcomes, but parametrized by quantum states.
Positive operator-valued measures
A Positive Operator-Valued Measure (POVM) $M$ is not an operator. It is typically defined as a collection of positive operators $M_i$ labeled by measurement outcomes $i\in A$ where $A$ is the set of all possible measurement results with the extra requirement that $\sum_{i\in A}M_i=I$.
We can also think of $M$ as a two-argument function that takes a measurement outcome $i$ and a quantum state $\rho$ and yields the probability of measuring $i$ on $\rho$
$$
M: A\times D(\mathcal{H})\to[0,1]\tag1
$$
where $D(\mathcal{H})$ denotes the set of density operators on some Hilbert space $\mathcal{H}$. We can write down an explicit formula for the function $M$
$$
M(i,\rho)=\mathrm{tr}(M_i\rho)\tag2
$$
which is just another statement of the Born rule. In certain ways, the function $M$ is similar to a good old probability distribution over $A$ with the caveat that the distribution is parametrized by quantum states $\rho$.
Probability distributions
A probability distribution on $A$ is formalized as a probability measure defined as a function on a certain $\sigma$-algebra of subsets of $A$ called events. However, in the countable case, the fact that the definition requires a measure to be $\sigma$-additive implies that it is completely defined by its values on a partitioning of $A$. In particular, it is completely defined by specifying its value on singleton sets. For this reason, we can often forget about the $\sigma$-algebra and think of a probability distribution on $A$ as a function $p:A\to[0,1]$ that assigns a probability $p(i)$ to each $i\in A$.
Now, the POVM $M$ is not quite as direct. Instead of assigning to $i\in A$ a probability $p(i)\in[0,1]$, it assigns to $i\in A$ a positive operator $M_i$ (which explains the name Positive Operator-Valued Measure). To obtain an actual probability (of $i$ being the outcome of a measurement), a POVM needs one more piece of information: the quantum state $\rho$. This accounts for the fact that in quantum mechanics measurement outcome probability depends on both: the outcome $i$ and the quantum state $\rho$.
Further similarities between POVM $M$ and probability distribution $p$ can be seen in the constraints they are required to satisfy. The fact that $p(i)\ge 0$ for any $i\in A$ corresponds to the fact that $M_i$ is a positive operator. Similarly, the requirement that $\sum_{i\in A}p(i)=1$ corresponds to $\sum_{i\in A}M_i=I$.
How to make sense of $\langle\psi|M|\psi\rangle$?
The discussion above suggests that we could make sense of an expression such as $\langle\psi|M|\psi\rangle$ by currying, i.e. by interpreting $\langle\psi|M|\psi\rangle$ as a function obtained from $M$ by fixing the quantum state. The result would be a single-argument function mapping a measurement outcome $i\in A$ to its probability when the measurement is performed on state $|\psi\rangle$
$$
\langle\psi|M|\psi\rangle: i\mapsto\langle\psi|M_i|\psi\rangle\in[0,1].\tag3
$$
Since $M$ is a positive operator-valued measure on $A$, the function $\langle\psi|M|\psi\rangle$ would be a probability measure on $A$ for every $|\psi\rangle$. That said, I have never seen anyone using an expression such as $\langle\psi|M|\psi\rangle$.
