Usually these terms are used interchangeably. But It seems Thermal energy is the average kinetic energy of all of the particles in the system, while Heat is the transfer of energy. So we can't say that a system have "Heat content". Correct?
What is the difference between the two? And how we define them exactly?
The terms are used interchangeably only outside of a scientific context, for example, in your kitchen, in the popular press or poor blogs, and even a few bad textbooks.
In a scientific context, you have it almost correct. Heat is the energy that enters or leaves a system on account of a difference in temperature (no work done).
Thermal energy is a component of the internal energy of the system. It is associated with properties that have a quadratic dependency on some parameter. It includes translational kinetic energy ($\frac{1}{2}mv^2$) as you point out, but it also includes rotational energy ($\frac{1}{2} I\omega^2$), and harmonic vibrational potential energy ($\frac{1}{2}kx^2$). Not included are things that do not have a quadratic dependence on energy. The most familiar perhaps is chemical binding energy (including the intermolecular binding energies in liquids and solids) but there can be others. The total of the thermal energy and the other energies is the internal energy.
The ideal gas particle has no internal structure, so it has no rotational energy, no vibration energy, and no chemical energy. So for that special case the thermal energy is equal to the internal energy, equal to the total kinetic energy of all of the particles in the gas.
There's some confusion about all this due in part to the fact that the basic concepts are introduced with respect to the ideal gas, but the distinction that occurs in applying the concepts to a real gas is often not made clear. A further complication is that it is a challenge to introduce the equipartition principle at a pedagogically early stage.
You already have all the ingredients for an answer. Internal Energy ($U$) is a property of the system (e.g. the gas has a definite energy) while heat ($Q$) is transfer of energy (e.g. if i put two systems with different temperature together they will exchange some energy and we call that heat).
Everything is linked together via the first principle of TD $U=Q-W$ where $W$ is the work applied to the system or done by the system (the signs are such that we are using the "point of view" of the system, so that $Q>0$ if the system is receiving heat, and $W>0$ if the system is doing work). Thus if I want to decrease (increase) a system's energy I can:
a-have it do some work
b-extract some energy from it, for example by placing a colder system near it: the system will transfer some of its energy to the other system and that is heat.
Now, this is very general.
As you said particles in the system move and thus part of the internal energy is given by the particles' kinetic energy. Thermodynamics tells us that the average kinetic energy depends on the temperature and is therefore dubbed "Thermal Energy". Thermal Energy is of course energy, it is a quantity associated to a system at a given temperature. In the example above in which we extracted energy from a system putting it near a colder one, what happens eventually is that our system will be now colder thus it will have less kinetic energy and thus less internal energy. The missing energy has been transferred to the other system as heat.
As you said, a system does not have a "heat content", it only has "energy content" BUT sometimes terms such as heat content or heat reservoir are used to describe the amount of heat one can extract from a system, i.e. how much energy can I transfer from the system to somewhere else.
Temperature is the average kinetic energy Thermal Energy is the total kinetic energy Heat is the transfer of thermal energy from an object of higher temperature to an object of lower temperature
I totally agree with Jalfred P's correct explanation of heat being defined through the 1st Law of Thermodynamics as partitioning the internal energy $U$ into the heat part $Q$ and the work part $W$ $$ \Delta U=Q+W, $$ as well as seeing thermal energy as the average kinetic energy of a system of particles.
I want to elaborate a bit more though on the interpretational difference to temperature energy.
The definition of heat, is linked to the definition of work. This in turn is depending on a specific task. Heat is simply the residual energy, that is not used after performing such a task. Typically one imagines this residual to be changed into temperature due to friction or maybe conduction, leading to a change in temperature and thus thermal energy.I believe this is where the source of confusion stems from.
This is not necesarily the case, for example "heat" may also perform some secondary task such as undesirably deforming a mechanical body in a collision. Deformation may absorb energy without change in thmperature. Heat should rather be envisioned as energy undesirably transfered into "the environment" (i.e. where you dont want it!). What it does there is entirely open, it may change thermal energy but it does not have to.
Furthermore heat is usually considered something uncontrolable, that may not be used for performing anything sensible. In contrast thermal energy is probably among the most widespread sources of energy and thus work. If two baths have a different temperature, the flow of energy between them provides usable energy! This further illustrates that heat and thermal energy are not the same!
