Suppose I have a model M which outputs a three-dimensional tensor of size 3x3x3. I have another model N which outputs a one-dimensional tensor of size 27.
Train both models on some arbitrary objective which requires the knowledge of space. For example:
M: Output 1 in any corner, 1 in the exact opposite corner, and 0 elsewhere. (3x3x3)
N: Output 1 in the indices which would correspond to any corner and its opposite, if you rebuilt the 3x3x3 cube from the 1D (1x27) array. Output 0 elsewhere.
Now, this might not be a perfect objective to evaluate spatial-competency, but you get the idea.
Will model M be as proficient as model N? Why or why not?
