Children think with their minds, not with their hands. So when we assign activities because they are “hands on” it is more appropriate to think of these activities as being “minds on”. But of course, our minds are always on (to some degree) so now we want to specify, what exactly is it that is different about this task and what do I hope children will learn from this activity that they cannot learn without this activity?1 Or alternatively, what does this activity allow that another activity might not?
For example, giving kids algebra tiles doesn’t mean that they will learn something important from using those algebra tiles. They might, but they might also learn the same thing from a structured activity using the area model.
My experience is that when I have vague hope that children will learn something from an activity that is related to the mathematics I want them to learn, they usually don’t.
To be clear, I see nothing wrong with students playing with manipulatives when free play is the goal. Sometimes having vague goals and experimenting with the use of manipulatives as a teacher is a good starting place for thinking about learning, but we should not conflate the ability of manipulatives to support learning with the actual learning students do as a result of using them.