Big BIG kudos for taking the time to completely check him out.
By this time his habits of pulling back, because he's uncomfortable being touched, will have to be un-learned.
My thoughts are two-fold:
He is either really thin-skinned and just can't tolerate much pressure when being brushed, and/or he has food allergies that are so very subtle that you don't notice them ---- except when you brush him
I said for years that one of my TWH's was "uncomfortable in his own skin". He only tolerated being brushed because he got disciplined if he didn't but, I was very careful to be respectful of his feelings as well.
Then five years ago, my senior TWH was diagnosed with metabolic issues. It was easier to change everyone's diet, along with the metabolic guy.
That was when I discovered the OTHER TWH was oat/corn/soy intolerant. It didn't take ten days for him to become a completely different horse. The horse I knew was locked inside somewhere because, occasionally, I would see glimmers of a sweeter personality.
This horse has gone from not tolerating brushing (or a bath) to enjoying a shower on a hot summer night and letting me rake the Furminator down his sides and back with the same pressure I use on the other horses.
If you tell me all this horse gets is pasture and hay, then I would have him tested for mineral deficiencies as I still think it might be diet related.
The learned habit of pulling away with my food allergy horse, actually corrected itself. I started out with very soft brushes and worked my way up to the de-shedder blade over time.
Regarding getting his head brushed - for now - go to somebody's garden center and buy those cotton gloves with the knubbies on the palms. My horses LOVE to be massaged with those.
Even the food allergy guy now tolerates having the insides of his ears lightly massaged. He argued for years over the left ear but I attributed that to some idiot grabbing and pulling his left ear to get him to do something before I bought him.
Hope this helps a little:)