Well, IMO you shouldn't "get into his mouth" for any reason much less whinnying which is mostly harmless. Stay away from that "trainer"!!!
I don't have any particular experience with your problem, but what I would try is a simple reprimand with something he doesn't particularly like doing or that's harder work. For instance, every time he whinnies turn him in one or two tight circles then walk back off like nothing happened. Backing might work as well, though using backing as a punishment has the downfall of them possibly thinking they are being punished when you simply want them to back. It doesn't have to be forceful, painful, or dramatic or anything. He'll likely learn to associate the whinnying with the relatively unpleasant tight circles and decide it's best not to scream on the trail. I'm not a big fan of "bandaid" fixes that do little more than mask or forcibly prevent the problem (the cavesson in your case); better to address and actually fix the issue, IMO.
You should also try to address the underlying issue which is likely some form of separation anxiety or fear (or a combination). Help him build his confidence in being alone, especially out on the trail. Helping him gain confidence in you will also help.