You've gotten a lot of really great suggestions so far! My first recommendation would be to work without stirrups on the flat at all three gaits. When you can post (no, it won't be pretty) for a whole minute or so, you'll have tons of the right muscles and balance to help keep you secure during your horse's more green moments.
Once you're comfortable without stirrups, try trot cavaletti both with and without stirrups. If possible, don't just put poles on the ground, as they can get spun around by a hoof, or get stepped on. Something a few inches off the ground would be plenty. Start with just one or two poles until your horse gets the idea that he should just trot through rather than jumping the whole lot of them! Add poles as he gets comfortable; probably not much more than 4 or 5. Don't overdo them, because they're very tiring for the horse.
THEN... Once you're comfortable with all this fun stuff, try going through canter-length cavaletti. The really fun part is putting a crop (slim and bendy if possible) under your seatbones and then go through the canter cavaletti, keeping the crop underneath you the whole time. After all this, you'll be very well attached to your horse, with legs as tight as a tick, and so well balanced that everything will be easier.
Good luck!