Not sure how far you have progressed with pony, but wondered if you have done any of your training work with an open bridle? A safe driving pony is not afraid of anything behind him, wears blinkers to prevent SMART equine from reading your body language and "helping" because he knows the next step. Not being able to see you, he must WAIT for directions, and this idea needs to be FIRMLY enforced. Pony gets to mane NO CHOICES, has to wait for a command before moving or turning etc.
We do most all our ground training in open bridle so the animal sees everything going on, gets used to it and ignores it. We may put a blinker bridle on for getting horse used to it, maybe do open one day, blinkers the next day, back to open. Horse is NEVER afraid of things behind, tires dragging, cart, children and stuff going on around him, blinkers are not for that. Blinkers do help focus his attention FORWARD, not allow him to see many of the distractions around him, over react to them. Usually a scared horse will STAY scared, react strongly to things when wearing blinkers. He doesn't go deaf when you put the blinkers on! So much better for everyone to train in an open bridle, let him see stuff, then change over to blinker once the cart is attached.
I would strongly suggest you start dragging with a tire. You will want to add some binder twines to his traces for length, and we have a singletree that gets dragged on the twine parts. Helper may want to hold back a bit on the singletree, so it doesn't jump about. Then you can add the tire with just running a twine or rope thru it and held by helper. Helper can then drop the rope end and free tire if horse jumps or spooks at the weight. Driver then goes back to tire, helper runs rope thru, you try dragging tire again. Having tire able to be instantly freed, will prevent pony being "chased" and badly frightened. May take a couple sessions of helper and releasing tire, reattaching the tire, before pony gets steady and willing to pull steadily. You don't want his dragging item firmly attached to the singletree before he is comfortable and RELIABLE at going forward and doing WHOA as quick as you say the word. NO WHIRLY turns, he has gotten over them because that twirl will just be really bad for pony, tangled in lines and traces. Someone will get hurt then, probably pony, which will not make him like being driven at all!
I would not suggest dragging brush or logs, fence posts. with ends blunt cut, they tend to dig into the dirt, then jump as pull continues. The long ends swing WAY out on turns, make pulling FEEL weird with wiggling and jumping about. Ends can snag on posts, jerking the pulling animal. Logs and post dragging are best done by a TRAINED animal, not a newly started driving animal who might spook or run from the pulled object. Tires can be sized to the animal being driven, small ones for ponies. Round sides prevent much snagging on things, not heavy, just a steady drag behind him. You don't want weight being pulled, a jumpy load, just that friction of resistance so he gets used to the pulling feel.
We use lunge lines for reins when starting, so driver can stand in the center and make the animal go around them in large 60ft circles. We do this in the open riding arena so fence is not controling the diameter. We can work horse at all the gaits, speeds of walk and trot we wwant, without getting exhausted, hanging on his mouth, as ground driving can do. Few folks can actually walk fast enough to keep up with a good moving horse or pony, without interfering with reins. We can see the side of animal, to tell if he is giving his mouth, collecting as asked, actually doing different trots and walks. He can drag stuff a LONG time out on that circle, getting the feel of light pull, gaining fitness, ignoring pull/rub of traces on his side and stifle as tire swings outward at faster gaits. Technical name for this is long lining or long reining. Have to say it is a big time saver in training here, save our body running behind horse!!
A last question here. Does pony tie well, for longer times? A lot of animals seem to miss that step in training, so they have never developed patience about things. Could be a step you would want to review with pony. Always a benefit to having an animal that ties well, for long times of several hours.