In this context, the origins of the term "Pony Car" comes from a writers pen in the mid sixties, in talking about the new Ford Mustang. The name stuck.
The main competitor to the mustang was the chevy Camaro, and it too was referred to as a Pony car. With their prevalence in Trans Am, this led to the early Trans Am era being referred to as "The Pony Car Wars".
Cars considered to be "Pony Cars" would likely include the Mustang, Camaro, AMC Javelin, Ford Cougar, Pontiac Firebird, Plymouth barracuda, Dodge Challenger. Racing in Trans Am circa 68-72 probably grants one the permanent title of being a Pony car.
In general, these were cars considered small or midsize in the american line up, but they came with performance packages and big V8's.
To call any of these cars "muscle cars" would also likely be correct, unless you are talking about Aunt Matilda's six cylinder powered version of any of the above. A Pony car is a muscle car, but a muscle car is not necessarily a Pony car.
The term Muscle car would apply to Chevelles, Impalas, Buick Gran Sport, GTO, 442, Torinos, Road Runners and the like, typically larger cars that were available with big power but preferred to go in a straight line. These larger cars while Muscle cars were not referred to as Pony cars.
There were other small cars available with big power from the factory that never got the monicker of "Pony Car". Probably because they didnt go head to head in high profile racing like Trans Am. The Chevy Nova (chevy too), Ford maverick, Ford Falcon and the AMC Gremlin,which was a hornet with half the back end cut off.

. All of these small cars could be had with V8 power from the factory.
Thats the best that I can explain it, there are no hard and fast rules for any of the above. Which might make your racing series a bit exciting even before anyone turns a lap. If you have a car that you want to enter as a Pony car, send me ten bucks and I will update this message to include mention of your car as a Pony car.
cheers,
j