You can buy a substance called Bt (short for bacillus thuringiensis). It is sold at many garden nurseries. It is a natural bacteria that kills caterpillars of all kinds. This should help eliminate them.

You can also use neem oil. It is not a quick kill pesticide, but causes the caterpillars to stop eating and they starve to death. The nice thing about neem oil is the tree absorbs it, so even if the worms can burrowed into the tree, the neem oil will still reach them and kill them.

Both of these substances are harmless to people and pets.