yellow trumpet bush has tiny white worms or caterpillars
We have a yellow trumpet bush. It is plagued by tiny white worms or maybe caterpillars. The leaves end up skeletonized and stuck together with some sort of silk like substance. Besides the leaves there are profuse amounts of black things that look like droppings. My wife says she has see the worms, 1/8 to 1/4 inch long and white. Very tiny she says. We have tried Captain Jack's, Bayer spray, Ortho and homemade soap spray.
It's hard to tell, from your description, what kind of worm you're looking at; you could take some to the county extension service, who will be able to tell you what they are. This link will help you locate the nearest one:
Have you tried a Bt product? It works only on caterpillar-type bugs, and doesn't affect other kinds of (possibly valuable) insects. https://www.gardeningknowhow.com/plant-problems/pests/pesticides/using-bacillus-thuringiensis.htm