Q.Small black worms eating flowers
Small worms that I also find falling off trees into my pool, very strange looking – not smooth skinned but prickly looking.
More likely a caterpillar then a worm.
Some caterpillars can be eliminated with something like Neem Oil.
Correctly identifying the infestation will make the treatment more effective.
Here are some links for you.
https://www.gardeningknowhow.com/plant-problems/pests/insects/tent-worms-control.htm
https://www.gardeningknowhow.com/plant-problems/pests/insects/prevent-caterpillars.htm
https://www.gardeningknowhow.com/plant-problems/pests/pesticides/using-bacillus-thuringiensis.htm