Q.Is Sweet Annie harmful to grow near any vegetables?
Specifically tomatoes and pole beans

Certified GKH Gardening Expert
I did a little research on this and it seems that this plant should not be grown next to other herb-like plants, such as caraway or anise, but it's great for warding off harmful insects in your vegetable garden. I know it has a strong scent, but it should be more helpful in your garden than harmful. You may want to give it a bit of distance, simply because it can become invasive. This article may be helpful:
https://www.gardeningknowhow.com/edible/herbs/wormwood/wormwood-plant-growing-sweet-annie.htm