What's your question? Ask

Unfortunately, your photo did not come through. I am unable to see the damage. Sometimes, too much nutrients can cause these types of issues, but so can virus and diseases.

The trees require pruning, quite frequently. This can help regenerate a slow tree.

In the meantime, this article will help you to understand the care that these trees require:

https://www.gardeningknowhow.com/ornamental/trees/chaste-tree/chaste-tree-cultivation.htm

Wilted leaves suggest a watering problem, possibly made worse by hot summer weather and the roots being cut off by the transplant. The watering problem can be too much water too: the fungus you refer to is simply lichen, a harmless organism that likes humid environments. The brown boxwood leaves suggest a winter injury, a lack of water or root rot.

I am located in Dallas and in the rare occasion that I prune Vitex, I try to prune before May because May is a very warm month here and pruning will trigger plants to produce new growth, something I try to avoid starting in May. But if you are only removing dead wood, May is fine.