Spots on peony leaves or mold on the plants is usually caused by one of two peony fungal diseases, botrytis blight (gray mold) or leaf blotch.
These diseases tend to show up in wet weather and plants that are infected early in the season may have severe leaf damage by late summer.
Neem Oil won't hurt, but avoid overhead watering, try to increase airflow in the area. Remove spent flower heads right away and cut back the plants in fall and discard the foliage. (do not compost)