Certified GKH Gardening Expert
There could be a number of issues that could be affected the growth. If you concerned about fertilization, make sure to use a good all-purpose fertilizer.
Now, as far as diseases go... You can usually tell if the tree is damaged from disease. There will be leaf spots, shriveled fruit, mold, or sap oozing everywhere. These can be treated in various ways depending on the specific disease.
Here are two articles
1. This one will give you more information on cold hardy figs:https://www.gardeningknowhow.com/edible/fruits/figs/cold-hardy-fig-trees.htm
2 Here is one to help you identify any diseases starting:https://www.gardeningknowhow.com/edible/fruits/figs/cold-hardy-fig-trees.htm