What's your question? Ask

Smaller leaves and less leaves can be a symptom of some sort of stress or root damage or inner wood disease.

The leaves are generally a late symptom and the damage has all ready occurred.
You can mulch around the tree with 2 to 3 inches of hardwood chips and put the tree on a regular watering schedule.
I would not try to fertilize the tree into health.
Wait and see.

Watch for signs of fungus on the tree and also around the base of the tree.
There is a disease called Armillaria Root Rot.

Here are a few links for you.

http://www.na.fs.fed.us/spfo/pubs/fidls/armillaria/armillaria.htm

https://www.gardeningknowhow.com/ornamental/trees/fruitless-mulberry/fruitless-mulberry-trees.htm

This is likely Leaf Spot, a fungus.

Here is a link with more information.

https://www.gardeningknowhow.com/ornamental/trees/fruitless-mulberry/possible-causes-of-a-fruitless-mulberry-with-yellow-leaves.htm

The leaf issue on the branches may have been from environmental stresses. Cold weather in the spring, heavy rains or dry spells.

You can prune any dead or damaged branches anytime but waiting until winter is always a good idea.
Here is a link to refresh you on the care information.

https://www.gardeningknowhow.com/ornamental/trees/fruitless-mulberry/fruitless-mulberry-trees.htm

Yellowing leaves generally do indicate a watering issue. To much or to little.

With the reports of heavy rain, this is likely the issue.
There are a few other issues for the Fruitless Mulberry with symptoms of yellowing leaves.
I would suggest contacting your local county extension office and expressing your findings on the multiple trees with symptoms.
They may be aware of any widespread problems.
The links below can also help you find the nearest office and have more information.

https://www.gardeningknowhow.com/ornamental/trees/fruitless-mulberry/possible-causes-of-a-fruitless-mulberry-with-yellow-leaves.htm
https://www.gardeningknowhow.com/extension-search/

There can be a few reasons for the yellowing leaves.

Here is a link that may help you determine the cause.

https://www.gardeningknowhow.com/ornamental/trees/fruitless-mulberry/possible-causes-of-a-fruitless-mulberry-with-yellow-leaves.htm

Do a Google search for the fruitless Mulberry.

You should be able to locate a nursery that can ship to you.

Also check Ebay, they can be a good source for plants and trees. Make sure you verify the sellers feedback and that they are a reputable seller.