I have a peach tree little over year old has a white looking fuzz on the trunk and some branches. The tree had nice peaches this year made sure it didn't overdo itself and took blooms off. It's about an 8 to 10 ft tree.
Peach Tree With White Fuzz
Certified GKH Gardening Expert
Without seeing an image it is difficult, but from your description this could be a serious issue with a wood rot fungus.
The white fuzz as you mention is the reproductive bodies of the fungus.
Stressed or injured trees are susceptible to these fungi.
The tree is generally not salvageable and should be removed from the yard.
I do think you should do a bit of inspection and checking to be sure you don't have another issue instead since I don't have an image to look at.
Mealybug and Wooly Aphids produce white looking material. They would be treatable with Neem Oil.
Also Lichens grow on trees and cause no issue to the tree health, they also could appear white.
I am listing several links for you.