Q.Victoria Plum Tree Problem
We have a 7 year old Victoria plum tree which has never fruited very well. Last year what fruit ripened was marked and scarred. This year it appeared to be budding prolifically but some leaves have got holes in them without evidence of ‘bugs’ and the end of the new growth are drying and curling. Also, the buds are all going bright yellow and falling off. Any advice would be appreciated.

It does sound like aphid damage to your tree and fruit.
Neem Oil is a good treatment.
https://www.gardeningknowhow.com/edible/fruits/plum/control-leaf-curl-plum-aphids.htm
https://www.gardeningknowhow.com/plant-problems/pests/pesticides/neem-oil-uses.htm