Certified GKH Gardening Expert
Fruit on most fruit trees will have an off flavor if the tree is stressed. Lack of water or nutrients is the most common reason for stress, so make sure that you are providing enough of these two things. This article will help with feeding your fruit trees: https://www.gardeningknowhow.com/edible/fruits/citrus/fertilizing-citrus-trees-best-practices-for-citrus-fertilizing.htm
How often you harvest really depends on the type of fruit tree you have.