Water deficit (drought stress) and/or nutrient deficiency are the most common reasons for this. The tree will naturally drop the fruit that it cannot support to maturity with available water and nutrients. The way to help the tree retain the most fruit until they ripen, is to provide adequate water and soil fertility. Mulching the soil surface can help to retain soil moisture.
Sometimes some natural thinning of fruit, especially for a young tree, is not all bad. Excessive weight of a heavy fruit load can cause branch breakage. And the fruit that remains to mature and ripen will be larger and better quality than with a lot of competition for water and nutrients.
Here's an example of a slow release organic fertilizer: https://www.downtoearthfertilizer.com/products/blended_fertilizer/citrus-mix-6-3-3/