Our Avocado tree produces hundreds, maybe thousands of fruit, but all but a half dozen or less actually grow up and survive as regular sized fruit. The fruit starts falling off when they become golfball-ish size. The tree seems healthy. I live in Cape Coral, Florida (SW).
It is very normal for them to drop many fruits, as they will not make it to maturity anyways. This is a survival adaptation so that they get, at least, a few fruit.
It is likely that the soil lacks a sufficient amount of phosphorous and potassium to sustain a large fruit crop. I would make sure to fertilize the tree with a specific avocado fertilizer yearly, or add a good layer of compost around the area each year. This will help maximize the fruit production, as well as keep your tree healthy throughout the year. This article will help: https://www.gardeningknowhow.com/edible/fruits/avocado/avocado-tree-growing.htm