Last year our tree had small and hard fruit. We got no usable fruit. Two years before we had probably 100 or more, next year was just a few, then last year they were 20-30 but none were usable.
Meyer Lemon Tree
Certified GKH Gardening Expert
This sounds like a case of phosphorus and potassium deficiencies. You can correct this by using a fertilizer that is specific to citrus to take out the guesswork. Use this once per year in spring. This will correct the issue.
Here is an article that will explain in more detail: https://www.gardeningknowhow.com/edible/fruits/citrus/citrus-fruit-thick-rind.htm