My lime tree has suffered from curly leaf and also shows discoloration of leaves, exhibiting yellow tinges around the spine of the leaf. It has been in good sun and soil for five years and buds but drops the small fruit off the plant before it can mature.
Curly Leaf on Lime Tree
Certified GKH Gardening Expert
It is probably a lack of water and nutrients. Regular watering is very important to lime tree health and irregular watering would cause both fruit falling off and the problem with the leaves.
These article should help: https://www.gardeningknowhow.com/edible/fruits/lime/is-lime-fruit-and-lime-blossoms-falling-off-tree-normal.htm