Certified GKH Gardening Expert
Sounds like spider mites may be the culprit. Here is an article or two that you may find helpful: https://www.gardeningknowhow.com/plant-problems/pests/spider-mite-detection-and-spider-mite-natural-control.htm
I also had that problem with my gardenia, it, to me was a mite, I purchased an indoor spray for plants and soaked it....I had to do this a couple of times before all the mites were gone. I actually washed my plant off with water before I sprayed it. hope this helps.....