Certified GKH Gardening Expert
It may be that you are just outside of the upper end of their hardiness zone. They are hardy to our USDA zone 8, where much of your area seems to be in USDA zone 9.
This could cause slower growth. You can try taking a soil test. This will identify any pH issues or nutrient imbalances to rule them out. If correction is necessary, then this may help to give them the conditions to help spread.
https://www.gardeningknowhow.com/garden-how-to/soil-fertilizers/soil-ph-plants.htm
https://www.gardeningknowhow.com/garden-how-to/soil-fertilizers/testing-soil.htm