Certified GKH Gardening Expert
For the rose bushes that come in pots and are growing and blooming, usually the end of May is okay, but check your location for the last hard frost date in your area. If a hard frost coming once it's been planted, wrap a bath towel around the bush, then put a trashbag over the entire thing. The bath towel provides some warm insulation and the plastic bag keeps the towel from getting wet. Place some large stones over the edges of the plastic bag on the ground to hold the bag in place. Be sure to remove it the next morning.
For additional help, read these articles: https://www.gardeningknowhow.com/ornamental/flowers/roses/location-to-grow-roses.htm and https://www.gardeningknowhow.com/ornamental/flowers/roses/planting-rose-bushes.htm