Certified GKH Gardening Expert
Burning bushes are hardy to zone 4 (you are 4b) but perhaps you had an extra harsh winter. You can break off a couple inches of one of the stems or just scrape off some bark. If there is green showing, then it is still alive. If the inside is brittle and white, I'd say replace it.
Here is more information on burning bushes: