If they stay on the tree through to the fall, it may be that your tree is getting just a little bit too little chilling. Without enough chilling, peach trees will not flower properly. If the chilling time is just a little bit short, the fruit will form, but will not ripen. You may want o check the variety and make sure it is appropriate for your area in terms of chilling time.
If the fruit falls off before fall, then this is a pollination issue. These articles will help you with that: