Most plants will not survive a winter with temps. below freezing with their roots above ground, that is in a pot. Dig a hole and 'plant' the pot, that way you can bring it up in the spring. Rosemary will not overwinter outside but most other common herbs, thyme, oregano, common sage, tarragon, will.
In asking about winter and plants, the first piece of information should be what kind of winter do you have? What are your average and minimum temperatures. This is described by knowing your USDA Hardiness zone, or simply "zone." This site will tell you more about that: http://planthardiness.ars.usda.gov/PHZMWeb/
Next is to consider what kinds of herbs you're attempting to grow. Not all herbs have the same hardiness. You can look up information for the different types you have.
These articles will give you general information: https://www.gardeningknowhow.com/edible/herbs/hgen/how-to-take-care-of-container-herbs-during-cold-weather.htm
As a help for a beginning gardener, you should look into the resources at the local county extension service. These are part of your state dept of agriculture, and most services are free. This link will help you find the nearest office: