Certified GKH Gardening Expert
This is a pretty hotly debated topic, and depending upon who you ask, you're going to find pretty strong opinions for both sides. My opinion, however, and the opinion of many scientists, is that it's not a problem. Plastic is everywhere in our lives, and we all come into contact with non-food-grade plastic more often than we're even aware. One of the big concerns is, of course, BPA, but the scientific consensus seems to be that BPA is too big to pass easily through the cell walls of plants.
This article goes into more depth about gardening with plastic: