Certified GKH Gardening Expert
According to the article below, picking the dates when ripe or just before is desirable. Sometimes weather or humidity can cause fruit drop. When you say stems are you referring to the stems of the fruit cluster? Those can be cut off although not all the fruit in the cluster ripens at the same time.