Certified GKH Gardening Expert
Generally, this is due to poor pollination. Squash will produce fruits even if no pollination has occurred, but they will only grow to only a few inches and then die. This can easily be corrected by growing bright-colored flowers (like marigolds or zinnias) close by to entice bees to the area. You could also hand pollinate them. These articles should help:
Too much fruit set can also lead to small fruit size as well. In addition, a lack of water can decrease the size of your squash.