I have a couple of questions. Towards the end of the season my blackberries started looking a little odd. The berries were irregular. Each berry would be normal and plump as well as very small berries contained in each berry overall. Yet, on the same bush there would be normal size berries. This didn't occur until the end of the season and seemed to be on the ones that were exposed to the sun. Those under the leaves were of normal size. The second question is I've noticed quite a few bugs that resemble stink bugs on the plants and on the ground at the base of the plants. They don't seem to be causing any damage but I wonder if they could be a problem. I haven't sprayed with any pesticides or chemical fertilizers at all this season.
It sounds like you have insect damage. Consider using an organic type of insecticide to help control that. Kaolin Clay is a good choice to use on berry bushes.
Also you may find this article helpful: