It is doubtful that the trees prevented the garden from producing, especially if the plants were healthy. However, if they cast too much shade, then this could be an issue. Also, the walnut tree is nothing to worry about unless it's a black walnut, which would literally poison the ground around it.
If you are getting plenty of flowers but nothing produces in the garden, then it is most likely due to poor pollination. Hand pollination often helps. These articles will help you with attracting more pollinators: