I planted about 6 elderberry bushes, and 2 have fruit on them and the rest do not. They get sun through out the day. The 2 that produces berries are on the end, the others behind them.
You may want to start with a soil test; this will let you know if you can make some soil amendments to fix the issue.
Your County Extension Office can help you with this.
Neem Oil is a good treatment for pest issues.