Would this be a weed??
Yes. That is an invasive shrub. In fact, it looks to be Solanum aviculare or the Kangaroo Apple. It is quite beautiful, but very toxic.
This article will offer more information on the subject: https://www.gardeningknowhow.com/ornamental/shrubs/kangaroo-apple/kangaroo-apple-growing.htm