What's your question? Ask

This is normal, for the most part. They will produce many more than they can sustain on the tree. Once the majority of the fruit drops, what is left should mature.

If none of the fruit stays on to mature, I would look at soil nutrient levels. The tree may be lacking in Phosphorus and Potassium, or may have too much Nitrogen in the soil. This will cause the symptoms that you are seeing.

These articles will help: https://www.gardeningknowhow.com/edible/fruits/fegen/reasons-for-small-hard-fruit.htm

https://www.gardeningknowhow.com/edible/fruits/plum/plum-trees-not-fruiting.htm

https://www.gardeningknowhow.com/edible/fruits/plum/growing-plum-trees.htm

If you intend to kill them, and they are your trees, then yes, doing so would be appropriate.

If these are not your trees, or are on public property, you may want to contact your local extension service for help with this.

This article will help you to find the closest extension service to you: https://www.gardeningknowhow.com/extension-search

Here are some suggestions:
Double Take 'Orange Storm' flowering quince (Chaenomeles speciosa "Orange Storm") blooms in early spring
'Klondyke' azalea (Azalea x 'Klondyke') blooms in spring
Rhododendron 'Geisha Orange'
Noisette rose (Rosa 'Crepuscule')
Witch Hazel ‘Aphrodite’, ‘Jelena’ and ‘Orange Beauty’
Butterfly weed Asclepias tuberosa
'Orange Rocket' barberry has new orange foliage
Center Glow ninebark Physocarpus opulifolius 'Center Glow'

There are a number of orange lilies, including daylilies and true lilies that form large clumps, but do not get as large as a shrub.

It's best to take them out of the bags and give them air, light and water as you wait for favorable planting conditions.

When you open the bags, there should be instructions for what to do with the plants on arrival.

Your concern is legitimate. Avoid using the Pinesol solution or any other chemical cleaning products for watering trees. Pine oil is a fragrance only and the primary ingredients are petroleum and alcohol products.

As far as where to purchase- We don't usually recommend specific vendors. You can try your local nurseries and sources, or online vendors such as Amazon, Etsy, or Ebay.

Here is some information on the tree that you will find useful: https://www.gardeningknowhow.com/ornamental/trees/yellowhorn/yellowhorn-tree-information.htm