What's your question? Ask

Osage Orange is dioecious (have male and female plants), but the female plant (pistillate) will still produce fruit without pollination… it just lacks seeds! The trees take about 10 years to mature, and it is not really possible to determine gender of the tree before then

You may want to remove the small fruit from your young tree. This will help your tree use it's energy in putting down roots instead of fruit.

https://www.gardeningknowhow.com/edible/fruits/oranges/growing-an-orange-tree.htm
https://www.gardeningknowhow.com/edible/fruits/oranges/pruning-orange-trees.htm
https://www.gardeningknowhow.com/edible/fruits/oranges/harvesting-oranges.htm

What you describe is not natural, but normally would indicate a water deficit (drought stress) or toxic reaction to alchohol or a chemical in the soil, or a toxic foliar spray of some kind.
Water deficit is the most likely, but I am surprised that the plant couldn't go one week without wilting and dropping leaves. Was it real hot in the house, is the pot very small, or does the soil drain very fast and not retain soil moisture?

Water deficit (drought stress) and/or nutrient deficiency are the most common reasons for this. The tree will naturally drop the fruit that it cannot support to maturity with available water and nutrients. The way to help the tree retain the most fruit until they ripen, is to provide adequate water and soil fertility. Mulching the soil surface can help to retain soil moisture.

Sometimes some natural thinning of fruit, especially for a young tree, is not all bad. Excessive weight of a heavy fruit load can cause branch breakage. And the fruit that remains to mature and ripen will be larger and better quality than with a lot of competition for water and nutrients.

Here's an example of a slow release organic fertilizer: https://www.downtoearthfertilizer.com/products/blended_fertilizer/citrus-mix-6-3-3/

This sounds like a fungal issue. Sometimes, it can resolve itself, but to be sure, you will want to apply dolomitic lime, and wettable sulfur to kill off any disease in the soil. Here are some citrus tree articles for further reading: https://www.gardeningknowhow.com/edible/fruits/citrus/

You may have a calamondin orange or kumquat. These are genetically predisposed with sour taste and cannont be made sweet.
But if you are certain that you have a sweet orange variety then the sour taste could be due in nutrient deficiency and water deficit.
A full range of plant nutrients, adequate water and sunlight are needed to for the process of photosynthesis which makes sugars for metabolism and for storage in the fruit.
There are all purpose fertilizers like the Dr Earth Organic All Purpose on this page:
http://www.fertileearthlandcare.com/domyown/

As well as special fertilizers formulated for citrus like this:
https://www.downtoearthfertilizer.com/products/blended_fertilizer/citrus-mix-6-3-3/and citrus fertilizers like this