After weeks & months of watering, my soil has become compacted. Should I loosen it or leave it?
I would not recommend disturbing the roots of your plants. If you are concerned that the dirt is too compacted you can go down to your local wallmart or fishing supply (sometimes even the local corner gas station) and get a container of worms. Place a few in each container and let them go to work. If the containers are outside dig up a little area near the edge and bury the worms you are adding so the birds don't see them.
you could leave it, but for container planting there's no reason you cant go to a local bait shop and buy a dozen night crawlers. dump them in the box, they will aerate the soil for you.