Certified GKH Gardening Expert
If the blooms have simply faded and are now dropping, the plant could be trying to go into dormancy. After blooming, calla lilies need a rest period. Give it time to go dormant and the plant should come back fine after this. Hold back on water do help it enter dormancy.
If the blooms were fine and then suddenly began dropping, this could be due to overwatering, which would also cause the leaf yellowing. Back off on the watering and allow the plant to dry out some before watering again.