I am a big fan of the XC 15-45, and while I do not own the 23 mm prime, I do have a 35 mm prime. I shot both lenses ad nauseum to see which was better, and as much as I like the XC 15-45, the prime offers the faster aperture and sharper detail, which doesn't really become a factor unless you print the photo or pixel peep it. The XC zoom, however, takes a mighty fine photo, and the utility of a zoom comes in handy when shooting while you are on the move. Not mention the XC's miniscule weight, also important when you are on the move.
The way I decide which to shoot relies mainly on my plans for the session. If I'm shooting creative compositions in a fixed location, I choose the prime every time. If I'm out and about, the XC 15-45 is my tool of choice. I am a proponent of having both, once you get used to the unconventional operation of the XC 15-45.