Markymark,
you've already been owning the XF18/55. Reading the exifs of your photos, you can verifie your personally most often used focal length...
Btw, I also suggest the XF 35/1.4 with a very nice rendering and Bokeh, the XF 27/2.8 for its minimalistic character and astonishing optical qualities and last but not least the often underestimated XF 18/2.0 for its wide angle and aperture (especially for street).
To be honest, It's your decission which kind of viewing angle you prefer most... nobody will be able to tell you... you'll have to find out yourself...
All those lenses listed up above can be had for much less money if you buy a used copy.
Another way to find a wonderfull 23mm/2.0 prime lense ist to think about a used copy of the X100S/T...
Good luck
Gato