I have the same problem. Easy to reproduce . With both the xf 23mm f/2 and the xf 18mm f/2.
OVF, manual focus, manual aperture and manual shutter speed . ISO settings has no influence. If you set either the aperture or shutter speed to A, it works fine. A strange bug...
Hopefully there will be a fix soon. You could of course set it to EVF, so not that critical, but rather annoying ;-)