I had the same problem with the XF27mm f2.8.
Found the below on a Reddit thread.
"There is apparently a design flaw with this lens that causes it to get stuck sometimes when the camera is turned off at the closest focal distance (i.e. when the lens is fully extended.)
The fix is to just push it back in manually (gently, of course.)"
It fixed the problem straight away.