Newforest

It is the internal orientation sensor(s); not sure how many Ther are if there is more than one.
Treat the camera like a phone and watch how you handle it. Your camera's sensor may be slightly off in how it calibrates orientation. Although it is not a solution, you may get better results starting it from level and sneaking up on where you want to hold it or giving it a gentle shake at level.

Just a guess, but is it possible you sometimes take photos with the camera pointed (more or less) straight up or down? I have found the X-T cameras (1 through 4) have all on occasion gotten "confused" as to photo orientation when I do that - say, pointing straight down at a flower or something.