Ejecting the tray with the two batteries does fix the issue, but only temporarily. After awhile using the camera, the icons eventually will turn red.
It seems the batteries last just fine, but the display just doesn't reflect the actual charge level. I've also found that it happens much more frequently when shooting video.