your post intrigued me because of the use of the word "worms" in that it described exactly what I saw when I took some images of some archways at a local mosque. The archways are white plaster, nothing more. But when I got the images home, at 100% in LR the images seemed to consist of, well, little worms. I tried to clear it up and submit an image to a stock agency and it was rejected for artifacts. That never happens to me. Ever. So, am I suffering from the Fuji v Adobe issue.
I wish I could upload an image to show you but I'm not at home right now.