Jump to content

Software to gain pixelshift-like superresolution with Fuji X cameras


wilecoyote

Recommended Posts

Hi all,

 

I am working on a program, called VerySharp, which is inspired by the pixel-shifting feature used by some modern cameras.

It aims to provide an algorithm that enhances photo resolution using multiple exposures. This is realized by extracting sub-pixel information from the little image shifts in a series of hand-held images, yielding a relolution gain of approximately factor two under good circumstances.

 

I am a Fuji X user and therefore development an testing of the program was mainly done using my X-T10. It seems like the X-Trans sensor works well with VerySharp.
Verysharp is open source and therefore completely free. Please keep in mind that I am developing this in my free time and it is still in a quite early development stage. Especially, as I am a Linux user, I am still experimenting regarding packagin VerySharp for Windows users. So far, the provided exe file seems to run well on Windows 8 and above if recent updates are installed. So have fun while giving it a try please feel free to share feedback and any bugs you encounter :)

 

https://wilecoyote2015.github.io/VerySharp/

Link to post
Share on other sites

Interesting... a while ago I read about the automated focus rail slides that some people build to take stacked photos (to achieve greater depth of field in macrophotography). They advance the camera by the means of an Arduino-controlled stepper motor turning a wormdrive.

 

Would it be an idea to use something like this to move the camera along the x and y axis (as opposed to the z axis used in macrophotography, assuming z is the movement towards/from the subject) to achieve manageable pixel shifts ?

Link to post
Share on other sites

Interesting... a while ago I read about the automated focus rail slides that some people build to take stacked photos (to achieve greater depth of field in macrophotography). They advance the camera by the means of an Arduino-controlled stepper motor turning a wormdrive.

 

Would it be an idea to use something like this to move the camera along the x and y axis (as opposed to the z axis used in macrophotography, assuming z is the movement towards/from the subject) to achieve manageable pixel shifts ?

 

That would cost more and be harder to use than a Pentax or Olympus with pixel-shift built-in, so what would be the point?

Link to post
Share on other sites

That would cost more and be harder to use than a Pentax or Olympus with pixel-shift built-in, so what would be the point?

 

The entertainment factor would be much bigger and who knows what kind of ground-breaking discovery might be made along the way :D ​ And one can teach one's kids a trick or two while at it. Btw, the cost of components for this sort of thing is next to nothing.

 

But you are right, of course. It was just an idea.

Edited by George_P
Link to post
Share on other sites

Is this any different than using align_image_stack on an image series and taking the mean?

 

Yes. The point is that VerySharp upscales the images prior to alignment so that, under the assumption that the shifts between the images are represented by an even distribution on a subpixel level, the resulting image stack on the upscaled subpixel grid can be interpreted as a high resolution image convolved with the superposition of a box kernel with the size of the upscaling factor and the upscaling operator. Thus, proper deconvolution recovers that high resolution image to some extent, which is also performed by VerySharp.

 

Also, as far as I know, panotools use a feature-based approach for image alignment, which is the best solution for the purpose of panotools, whereas most probably it does not provide the accuracy needed here, which is the reason for using the slower iterative ECC algorithm provided by OpenCV.

 

 

Interesting... a while ago I read about the automated focus rail slides that some people build to take stacked photos (to achieve greater depth of field in macrophotography). They advance the camera by the means of an Arduino-controlled stepper motor turning a wormdrive.

 

Would it be an idea to use something like this to move the camera along the x and y axis (as opposed to the z axis used in macrophotography, assuming z is the movement towards/from the subject) to achieve manageable pixel shifts ?

 

That is an interesting idea :D

However, rotating the camera (shifting would not work due to perspectivic distortions) could not be done with the required precision using normal tools, and the projection of the lenses prevents a homogeneous shift of the image over the whole sensor area during rotation...

Doing sub-pixel alignments when shooting the images is not even necessary because doing random sampling (shooting handheld  ;)  ) and reconstructing the shifts with sub-pixel alignment does the job quite well. Of cause there is always room for improvement, but I doubt that the extensive effort would pay off in terms of resolution gain.

Edited by wilecoyote
Link to post
Share on other sites

That is an interesting idea :D

However, rotating the camera (shifting would not work due to perspectivic distortions) ...

 

May I ask, maybe you know, does the pixel-shift mechanism in Olympus rotate the sensor ? I thought it would be shifting the sensor. I am asking just out of curiosity, I know nothing about it. Another question that comes to my mind is if we are theoretically considering moving the camera to the order of the size of a pixel, if distortion plays a role at normal focal lengths. But you must have reason for mentioning that and you know the subject, as opposed to me, so I don't argue, I am just wondering.

 

To come back (just for fun) to my idiotic idea, to rotate the camera would possibly be easier than to move it along two axis and it could be built to make very small increments in the angle of rotation. But I say this just for amusement or maybe for some special purpose that is totally remote from normal photography, don't take this seriously.

 

On the other hand, your effort to use a set of random hand-held images to calculate an increase in resolution could be very practical, it is a stroke of genius if you ask me and I find it very interesting. Not because I think I need more pixels or because I think it would improve my photographic or artistic skills, but because as a technical idea and for people who do need more resolution for their work, I think it is a fascinating idea and effort on your part.

 

Compliments & Cheers.

 

Edit: For macro, rotate the subject ? :D  :D 

Edited by George_P
Link to post
Share on other sites

First of all, thanks for making this available. I'm a big fan of free/open source software and this one will come in very handy.

Just downloaded it and tried to run it, but encountered a few snags. Initially, I got the entry point error, which went away after multiple OS updates and installing the latest C++ redistributable. But running the exe after that just leads to a cmd window quickly flashing by and disappearing. After many tries, I managed to get a screenshot:

Welcome, dear visitor! As registered member you'd see an image here…

Simply register for free here – We are always happy to welcome new members!

Link to post
Share on other sites

  • 2 weeks later...

Thank you both for your kind words!

 

 

 

May I ask, maybe you know, does the pixel-shift mechanism in Olympus rotate the sensor ? I thought it would be shifting the sensor. I am asking just out of curiosity, I know nothing about it. Another question that comes to my mind is if we are theoretically considering moving the camera to the order of the size of a pixel, if distortion plays a role at normal focal lengths. But you must have reason for mentioning that and you know the subject, as opposed to me, so I don't argue, I am just wondering.

 

I have used the wrong term, sorry ;) In fact, you have to pan and tilt the camera, which corresponds to the shifting of the sensor in Olympus cameras.

 

 

First of all, thanks for making this available. I'm a big fan of free/open source software and this one will come in very handy.

Just downloaded it and tried to run it, but encountered a few snags. Initially, I got the entry point error, which went away after multiple OS updates and installing the latest C++ redistributable. But running the exe after that just leads to a cmd window quickly flashing by and disappearing. After many tries, I managed to get a screenshot:

attachicon.gifClipboard01.jpg

 

Thank you for that useful report! I will look into it as soon as I have access to a Windows machine.

Link to post
Share on other sites

Hi,

As you can easily see on the link, this is not a "download from a forum post", but a project hosted on github, which one of the most popular hosting services for Open Source software and considerably more reliable than many portals providing bloatware-infected "freeware" . Also, as the source code is provided, you are free to check the software for malware by yourself if you feel inconfident with downloading binaries.

Link to post
Share on other sites

  • 2 weeks later...

It was my take on the word git that was the issue, and it was meant humorously.

 

I'm a user of software, certainly not a developer.

Ahh yes, but Github is an essential tool nowadays for software development teams. "Everyone" uses it.

 

@OP: thanks! I will certainly take a look!

Link to post
Share on other sites

As I said I'm not decrying the use of a hub, forum, testing bed, call it what you will to enable software developers to develop.

 

Even though personally I've never heard of it.

 

Its more the name that doesn't do it justice and gives me a sense of wariness. If cola was called shitcola I probably wouldn't buy a can. A hub called github makes me move on.

 

Think they should market themselves more professionally. Maybe its a cultural thing with the word git. It no doubt stands for something technical or maybe they just don't care. They haven't got my custom, and that's the point I was making.

Link to post
Share on other sites

  • 2 weeks later...

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...