Clicky

Jump to content


Photo

Software to gain pixelshift-like superresolution with Fuji X cameras


  • Please log in to reply
19 replies to this topic

#1 wilecoyote

wilecoyote

    Newbie

  • Members
  • Pip
  • 5 posts

Posted 15 November 2016 - 12:23 PM

Advertisement (Gone after free registration)

Hi all,

 

I am working on a program, called VerySharp, which is inspired by the pixel-shifting feature used by some modern cameras.

It aims to provide an algorithm that enhances photo resolution using multiple exposures. This is realized by extracting sub-pixel information from the little image shifts in a series of hand-held images, yielding a relolution gain of approximately factor two under good circumstances.

 

I am a Fuji X user and therefore development an testing of the program was mainly done using my X-T10. It seems like the X-Trans sensor works well with VerySharp.
Verysharp is open source and therefore completely free. Please keep in mind that I am developing this in my free time and it is still in a quite early development stage. Especially, as I am a Linux user, I am still experimenting regarding packagin VerySharp for Windows users. So far, the provided exe file seems to run well on Windows 8 and above if recent updates are installed. So have fun while giving it a try please feel free to share feedback and any bugs you encounter :)

 

https://wilecoyote20...b.io/VerySharp/


  • milandro, Vidalgo and George_P like this

#2 kimcarsons

kimcarsons

    Advanced Member

  • Members
  • PipPipPip
  • 206 posts

Posted 15 November 2016 - 04:35 PM

Is this any different than using align_image_stack on an image series and taking the mean?



#3 George_P

George_P

    Advanced Member

  • Members
  • PipPipPip
  • 65 posts
  • LocationBratislava

Posted 15 November 2016 - 06:07 PM

Interesting... a while ago I read about the automated focus rail slides that some people build to take stacked photos (to achieve greater depth of field in macrophotography). They advance the camera by the means of an Arduino-controlled stepper motor turning a wormdrive.

 

Would it be an idea to use something like this to move the camera along the x and y axis (as opposed to the z axis used in macrophotography, assuming z is the movement towards/from the subject) to achieve manageable pixel shifts ?


X-T10 | 8mm | 18-55 | 35/2 | 90 | 50-230 | FlashQ


#4 kimcarsons

kimcarsons

    Advanced Member

  • Members
  • PipPipPip
  • 206 posts

Posted 15 November 2016 - 06:18 PM

Interesting... a while ago I read about the automated focus rail slides that some people build to take stacked photos (to achieve greater depth of field in macrophotography). They advance the camera by the means of an Arduino-controlled stepper motor turning a wormdrive.

 

Would it be an idea to use something like this to move the camera along the x and y axis (as opposed to the z axis used in macrophotography, assuming z is the movement towards/from the subject) to achieve manageable pixel shifts ?

 

That would cost more and be harder to use than a Pentax or Olympus with pixel-shift built-in, so what would be the point?



#5 George_P

George_P

    Advanced Member

  • Members
  • PipPipPip
  • 65 posts
  • LocationBratislava

Posted 15 November 2016 - 06:35 PM

That would cost more and be harder to use than a Pentax or Olympus with pixel-shift built-in, so what would be the point?

 

The entertainment factor would be much bigger and who knows what kind of ground-breaking discovery might be made along the way :D ‚Äč And one can teach one's kids a trick or two while at it. Btw, the cost of components for this sort of thing is next to nothing.

 

But you are right, of course. It was just an idea.


Edited by George_P, 15 November 2016 - 07:15 PM.

X-T10 | 8mm | 18-55 | 35/2 | 90 | 50-230 | FlashQ


#6 wilecoyote

wilecoyote

    Newbie

  • Members
  • Pip
  • 5 posts

Posted 15 November 2016 - 09:57 PM

Is this any different than using align_image_stack on an image series and taking the mean?

 

Yes. The point is that VerySharp upscales the images prior to alignment so that, under the assumption that the shifts between the images are represented by an even distribution on a subpixel level, the resulting image stack on the upscaled subpixel grid can be interpreted as a high resolution image convolved with the superposition of a box kernel with the size of the upscaling factor and the upscaling operator. Thus, proper deconvolution recovers that high resolution image to some extent, which is also performed by VerySharp.

 

Also, as far as I know, panotools use a feature-based approach for image alignment, which is the best solution for the purpose of panotools, whereas most probably it does not provide the accuracy needed here, which is the reason for using the slower iterative ECC algorithm provided by OpenCV.

 

 

Interesting... a while ago I read about the automated focus rail slides that some people build to take stacked photos (to achieve greater depth of field in macrophotography). They advance the camera by the means of an Arduino-controlled stepper motor turning a wormdrive.

 

Would it be an idea to use something like this to move the camera along the x and y axis (as opposed to the z axis used in macrophotography, assuming z is the movement towards/from the subject) to achieve manageable pixel shifts ?

 

That is an interesting idea :D

However, rotating the camera (shifting would not work due to perspectivic distortions) could not be done with the required precision using normal tools, and the projection of the lenses prevents a homogeneous shift of the image over the whole sensor area during rotation...

Doing sub-pixel alignments when shooting the images is not even necessary because doing random sampling (shooting handheld  ;)  ) and reconstructing the shifts with sub-pixel alignment does the job quite well. Of cause there is always room for improvement, but I doubt that the extensive effort would pay off in terms of resolution gain.


Edited by wilecoyote, 15 November 2016 - 10:22 PM.

  • George_P likes this

#7 George_P

George_P

    Advanced Member

  • Members
  • PipPipPip
  • 65 posts
  • LocationBratislava

Posted 16 November 2016 - 09:45 AM

That is an interesting idea :D

However, rotating the camera (shifting would not work due to perspectivic distortions) ...

 

May I ask, maybe you know, does the pixel-shift mechanism in Olympus rotate the sensor ? I thought it would be shifting the sensor. I am asking just out of curiosity, I know nothing about it. Another question that comes to my mind is if we are theoretically considering moving the camera to the order of the size of a pixel, if distortion plays a role at normal focal lengths. But you must have reason for mentioning that and you know the subject, as opposed to me, so I don't argue, I am just wondering.

 

To come back (just for fun) to my idiotic idea, to rotate the camera would possibly be easier than to move it along two axis and it could be built to make very small increments in the angle of rotation. But I say this just for amusement or maybe for some special purpose that is totally remote from normal photography, don't take this seriously.

 

On the other hand, your effort to use a set of random hand-held images to calculate an increase in resolution could be very practical, it is a stroke of genius if you ask me and I find it very interesting. Not because I think I need more pixels or because I think it would improve my photographic or artistic skills, but because as a technical idea and for people who do need more resolution for their work, I think it is a fascinating idea and effort on your part.

 

Compliments & Cheers.

 

Edit: For macro, rotate the subject ? :D  :D 


Edited by George_P, 16 November 2016 - 10:52 AM.

X-T10 | 8mm | 18-55 | 35/2 | 90 | 50-230 | FlashQ


#8 pipetman2

pipetman2

    Newbie

  • Members
  • Pip
  • 1 posts

Posted 16 November 2016 - 10:21 PM

First of all, thanks for making this available. I'm a big fan of free/open source software and this one will come in very handy.

Just downloaded it and tried to run it, but encountered a few snags. Initially, I got the entry point error, which went away after multiple OS updates and installing the latest C++ redistributable. But running the exe after that just leads to a cmd window quickly flashing by and disappearing. After many tries, I managed to get a screenshot:

Attached File  Clipboard01.jpg   113.43KB   16 downloads



#9 wilecoyote

wilecoyote

    Newbie

  • Members
  • Pip
  • 5 posts

Posted 27 November 2016 - 09:55 AM

Thank you both for your kind words!

 

 

 

May I ask, maybe you know, does the pixel-shift mechanism in Olympus rotate the sensor ? I thought it would be shifting the sensor. I am asking just out of curiosity, I know nothing about it. Another question that comes to my mind is if we are theoretically considering moving the camera to the order of the size of a pixel, if distortion plays a role at normal focal lengths. But you must have reason for mentioning that and you know the subject, as opposed to me, so I don't argue, I am just wondering.

 

I have used the wrong term, sorry ;) In fact, you have to pan and tilt the camera, which corresponds to the shifting of the sensor in Olympus cameras.

 

 

First of all, thanks for making this available. I'm a big fan of free/open source software and this one will come in very handy.

Just downloaded it and tried to run it, but encountered a few snags. Initially, I got the entry point error, which went away after multiple OS updates and installing the latest C++ redistributable. But running the exe after that just leads to a cmd window quickly flashing by and disappearing. After many tries, I managed to get a screenshot:

attachicon.gifClipboard01.jpg

 

Thank you for that useful report! I will look into it as soon as I have access to a Windows machine.



#10 MirrorMirror

MirrorMirror

    Advanced Member

  • Validating
  • PipPipPip
  • 79 posts

Posted 27 November 2016 - 11:19 AM

Sorry mate. I'm sure your intentions are honourable but personally I wouldn't dream of downloading and installing software from someone's forum post. It's just too dodgy in my opinion. Each to their own though, I'm sure you'll get some good feedback from others.
  • Rip57 likes this

#11 wilecoyote

wilecoyote

    Newbie

  • Members
  • Pip
  • 5 posts

Posted 27 November 2016 - 11:33 AM

Hi,

As you can easily see on the link, this is not a "download from a forum post", but a project hosted on github, which one of the most popular hosting services for Open Source software and considerably more reliable than many portals providing bloatware-infected "freeware" . Also, as the source code is provided, you are free to check the software for malware by yourself if you feel inconfident with downloading binaries.


  • George_P likes this

#12 mdm

mdm

    Advanced Member

  • Members
  • PipPipPip
  • 339 posts

Posted 27 November 2016 - 08:46 PM

as the source code is provided, you are free to check the software for malware by yourself...

 

nice advice but wrong forum...



#13 MirrorMirror

MirrorMirror

    Advanced Member

  • Validating
  • PipPipPip
  • 79 posts

Posted 27 November 2016 - 09:47 PM

A host site called github doesn't exactly inspire me with confidence! :)

http://www.urbandict...ne.php?term=Git

 

Seriously though, not for me.  I wouldn't know what to do or like to take the risk; software fully developed for my PC is for me

 

Good luck though....


Edited by MirrorMirror, 27 November 2016 - 10:06 PM.


#14 wilecoyote

wilecoyote

    Newbie

  • Members
  • Pip
  • 5 posts

Posted 27 November 2016 - 11:06 PM

 

Seriously though, not for me.  I wouldn't know what to do or like to take the risk; software fully developed for my PC is for me

 

 

That is a very understandable view, of course :)
 

 

Good luck though....

 

 

Thank you!



#15 johant

johant

    Advanced Member

  • Members
  • PipPipPip
  • 620 posts

Posted 10 December 2016 - 11:00 AM



A host site called github doesn't exactly inspire me with confidence! :)


You are clearly not a software developer

#16 MirrorMirror

MirrorMirror

    Advanced Member

  • Validating
  • PipPipPip
  • 79 posts

Posted 10 December 2016 - 11:07 AM

It was my take on the word git that was the issue, and it was meant humorously.

I'm a user of software, certainly not a developer.

#17 johant

johant

    Advanced Member

  • Members
  • PipPipPip
  • 620 posts

Posted 10 December 2016 - 11:16 AM

It was my take on the word git that was the issue, and it was meant humorously.

I'm a user of software, certainly not a developer.

Ahh yes, but Github is an essential tool nowadays for software development teams. "Everyone" uses it.

@OP: thanks! I will certainly take a look!

#18 MirrorMirror

MirrorMirror

    Advanced Member

  • Validating
  • PipPipPip
  • 79 posts

Posted 10 December 2016 - 11:27 AM

As I said I'm not decrying the use of a hub, forum, testing bed, call it what you will to enable software developers to develop.

Even though personally I've never heard of it.

Its more the name that doesn't do it justice and gives me a sense of wariness. If cola was called shitcola I probably wouldn't buy a can. A hub called github makes me move on.

Think they should market themselves more professionally. Maybe its a cultural thing with the word git. It no doubt stands for something technical or maybe they just don't care. They haven't got my custom, and that's the point I was making.

#19 lauriebrett

lauriebrett

    Member

  • Members
  • PipPip
  • 24 posts
  • LocationEast Lothian, Scotland, UK

Posted 13 December 2016 - 10:55 PM

This sounds like a really exciting piece of software. I'm not at all programming literate and the install for Mac sounds daunting for me. If you make a easy to install version for Mac I would be happy to pay for it.
Good luck though, I hope this takes off.

#20 dlabrosse

dlabrosse

    Newbie

  • Members
  • Pip
  • 7 posts

Posted 25 December 2016 - 06:08 PM

This sounds very interesting. I will wait however until a more friendly Mac installation exists.




 
x

Registration is free

Not registered? Really?

Discover the full potential of the Fuji X Forum... register now!
Registration is free and done in a few minutes!

As registered member you can discuss, post your questions and present your images.
And get in contact with Fuji X photographers worldwide!

We are looking forward to you!

The Fuji X Forum Team

Register Close