Yet again, the target is still moving! In this post we approach the finishing line. We have the choice of 3 different technologies when trying to image moving objects and we would like to compare them head-to-head: global shutter and rolling shutter sensors represent the classic image acquisition methods and event-based (EVB) cameras represent a different class of its own and which we mentioned before has its advantages with movement imaging.
This is the third post in this series. If you have not yet read the previous posts in this series about global shutter and rolling shutter technologies, I strongly suggest you do since we will mention a lot of what was discussed there in this post.
Introduction of the Target and the Candidates
Our target is the 4 blades fan target that we used in the previous posts. See image below:
We have 3 candidate sensor technologies
- The global shutter camera which was introduced in the first post in this series
- The rolling shutter camera which was introduced in the second post in this series
- The Event-Based camera which we have previously introduced in a different post series
We do the comparison in the visible range, grey-level imaging.
The lens used for all images is a Mitutoyo X5 objective. The camera used for the global and rolling shutter images is the IDS UI-3250CP Rev. 2 and the EVB camera is the IDS Imaging EventBasedSensor Eval-Kit ES with the Sony IMX636 and the Prophesee Metavision SDK. All cameras provided by courtesy of OpteamX.
Our head-to-head comparison will be an overall comparison i.e. not aiming for a single requirement or feature but looking at all the pros and cons each camera technology may present for a simple static and moving target.
Static Target
First, let’s look at the resulting images of the static fan that is not moving, all side-by-side:
Note that the EVB static image needed a little “help” from an additional fluorescent illumination source (unstable in time of course) in order to have some kind of a signal, otherwise we would not be able to see it at all.
Although we repeatedly mentioned in the previous posts that the static target is not interesting, there are a few functions for a static target proper imaging even when we aim to image a moving target such as reference image, calibration, alignment and so on.
So, for these functions, let’s see below the comparison table for these 3 imaging technologies for a static scene:
| Category | Global Shutter | Rolling Shutter | EVB | |
|---|---|---|---|---|
| Clear Image | Yes | Yes | Requires movement of some kind to get an image (either target or illumination) | |
| Background Image (non-target) | Yes | Yes, requires stable illumination or photo averaging | Irrelevant for EVB imaging | |
| Image Resolution | As high as the optics | As high as the optics | As high as the optics with the induced movement | |
| Pixel Sensitivity | Good QEs are in the 70%+ | Generally experience higher QEs than global shutter’s sensitivity but also in the 70% | Typical dynamic range of 120dB, which suggests high QE equivalent | |
| Focusing | Simple, classic image processing | Simple, classic image processing | Challenging, requires systematic change of target or optics with non-trivial image processing | |
| Camera Price | Low for non demanding FPS | Low for non demanding FPS | Medium priced | |
I marked in green the winner(s) in each category.
It is easy to see that, as expected, the EVB is a poor choice for a static image but there are hardly any differences between the global and rolling shutter technologies. That is why we cannot really tell the difference between the images of the static fan blade.
Note that generally rolling shutters have inherently slightly better sensitivity than global shutters so, for static scenes, where each photon counts, the rolling shutter sensors may have an advantage.
Moving Target
Let’s face it, we are here for the case of the moving target.
See below side-by-side the 3 images compared where in all 3 I have chosen to show the best image that shows the fan’s blade in a certain moment. I have also added images after processing where a reminder was needed for reference.
You must admit that it is definitely an interesting variety to choose from.
We can see that for the human eye the easiest raw image to decipher is the global shutter image but that is only because our eyes are accustomed to a certain kind of imaging. As usual, as much as I appreciate our human perception, the criteria for image quality are what image processing may do with this data, as follows:
| Category | Global Shutter | Rolling Shutter | EVB | |
|---|---|---|---|---|
| Unblurred Image | Bound by exposure time vs. target speed | Bound by exposure time vs. target speed | Can be extracted by image processing | |
| Full ROI Image | Yes | No – only bits of the target are in view in certain rows which may serve as twisted multi-ROI Requires extremely fast “roll” which is equivalent to global-like shutter | Yes | |
| Image Contrast | Bound by light intensity, exposure time and overall sensor sensitivity | Bound by light intensity, exposure time and overall sensor sensitivity | High, can operate in low light conditions | |
| Pixel Sensitivity | Good QEs are in the 70%+ | Generally experience higher QEs than global shutter’s sensitivity but also in the 70% | Typical dynamic range of 120dB, which suggests high QE equivalent | |
| Missed Movements | FPS dependent | FPS dependent | No | |
| Image Processing | Straight forward | Challenging | Specialty know how | |
| Focusing Method | Straight forward | Challenging | Specialty know how | |
| Color Available | Yes | Yes | No | |
| Camera Price | Medium for medium rate FPS High rate FPS tend to be medium-very high priced | Fast roll which may reduce the “smearing” problems tend to be medium-high priced | Medium priced |
Let’s dwell on it a little.
First, the rolling shutter is certainly out of the race – it is the winner only in the sensitivity category and even that is debatable. We cannot get our full ROI unless the roll time of the sensor is slower than the time constant of the target speed on the sensor which is in its essence a “global-like” shutter. In the cases that the shutter is indeed rolling we get a partial image of the target and we need a non-trivial image processing and algo to get something out of it.
The global shutter in our case is the next choice on the list. When our target moves fast enough, we are required to use shorter and shorter exposure times and high frame rates which becomes more challenging on the one hand in terms of light intensity and on the other hand also in terms of unit price. The global shutter has an unmatched lead in the ease and availability of relevant image processing tools compared to the other two technologies.
The EVB camera has the advantage of being an equivalent of a very high frame rate camera with high dynamic range at a medium price, which makes it the leader for very fast movements. See here for an example of the use of an EVB camera where a very high-rate FPS camera struggled and could not get the imaging done.
So, as always, we have to fit the tool we use to the task at hand. The above comparison table could have changed a bit if the requirements for the imaging system and target movement had been different but I assume the essence of it would have stayed the same. What’s left for us was to find the best-fit feature-requirement-price triangle.
That’s it!
To conclude these last 3 posts, when the target is moving using a rolling shutter camera would be a very poor choice since it poses challenges that we could simply avoid by choosing global shutter camera for the task. However, when the target is moving even faster the global shutter is already bound by camera FPS and the ratio between available light intensity to exposure time multiplied by the quantum efficiency of the sensor, which limits our availability of cameras and with that obviously their price. This paves the way to EVB cameras in these very fast moving targets to be handled with proper image processing.
Having said that, there is no clear cut decision between the global shutter and the EVB. As always it will all depend on the requirements of the optical system.
I truly hope you enjoyed these last 3 posts series as much as I did writing them and got the bits and bytes of the different technologies and how we can exploit them to our purposes and to our advantage.
Finally, I would like to thank Moshe Yanay from OpteamX for igniting the spark to pursue this kind of practical comparison between these 3 different technologies and for lending me the required cameras for the comparison.