View Single Post
Dorian Gray
Veteran Member
 
Join Date: May 2005
Location: Paris, France
 
2011-02-06, 16:40

It is a good read (thanks for linking), but the DxOMark Sensor score is given too much weight by the author. In practice, with the various sensor technologies and sizes available, it's impossible to meaningfully sum up sensor performance in one number. The real value of the DxOMark website lies in its comprehensive information, not its arbitrarily-weighted single-number scores.

Consider for example that DxOMark ranks the Nikon D7000 as a better landscape camera than the 40-megapixel medium-format Pentax 645D. Which do you think a typical landscape photographer would prefer?

Pentax gets revenge though: the 645D is ranked above the EOS 1D Mark III for sports.

DxOMark ignores the advantages offered by high pixel counts and fast sensor readouts, among many other sensor-related and non-sensor-related factors that should influence one's choice of camera. That's one reason why Canon cameras don't fare as well as you might expect. They've long had market-leading pixel counts, fast readouts in many models, and low noise at high ISOs; but their sensors have lagged behind in base-ISO dynamic range and hue resolution (the latter producing the "Canon skin tones" that many people like, particularly in low-Kelvin light).

Furthermore, although a camera's maximum dynamic range strongly affects its DxOMark Sensor score, the dynamic range measurement used by DxOMark isn't entirely useful. That's because it's the engineering definition of dynamic range, i.e. using a signal-to-noise ratio greater than 1 as the lower cut-off point. That is too low for most photographic purposes, since we don't like our images to be almost entirely swamped by noise.

DxOMark rate the D7000 better than the 645D for landscape because the D7000 has a higher maximum engineering dynamic range (13.87 stops versus 12.55 stops when normalised to 8 megapixels). But pull up the "Full SNR" curves (not available in comparison mode) for each of the cameras to see a different story.

Consider a common landscape scene with a subject brightness range of 7 stops. Adjusting slightly for typical lens flare, the important shadows of this subject will be recorded at 1% of pixel saturation, in an "exposed to the right" file. Which camera at base ISO produces more noise at 1% saturation? Turns out the 645D has a signal-to-noise ratio about 2 dB greater, after normalisation for its greater pixel count, so it produces less noise in the shadows. This runs counter to DxOMark's dynamic range measurement of the two cameras, since that measurement is based on a uselessly low signal-to-noise ratio (for landscape photography; for reading number plates in a crime lab, a signal-to-noise ratio greater than 1 may be all that's required!).

In the mid tones, where the interesting colour often occurs in landscape photos, the 645D is about 3 dB better than the D7000, i.e a full "stop" better, and that's before mapping the raw file to a working colour space during raw conversion. Doing that — which is obviously essential to view and use the raw file — gives the 645D a further advantage in chroma noise performance, because it uses stronger colour filters in the Bayer matrix than the D7000 (as seen in the Color Sensitivity graphs). So if you're yanking curves and saturation controls to bring out vivid natural colours, as traditional landscapers usually do, the 645D has a strong advantage over the D7000. (This shouldn't surprise anyone: I'm just using it to remind ourselves why the DxOMark landscape score isn't terribly useful for doing what its name suggests.)

In short, DxO Labs provide good information, but their summary figures of merit are almost useless. I therefore question the validity of any analysis that uses those figures of merit, as Peter van den Hamer's does. It's still a thought-provoking read, and van den Hamer doesn't only rely on the DxOMark Sensor score, of course.
  quote