Back To Listings RSS Print

Single Chip Camera Evaluation

The Big Chip Camera Shootout!

By Steve Hullfish | April 13, 2011

image

At the request of Zacuto in Chicago, Bob Primes, ASC led a group of over 100 intrepid volunteers and ASC colleagues in the creation of The Single Chip Camera Evaluation. Basically, a well-planned and fairly comprehensive shootout between the Arri Alexa, Sony F35, RED MX, Phantom Flex, WeissCam, Sony F3, Panny AF100. Canon 5D, 1D, 7D, Nikon D7000 and Kodak 5213 film.

The following write-up is my assessment of the presentation of the event. As the organizers point out, many of the results are subjective and there was no clear "winner," though several of the panelists expressed a pretty un-restrained love for the clarity, beauty and simplicity of the Alexa, closely followed for their adoration of the F35.

To start the event - held at the end of the day Tuesday at NAB to a packed, standing room only auditorium of appreciative attendees - they showed the video of the tests themselves. The organizers hope to soon release a Bluray DVD of the video. For more on the availability and more specific results of the tests, please visit thescce.org.

Here are my quick notes on the video itself:

They broke the testing down to several specific categories including: Sharpness, low light capability, exposure latitude, highlight detail, shadow detail, color reproduction, skin reproduction, shutter artifacts.

In the sharpness test, they were looking at fine detail and contrast. They used a Siemens star chart. In the sharpness test, the RED ONE and film were tops, while the 1D, 5D, 7D and Nikon were among the worst. The results were displayed as a graph.

As for the Comparative sharpness test, the image was blown up about 230% to determine how sharp the image was. Obviously, cameras with a larger size - like the 4K Red image faired a lot better being blown up. The results, however, were subjective and open to interpretation. Someone may disagree with me on my impressions. Many high level DPs were in the audience and I walked out of the event with Art Adams and Adam Wilt. Hopefully they will post their impressions of the test as well. With that caveat, I thought that, F35, Alexa, RED, film, and Phantom were good. While the Panny 100 and the slew of DSLRs were the worst.

Throughout the test, there were several times that the audience broke into knowing laughter or at least audible recognition of the fault of a specific camera. The softness of the 5D in this test was the first place that this communal realization happened.

Let me just point out that of ALL the cameras tested, the only one I personally OWN is the 5D, so please don't feel I'm slamming the DSLRs due to some elitist bias. I WANT the 5D to be come out ahead! But I'm realistic and trying to be objective.

In the low light Sensitivity test, they wanted to see which images could resolve an image in the lowest possible light - apart from the base noise floor. This was about the only category in the entire slate of tests that the Canon 5D came out on top, objectively.

In the real world shooting test for low light, it was hard to see a difference. In these subjective, comparative tests, the D.P.s determined one "best" image to use as a comparison to go back and forth between the other cameras, kind of as a reference. They explained that choosing that reference was subjective, but, in some since, was representative of the best camera performance in that particular test. In the case of the "real world" low light test, they chose the Alexa as the reference. The 5D was at the bottom, subjectively, though there wasn't a lot of time to judge nuances.

The Dynamic range test was an objective test based on how many stops of range could be seen. Film and Alexa were at the top. Actually, Alexa had the best latitude with film at something like 14 stopsand the Alexa beating it with something like 14.2 stops. The Red was also very good. The F3 and DSLRs also rated well.

In the Highlight detail real world test, they shot a scene with a bright window in the background. They exposed to blow out the window in the camera, then used color correction to bring back whatever they could using a Power Window. In this test, which was subjective, the Phantom. Weisscam, F3, AF100, and all of the DSLRs were bad. Again, a knowing group chuckle went up at the 5D. The organizers wanted to point out that the Weisscam and one of the other RAW cameras had some problem where they were unable to use the RAW camera data and had to use the HD-SDI. Obviously that's a BIG disadvantage to test a RAW camera with only the HD-SDI image, so those results should be viewed with a kind of an asterisk - like Barry Bonds' batting record.

In the lowlight shadow detail the F3 was excellent. The RED MX was used as the reference image. Film was actually bad. As Robert Primes pointed out, film actually has phenomenal highlight detail, but isn't actually that good in the shadow detail. This test proved that point. The F3 looked good, as did the AF100 good. The DSLRs also all looked good.

The Ringer chart - by DSC I believe - showed the color sharpness and moir© issues. This was subjective, but the F35 was used as the reference and was nearly perfect looking. The Weisscam was bad the F3 was nice, the Panny was nice, the 5D was bad, the 1D was worse, the 7D was bad and the Nikon was decent.

Page 1 of 2 pages 1 2 Next »

Editor's Choice
PVC Exclusive
From our Sponsors

Share This

Back To Listings RSS Print

Get articles like this in your inbox: Sign Up

Comments

EmiAllDay: | April, 13, 2011

It says in the article that the DSLR’s did well at The dynamic range test .“The F3 and DSLRs also rated well.”
h.264 is highly compressed.How would the DSLR’s perform well at theses kinds of tests?

David Williams: | April, 14, 2011

H.264 has nothing to do with limiting the overall dynamic range. The sensor with the electronics/firmware limit that.

Black is 0% and white is 100%, so you have 100% of what’s available from the camera. The only limit H.264 places on the dynamic range offered by the camera is that it’s 8 bit, so you can only have 256 discrete steps of luminance between black and white.

Erich Ocean: | April, 20, 2011

H.264 is not “8-bit.” It’s also not a single standard, but a collection of “profiles”, and profiles all the way up to 14-bit RGB are defined (the Hi444PP profile) in the H.264 spec.

David Williams: | April, 20, 2011

Erich, the H.264 implementation in DSLRs is limited to 8 bits. That’s the context of what we are posting here. Good luck finding the Hi444PP setting in your 7D.

georgemanzanilla: | April, 20, 2011

I still prefer film.

David Williams: | April, 21, 2011

And non sequitur apparently.

Mike Curtis: | April, 22, 2011

As one of the people organizing the test, I’ve had interesting discussions with others involved about film - feel free to prefer it, but the days of a flat statement of “film is better” are on the wane. In various attributes, Alexa and Red MX (and F35 and F3 to lesser, specific intents) are valid alternatives for some kinds of shots. Overall, I’d be looking to Alexa or Red MX (or Epic when more widely available) as a film replacement.

-mike

Mike Curtis: | April, 22, 2011

Clarifications - if you held a gun to my head and said “pick a film replacement for this shoot” I’d say Alexa or Red MX or F35, depending on the project and shooting environment (I’m leaving budget out of the equation here).

To argue that “film is better” - OK, I’d say for what, in what context, for what utilization? I can top or match the numbers in ALMOST all cases now.

If you prefer the aesthetics of film, well, no arguing with an opinion of preference. If you have the budget, go for it.

-mike

Steve Hullfish: | August, 25, 2011

The 8-bit limit is the problem. That is what limits color correction. H.264 is called out as the limiting factor because there are cameras that can shoot 10-bit and beyond, but if they are recording in 8-bit H.264 then the increased color space is lost before it gets to the colorist. Also, H.264 is a limiting factor because often the compression artifacts become obvious when trying to stretch out an image or push it beyond where you might be able to push the image if it didn’t have the compression artifacts.

Vancouver Videographer: | November, 13, 2011

Regardless of the on-board codec, if a camera has a clean HDMI or HD-SDI output then you can record a higher bitrate version either uncompressed or with less lossy codec, like ProRes.  Unfortunately too many of the cameras still only output an 8 bit feed.  Although it is not on the list because it wasn’t available at the time of the test, I’m thinking specifically of the Sony NEX-FS100 but I guess when I think of that camera my biggest beef is the HDMI out instead of an HD-SDI out.

Please login or register to comment