Sunday, October 11, 2015

More Raw File Examples

Indeed, Mr. Ken Rockwell might be right - if you know how to set up your DSLR or Mirrorless Camera well, you can get by with not using raw file processing, and shoot strictly JPG's. This is in support of his basic philosophy of "take more pictures... don't worry about fussing with them". But, for others who really like doing things with raw files, it's not "fussing", but rather, spending more time with your pictures, so that you will really discover things that are going on within a particular shot.

The following shows how with the raw file, I discovered the real underlying composition strength of one picture, and by spending time with it, found many different ways to enhance this particular composition. If all I had was the camera's JPG, I never would have discovered this, unless perhaps I had my camera set up for it ahead of time (which BTW would have been completely wrong when compared with my usual way of shooting). This was an extremely candid shot, taken through my car window as a big cement truck rolled by, so how could I possibly have known how to set my camera up ahead of time?

JPEG Straight Out of Camera (SOOC)
Not bad. For this shot, I used the camera's Auto Lighting Optimizer set to "Standard". I've never used this feature before now, because it's basically a Shadow Lifter, used in most Raw File Converters to reveal detail in shadows. It did a fair job - better than I expected. Along with other in-camera features, such as increased Sharpness and increased Saturation, it might indeed be possible to avoid shooting raw files.

But for this shot, I decided I wanted the opposite - don't care so much about the detail under the truck, but would rather emphasize the rear lights*, and have the rest of the picture darker. Keeping in mind the Auto Lighting optimizer only applies to the JPG output (not Raw), we're afforded the freedom to do as we please with lighting situations such as this in the super-wide latitude of the raw file.
* The red rear lights, along with the distant traffic light really make this composition - they happen to fall on, or very close to the grid lines. Placing emphasis on these lights while darkening the rest of the picture really gives the whole thing it's composition strength.
DxO Highlight Tone Priority
DxO Optics Pro gives many one-click exposure options, one of which is Highlight Tone Priority - 3 Levels of it in fact. I used it here^, so we notice the detail under the truck is lost in deep shadow (as it would be in the real world at sun-set). This is also available in-camera (Canon DSLR's), but it cannot be used at the same time as Auto Lighting Optimizer. In other words, in-camera, you cannot raise shadow detail and highlight detail at the same time... in fact, this would be High Dynamic Range (HDR) by definition, which CAN BE achieved in-camera JPG with one setting, which takes 3 shots in rapid succession, each at a different Exposure Value (EV) and combines them into one HDR image. It works very well if you're into that, but I always see HDR as a deviation from the real world so I tend to avoid it. See the HDR example below***: The above shot^ also reveals the excellent way in which DxO automatically optimizes the lens and camera - compare this shot with the SOOC JPG... it's subtle, but the absence of lens distortion makes anything processed with DxO software seem so much more relaxed, always with the feel that you can walk right into the scene. Remember, DxO Labs is the absolute master of this science.

Another nice trick in DxO Optics Pro is Camera Body simulation - the color profile for every camera body in the DxO database is available, so if you don't like the look of your own camera, you can make it look like you used another! The uber-expensive Leica M body is one of these, and it's known for having a deep, rich color profile, so by selecting it, I obtain a slightly darker picture, sacrificing enough shadow detail, while having the nice deep look of the Leica M camera, all the while strengthening the composition in the way I wanted to.

DxO Camera Body - Leica M
Photivo - Highlight Color Lift & Micro-Contrast
With Photivo, I darkened the shot down a bit, and lifted the highlight color values. But compared to the DxO processed shots, it's not quite enough for the red lights to add their strength to the composition. Secondly, Photivo does not do the DxO Lens Correction magic, so the truck doesn't pop out from the picture quite as nicely. This same "problem" exists with every raw file processor except DxO. Now, it's important to keep in mind that most raw developers do include lens correction ability - even Photoshop Elements and GIMP, which are not raw developers include "Lens Fun". But DxO has already done a vast number of modern lens / camera combo's under the most scientific conditions possible, and it's perfect. Unless you have the Optical Lab equipment like DxO has (which you don't) I don't see much sense in messing around with the optics in a picture, unless you want to deliberately create distortion, not eliminate it. To my eyes, this alone makes DxO superior to everything else, and equal to the automated lens correction now built into high-end cameras, such as Canon's EOS-xxD and xD series. But if you have DxO, you don't need this; you can go with a cheaper camera, like Canon's EOS Rebel series - even the oldest Digital Rebel body and kit lens from way back in 2002 are included in the DxO database.

Now, for the amazing Polarr software for Chromebook. It too includes manual lens correction (not the same as DxO per-packaged lens correction), along with everything else you'll need, and it's amazingly well thought out, and runs very quickly. To get it, you don't need to buy a Chromebook; you can simply use your existing computer (Windows, Mac or Linux), download the Google Chrome browser, set up a Google Account and download the Polarr App. Then if you do buy a Chromebook, everything you do within your Google Account will show up in your Chromebook, and vice-versa! Notice how I warmed the picture with Polarr by adjusting the White Balance below:

Chromebook Polarr - Warmer White Balance, Micro-Contrast & Reduced Exposure Value
DxO One Click HDR
This is not "proper" HDR made in-camera with the blending of 3 shots in rapid sequence at different exposures. In fact, as the truck was moving, I could not have done this. But with raw file processing, it can be simulated from a single shot: this is pretty close to what HDR looks like. Yes, all of the shadow detail is there, and if you look at the sky, you see clouds that were barely there in the brighter pictures. But this isn't real, folks! If this is the way your eyeballs see the world around you, then you're smoking something. I don't want to poop all over HDR because it has become a means of artistic expression through photography. But, for the realist photographer, or one who really wants to establish a mood, HDR is completely void of realism, magic and mood. It's just a mechanical thing (Hey... look what I did!). This picture when given the HDR treatment looses everything, especially the composition strength of the red tail-lights I had mentioned above.

Yes, I'm convinced that above all, photography is art, and to get better art that stands out from the mediocre, you have to learn how to "work it" in the "digital darkroom" - and that's raw file processing.

Friday, October 2, 2015

The Power of Raw Files

JPG From Camera

Raw File Converted to jpg With Photivo

I'm not going to tell you that the processed Raw File is "better" than the straight-out-of-camera ("SOOC") .JPG file. In photography, "better" is always in the eye of the beholder. Besides, as many of the pro's are always keen to point out, a .JPG can always be made to look better in the camera itself, without needing to fiddle around with Raw Files - a camera's image parameters for a .JPG can always be adjusted within limited latitudes for Sharpness, Brightness, Contrast, Colour Saturation, Hue, as well as Black and White, with it's own unique set of parameters. Also, in most newer cameras made since 2010, all kinds of Creative Filters are now available for selection within the camera when a photo is viewed on the back display on Playback. Even further, Fuji is even putting in film emulation (of it's own historic film types of course) that can be applied on Playback. Canon, on it's higher end DSLR's is adding in-camera Raw Processing on Playback. There are so many ways of handling a .JPG that a lot of photographers, even some Pro's, don't dare, or bother to venture into Raw File territory.

But whether you're shooting .JPG or Raw File, the secret's out - a .JPG is always an output ("export") adaptation of a Raw File. If you have a basic point 'n' shoot, or a long-zoom compact with no provision for a Raw File direct output, this may not be so obvious. However, a small but enthusiastic opensource software group, calling themselves CHDK, for "Canon Hack Development Kit", has managed to separate an actual Raw File output, among quite a few other nifty things, from Canon Powershot-series compact cameras that do not provide Raw output out of the box. The Raw File is simply a step in the camera's internal process, and it's always there - it's just that on cheaper cameras, they don't let you get at it, while on more expensive ones, they do. CHDK has proven this, by simply making the Raw File visible and down-loadable from the camera's USB output.

The problem I have with dealing with a Raw File in -camera, whether it's by setting the image parameters in a certain way (which only affects the .JPG by the way), or by processing Raw within the camera on Playback, is that on the camera's small screen, I can't really see what effect it's having on my picture. A small LCD now always looks great - this is especially noticeable on Smartphone cameras which make a shot look wonderful on the phone's screen, but on a normal computer screen, or when printed out, you see that Smartphones still don't make for good cameras.

Take the above two photos as an example - it's obvious that I have a preference for bright, punchy photos, and I use Raw Process Software on my full size computer screen to make the adjustments - on a small camera rear-screen, I wouldn't be able to tell if I've gone too bright, or oversaturated my colours, or how much detail I'm enhancing. So in my opinion, Raw File Processing is something intended for the big screen, not the small LCD.

At first, straight from the camera, a Raw File looks terrible - absolutely flat, almost murky, lifeless, and void of detail, UNLESS you open the file with a Raw Processing Software that applies a bit of it's magic - just enough to either save the file as-is, or to encourage you that it's not so bad - go further. Most Apps have a default that's applied when you open the Raw File that looks more appealing than normal camera JPG output.

Raw Files do have a huge advantage - they provide far more latitude for every known adjustment than can be had using conventional .JPG processing, either in-camera, or in-software. Raw Processing also provides many more parameters to adjust, and work in a 14-bit (or greater) colour depth, as opposed to conventional 8-bit used in .JPG. This means that if you brighten it up, plain old .JPG processing will only allow a couple of +EV, and that would be applied to the whole picture, with only a little allowance for lifting highlights from out of shadows, or raising a bit of detail out of the shadows. Compare this to Raw File Processing, which provides far greater latitude for all of the following:

Temperature - same as White Balance. Same choices available as those in a digital camera, including "As Shot" or "Auto (AWB)", plus manual fine-tuning, picking a white standard (eye-dropper mode)

Tint - A bit like Temperature, but less complex. You can use Tint to put a certain colour-mood over an entire picture.

Exposure - this is the part where Raw has far more data available to literally re-expose the shot - up to 5 EV )Stops), maybe more, which is applied to the whole picture if you really screwed up the exposure in camera, this is truly Raw to the rescue. For more creative exposure work, there are other parameters available in a Raw Processor, such as lifting detail from shadows while leaving everything else alone.

Contrast - again, many more variables are available in Raw than .JPG - basic Contrast is similar to adjusting this value on your TV or computer monitor, to help your eyes distinguish one area of detail from another.

Micro- Contrast - sometimes called "Clarity", this is Contrast applied over a very small range of Pixels, making a picture look more in focus, with greatly increased detail. It does this by making the bright side of edges get brighter and the dark side of edges get darker. I use this a lot, and you can see it in my processed shot above in the vastly increased detail. This is not available in ordinary .JPG Processing as far as I know, because with .JPG output, the detail required at the pixel level has already been removed.

Sharpness - is a lot like Micro-Contrast, but typically, various sharpening methods don't do the job as well as Micro-Contrast. Best known is "Unsharp Mask" (funny name, but it means that parts of the image that are not sharp, with a clear hard edge, are masked out). This is also available in .JPG software, and works to a degree to make an out of focus subject look more in-focus - but it doesn't work well enough to look natural. Photivo offers 5 other methods besides "Unsharp Mask", any of which might work better dependant on actual conditions.

Saturation - this is the degree of colour adjustment throughout the entire picture, ranging from "0", which is B&W, through to 100 which would be a ridiculously high coloured photo.

Vibrance - again, available only for Raw, this is like a "smart saturation", applying more or less colour value only where it's needed

Gamma - This can be seen as more or less haze in a photo - generally you'd want less haze, so a higher Gamma number would reduce haze and give more "pop" to details. Haze can be quite common in a .JPG, but no JPG software can effectively remove it as perfectly as the Gamma adjustment in a Raw Processor. Some JPG software "cheats" by using a strategy called "comb-filtering", which appears to eliminate haze, but it eliminates a lot of picture detail, hoping nobody will notice.

Highlights - Raw processing allows detail lost within washed out highlights to be at least partly restored, depending on how badly blown out the Highlights are. JPG allows for this also, but again, Raw Files have so much more data available, you may find that Highlight detail may not be lost at all - it's still there to be recovered.

Shadows - Raw processing allows detail lost within murky black shadows to be at least partly restored, depending on how badly under-exposed the picture is. JPG allows for this also, but again, Raw Files have so much more data available, you may find that Shadow detail may not be lost at all - it's still there to be recovered.

White Level - similar to Highlights, but typically works over a greater tonal range.

Black Level - similar to Shadows, but typically works over a greater tonal range.

Noise Reduction - This works in a similar fashion as NR built into your camera, along with White Balance, Exposure, Contrast and Sharpness, many of these functions behave very much like your camera, but with much more power and a wider range of control. Noise Reduction especially can be accomplished in many different ways in Raw Software, as opposed to the in-camera program, which typically offers only one way to reduce noise. In fact, Photivo for example has no less than 9 NR methods!

Curves and Histograms - These are graphical elements found in both .JPG and Raw File software which can be altered by dragging a mouse-cursor to make many of the changes named above (accomplished by dragging sliders or entering numeric values directly). Some software doesn't offer control sliders, so you have to work directly on the curves and histograms while watching the on-screen preview.

Presets and "Fun Filters" - Good Raw File software (and .JPG too) will offer many one-click preset options which will vary all of the above parameters at once to achieve a pre-defined effect. It will also allow you to create and save presets of your own. Film Emulation is my personal favourite kind of Preset - a single click can instantly transform your Raw File into a classic looking Kodachrome ASA-64 slide for example. Another personal favourite is that DxO optics Pro has Presets for dozens of different digital camera bodies, allowing you to transform a Raw File from a cheap Panasonic compact to look like a Leica M9, or any other camera on the list. Again, this is all done by varying any and all of the above parameters, and saving the resulting profile within the software.

These are the most common Raw File variables. If there is a downside, it might be that there is simply too much to deal with here. Ken Rockwelll seems to have a hate-on for Raw Processing, and what it seems to come down to is this very thing - too  much information. But aside from this, he makes so many points in this article that are just plain wrong. Raw processing computer software offers 10 times or more possibilities for adjustment, all of which work over a much wider range, because with a Raw File, all of the data about a picture is still there if you need it, and it will always be there if you want to do something else with the data to create something completely different at a later time. I think the trick is, to choose one Raw program, stick with it, learn all you can about how it works, and then create a style of your own by repeating the same moves. For example, you might want to always set your White Balance for "Daylight" no matter what, because back in the film days, all you could get were Daylight balanced film (or Tungsten with a special order). This could become part of your own personal style - no matter what the AWB did in-camera, you can change it to Daylight to match the built-in limitation that most colour films had to offer.

Finally, I'll provide a quick run-down of FREE! Raw File Software (well, except for one):

DxO Optics Pro combined with DxO Film pack - the only one that's not free, but it is really worth it, because DxO is the #1 company that does exhaustive lab testing of all digital camera bodies and lenses - they are the world authority on the subject, and they have built in perfect lens-to-body correction into their product, not to mention the colour profiles for most camera bodies on the market as I mentioned above. It's user interface is very simple, they have the best Noise Reduction in the business, they provide one-click exposure modes (Evaluative or Smart, Highlight priority and Centre-Weighted - just like in your camera). Film Emulation is bought separately, but is seamlessly integrated as a module when Optics Pro and Film Pack are both installed. Unfortunately, there's no Linux version.

Photivo - my second favourite. This is the one that deals with every shooting parametr possible - so it's like overkill at first... but nothing says you have to use all of them. It's super easy to build your own style with Photivo simply by picking parameters you grow to love and understand. Photivo also provides lens correction, but you have to work it yourself - unlike DxO which automatically detects which lenss-body combo you too the picture with, and then auto-corrects accordingly. If you have to do it yourself, as with Photivo, it's kind of pointless, because you don't have a shooting lab like DxO does - but it's there anyway - with Photivo, everything is there, and laid out in easy to understand functional groupings. Also, if you havve GIMP installed, clicking on the "Export to External Processor" button will cause the resulting jpg to open in GIMP for further work.Photivo is available for Windows and Linux absolutely free.

UFRaw - A hip-pocket sized Raw Processor that's made only for Linux as far as I know. If you try to open a Raw File with GIMP in Linux, UFRaw will open first, giving a comprehensive set of Raw cooking parameters. One reason I don't care for it is that it's set up for direct curve and histogram adjustment, with no adjustment sliders, although there are a handful of single-click options that do a pretty good job. I think the idea is to use the single-clicks, observe the effect these have on the curves, and then make small manual adjustments to the curves to get exactly where you want to go. When you click on the final Export button, that's when GIMP opens up for finishing touches in jpg.

RawTherapee - another comprehensive package available for Linux and Windows that is set up to do most things, although it doesn't have as many different ways of doing the same thing as you find in Photivo. It has a nice user interface, quite similar to DxO Optics Pro, with a Preview strip. The problem is, RT doesn't do a very good job - it results in strange colours and other image problems. It has some built in Presets that also look a bit off, although a Film Emulation plug-in is available that provides one-click film type selections that do look great - exactly the same as those provided in DxO Film pack. But unless you're willing to spend some time with it to figure out what I'm doing wrong, I wouldn't recommend RawTherapee.

Darktable - I've tried and tried, but I simply can't figure out the user interface for this package. Some call it a "Lightroom Clone", but I find it hard to believe that Lightroom could be this hard to use (I've never actually set eyes on Lightroom). If you are a Lightroom user, and Darktable really is a LR clone, and you want something to use in Linux, maybe this one's for you - but otherwise I don't recommend it.

Polarr - A really nice little program available for Chrome OS and now Windows-10 (Beta Testing right now). I've signed up to test the new Beta for Chrome. It is completely slider-orientated, has every adjustment I've listed above, has some built-in Instagram-like Presets, and generally does everything well. The downside of using the Chromebook version is if you have a low-end Chromebook, you don't have a very good screen to work on. There are high-end Chromebooks available, but then, what's the point? You buy a Chromebook to get the least expensive computer out there - available for less than $200. However, don't forget that Chrome OS is really just the Chrome browser, and once you sighn in to Gmail, you're into Chrome OS - on you're big computer, running it virtually over Linux, Windows or Mac. I have Chrome OS and Polarr both running right now over Ubuntu Linux. Recommended.

Silky-Pix - free if it comes bundled with your camera - Panasonic, and maybe Olympus? Not sure. I tried it once, found it very difficult and never tried it again. Not recommended.

Canon Digital Photo Professional - comes bundled with Canon cameras. If you run Windows and buy a Canon DSLR, this is probably the one you'll use. And it's OK, except it does not include all of the parameters I've listed above. In fact, it doesn't do a whole lot more than what you can set in-camera; it simply takes the in-camera application and puts it on your computer, and adds a couple of functional enhancements (like tethered shooting). This of course makes it very easy to use, but don't expect to be able to do much with it. Not recommended.

So, Raw is always a significant step upwards, no matter what Ken says. If you're stuck using .JPG's sure, you can still create a personal style, but, if you're like Ken, and in too much of a hurry to spend a little bit of time with your pictures, because you want to "take more pictures" (or review more cameras), that's fine. I'm of a different mind - not to just machine-gun off thousands of digital photos, but rather, to take far fewer pictures (pretending I'm still shooting film), make every shot count, and then use software to create art.