RAW files... why didn't I do this sooner

Camera equipment and technique for taking photos.
Forum rules
  • This is a mountaineering forum, so please keep your posts on-topic. Posts do not all have to be related to the 14ers but should at least be mountaineering-related.
  • Personal attacks and confrontational behavior will result in removal from the forum at the discretion of the administrators.
  • Do not use this forum to advertise, sell photos or other products or promote a commercial website.
  • Posts will be removed at the discretion of the site administrator or moderator(s), including: Troll posts, posts pushing political views or religious beliefs, and posts with the purpose of instigating conflict within the forum.
For more details, please see the Terms of Use you agreed to when joining the forum.
User avatar
kaiman
Posts: 1367
Joined: 5/3/2006
Trip Reports (10)
 

Re: RAW files... why didn't I do this sooner

Post by kaiman »

thatmushroom wrote:Yes and no, with a whole lot more no.

Yes, vector graphics are resolution independent because they are mathematical objects.
No, most of those file formats listed are not vector formats.

RAW gets its name because it is essentially the raw sensor data from your camera. Before you can even view it as a picture, it needs to be interpreted. Each camera maker has their own program to convert it (which may not be apparent to the user), and there are lots of 3rd party programs as well. Lightroom hides the conversion very well, so it's not really something you have to ever think about.

RAW files or TIFF files are generally considered lossless formats, where every pixel gets a value, meaning that it does have the most information. JPEG is a lossy format, so pixels which are similar might get grouped together. The more compressed a picture is, the more information is discarded.

http://en.wikipedia.org/wiki/Image_file_formats" onclick="window.open(this.href);return false;
Um, I think we're arguing over semantics here. The file types I mentioned are indeed vectors and Adobe developed the formats I listed to include pixels so that raster images could be imported into them and then saved as vector files for further manipulation later.

My main point was that whenever you convert vector images to a raster format or save images as JPGs in general you are saving in a lossy format that can alter the quality of your image no matter which software you use to do it.
djkest wrote:So my understanding might be a little OFF, so correct me if I'm wrong.

With a RAW file, each pixel has it's own value. With a JPEG, it is compressed so clumps of pixels are grouped together. This really doesn't matter much if you don't edit your photos, but if you do, the difference is staggering.

With a JPEG file, if the photo is really overexposed or underexposed, you have a slim chance of changing it into a good shot, because a lot of the data from the photo is lost in the compression process. Similarly, I have noticed that Colorado skies really suffer with the big gradient in blue tone as the angle changes.

When I used to edit JPEG files, it was a delicate balance in lightroom because too much adjustment and the image became visibly pixelated or blocky.

With RAW files, my ceiling is much higher. I can adjust the sky for example about 3x as much without artifacting, I can bring under/overexposed pictures back much better, and the clarity is much better. I also am able to change the white balance better for indoor photos.

I shoot Raw + JPEG so I can put the JPEGs on facebook, and after I am done editing the RAW files I convert them to JPEG. Although I'm thinking about starting to save them as .PNG, since the quality is so much better.
Yes you understand the issue and are correct that saving images as PNGs is a good way to go (particularly for web) as that file format supports 16 bit alpha channel colors (rather than just 8 bit colors) which make the images better quality while maintaining a comparable file size and also supports transparency which JPG does not. The pixelation and blockiness of JPG images you mention in lightroom has to do exactly with the compression and lossy file format issues I mentioned in my previous post.
"I want to keep the mountains clean of racism, religion and politics. In the mountains this should play no role."

- Joe Stettner

"I haven't climbed Everest, skied to the poles, or sailed single-handed around the world. The goals I set out to accomplish aren't easily measured or quantified by world records or "firsts." The reasons I climb, and the climbs I do, are about more than distance or altitude, they are about breaking barriers within myself."

- Andy Kirkpatrick
User avatar
Broken Knee
Posts: 470
Joined: 8/5/2014
13ers: 28
Trip Reports (1)
 

Re: RAW files... why didn't I do this sooner

Post by Broken Knee »

djkest wrote:With a JPEG file, if the photo is really overexposed or underexposed, you have a slim chance of changing it into a good shot, because a lot of the data from the photo is lost in the compression process. Similarly, I have noticed that Colorado skies really suffer with the big gradient in blue tone as the angle changes.
A good DSLR sensor may have about 14 bits of brightness resolution, so if your jpeg is compressed to 8 bits, you've thrown away much information. BUT! If you overexposed the sensor, you are hosed either way. You can always bracket an image then use a compositing application such as that in Photoshop (maybe Lightroom too, can't remember) to "assemble" a 32 bit or 16 bit image. Note that many printers fail to achieve 8 bits, as do many monitors.
djkest wrote:When I used to edit JPEG files, it was a delicate balance in lightroom because too much adjustment and the image became visibly pixelated or blocky.
You were probably seeing a phenomenon known as "banding" where the compression took away intermediate shades, making color transitions obvious.
When life gets you down, climb!
pseudoghost
Posts: 97
Joined: 8/15/2010
14ers: 32 
13ers: 2
Trip Reports (0)
 

Re: RAW files... why didn't I do this sooner

Post by pseudoghost »


Um, I think we're arguing over semantics here. The file types I mentioned are indeed vectors and Adobe developed the formats I listed to include pixels so that raster images could be imported into them and then saved as vector files for further manipulation later.

My main point was that whenever you convert vector images to a raster format or save images as JPGs in general you are saving in a lossy format that can alter the quality of your image no matter which software you use to do it.

No. Most of those file formats are not vector formats. AI is the only one that might be a vector format. Vector formats define an image by describing the mathematical representations of objects in the image. Then when the image is viewed, the image is reconstructed on the fly by interpreting the mathematical representations and redrawing the image.

JPEG is not inherently lossy either. Most people choose to use lossy JPEG compression, but there are "lossless" versions that will reproduce the image with complete fidelity (but with a file much larger than the usual JPEG). Most people accept the tradeoff because you really don't notice JPEG artifacts at the compression levels that most people use except in limited situations. Most of the time, the primary factors causing image quality degradation are the sensor and the lens.
djkest wrote:So my understanding might be a little OFF, so correct me if I'm wrong.

With a RAW file, each pixel has it's own value. With a JPEG, it is compressed so clumps of pixels are grouped together. This really doesn't matter much if you don't edit your photos, but if you do, the difference is staggering.

With a JPEG file, if the photo is really overexposed or underexposed, you have a slim chance of changing it into a good shot, because a lot of the data from the photo is lost in the compression process. Similarly, I have noticed that Colorado skies really suffer with the big gradient in blue tone as the angle changes.

When I used to edit JPEG files, it was a delicate balance in lightroom because too much adjustment and the image became visibly pixelated or blocky.

With RAW files, my ceiling is much higher. I can adjust the sky for example about 3x as much without artifacting, I can bring under/overexposed pictures back much better, and the clarity is much better. I also am able to change the white balance better for indoor photos.

I shoot Raw + JPEG so I can put the JPEGs on facebook, and after I am done editing the RAW files I convert them to JPEG. Although I'm thinking about starting to save them as .PNG, since the quality is so much better.
That's 1/2 of the story. The other major difference between a RAW file and a JPEG is that a JPEG can only store the brightness of a particular color for a particular pixel in 8-bits. RAW files typically use 12-14 bits. Which means that you can represent 16,384 brightness levels (14-bits) for a single pixel in a RAW file, versus only 256 brightness levels in a JPEG file. When the camera processes the image to a JPEG, that extra information is discarded during the quantization step of JPEG compression, where the processor determines the dynamic range of the image from the histogram, and then chooses which brightness levels to use. Typically you try to use a consistent sampling of brightness levels. However, if you have a bunch of pixels that fall in between those (fewer) JPEG brightness levels, then they get rounded up or down depending on how the algorithm works. This causes washout in overexposed images, and can introduce artifacts into otherwise smooth gradients (because you simply cannot reproduce all of the colors in the gradient). RAW files on the other hand allow you to work with the raw brightness data, so that you can correct the image before the quantization step in order to recover pieces of the image that might have been otherwise lost. In any case when you go from RAW to JPEG at the end of your processing, you're still doing the same thing as the camera, but you're doing it more intelligently than the algorithm is.

The compression part of JPEG, wherein the image is divided up into groups that look similar and reduced in size, can introduce artifacts as well depending on the compression level. But you won't see JPEG compression artifacts on most cameras today, with a notable exception being low contrast situations, because they tend to set the defaults to value quality over compression size.

Yes you understand the issue and are correct that saving images as PNGs is a good way to go (particularly for web) as that file format supports 16 bit alpha channel colors (rather than just 8 bit colors) which make the images better quality while maintaining a comparable file size and also supports transparency which JPG does not. The pixelation and blockiness of JPG images you mention in lightroom has to do exactly with the compression and lossy file format issues I mentioned in my previous post.
JPEG is the web standard for a reason. Size is king on the web. Most of the time you're posting to Facebook you're not posting a full size image anyways, and even if you were Facebook downsamples the image immediately to save space. Transparency is highly overrated.
User avatar
Broken Knee
Posts: 470
Joined: 8/5/2014
13ers: 28
Trip Reports (1)
 

Re: RAW files... why didn't I do this sooner

Post by Broken Knee »

pseudoghost wrote:This causes washout in overexposed images, and can introduce artifacts into otherwise smooth gradients (because you simply cannot reproduce all of the colors in the gradient).
An overexposed image IS washed out, period, regardless of compression or format. Once you expose beyond the dynamic range of the sensor, you're toast.
When life gets you down, climb!
User avatar
jdorje
Posts: 1388
Joined: 6/16/2010
14ers: 12 
13ers: 27
Trip Reports (16)
 

Re: RAW files... why didn't I do this sooner

Post by jdorje »

I'm no photographer or digital photographer, but I've read a bit and tried to translate it into English for myself. It's pretty interesting.

The RAW file stores whatever data the camera sensor wants it to. The RAW format differs between different manufacturers, and can include whatever they think will be of help in assembling or improving the image. The camera sensor can be extremely sensitive; when Matt says a 16 bit depth he means 16 bits per channel for each of the (r)ed, (g)reen, (b)lue channels for a total of 48 bits and 28 billion colors per pixel. But again, this is specific to the manufacturer; if your camera maker comes up with more sensitivity or some more extra metadata they can just slap on a tag in the header part of the file and add whatever they want. This is probably why programs to process those images are so expensive.

By comparison, PNG files typically use an 8-bit channel per color, plus 8 bits of alpha (unnecessary for photographs), for 32 bits but actually only 16 million colors. Note that while PNG is a "lossless" format, you are definitely losing data in changing from 28 billion colors to 16 million. The PNG standard does include "true color" 16-bit support, but even then you're missing out on whatever other data the camera manufacturer wanted to add. Also while PNG is compressed losslessly, this might strain the computational power of your camera (honestly this is the only reason I can think of why PNG would be worse, since it also includes support for arbitrary proprietary data chunks).

JPEG is a clever and efficient but lossy compression format. To give a simple example, if the compression software sees a color gradient (as of increasing shadow along a mountain) it won't store each pixel of the gradient but will store the gradient itself. This is great for compression of photographs but terrible for editing; in many cases you can't even open the file to add anything to it without having to recompress and lose slightly more data with each recompression. There are some standards for lossless jpeg compression but this gives away the entire purpose of using the format.

GIF is somewhat similar to PNG, in that it's compressed losslessly but with a much smaller color selection. GIF is limited to 256 colors in its palette, so, obviously no good for photographs though it was very helpful in the creation of the internet and games back when everything had to fit on a floppy disk or get transferred over a modem.

Also worth mentioning is TIFF, an extremely old format. Tiff has a bit of compression, but not much; however it's very flexible in its bit depth and meta data. It's significant since most RAW formats are based on TIFF.

As an aside, raster and vector graphics do not mean at all what was said above. Raster graphics are an array of pixels, while vector graphics are a list of strokes needed to draw an image. Only raster is suitable for photographs.

Assuming you have a camera that supports large SD cards, you should certainly get one (.50 $/gb, or $64 for a 128 gb) and save in both jpeg and raw.
"I don't think about the past, and the future is a mystery. Only the present matters."
User avatar
cpb145
Posts: 75
Joined: 7/15/2014
14ers: 12 
13ers: 1
Trip Reports (0)
 

Re: RAW files... why didn't I do this sooner

Post by cpb145 »

Alright, so I am messing around with the pics I took yesterday on Elbert, and all the different RAW development options that my factory Olympus software allows. This has unfortunately brought up more questions than answers for me.

For instance, how is it possible for a converted RAW to be a larger file size than the RAW sensor data? Some of the formats are extremely larger than the RAW ORF file.

Why can't I just have a bit perfect conversion? If the RAW is 14.2mb, why does it seem impossible to convert it to a .xxx viewable file at 14.2mb? At the moment, it appears my choices are less or more than 14.2mb.

Lastly, I am loading some of the converted files into Google where I typically store my online albums. Some of the "large" files take quite a bit longer, however when I look at the file details when it is done, the sizes of the files are not matching up at all to what they are on my PC. Is Google messing with them too? And surprisingly, Google loaded the RAW ORF file without an issue, albeit the specs are off on this one as well.

Let me give you an example of a RAW file, and it's converted specs in different formats, what Google specs are once loaded, and hopefully someone can attempt to explain this in plain English to me. I will also attempt to load the Google images here for comparison.

RAW picture - 14.2mb - (In Google, only 586kb)
Converted to jpeg at the highest quality - 6.2mb (In Google, 6.2mb - no change)
Converted to exif-TIFF (I believe at 8 bits p/channel, not given an option to choose) - 45mb! (in Google, only 660kb)
Converted to BMP - also 45mb (In Google, only 757kb)
Converted to TIFF - (chosen at 16 bits/channel) - 91mb! (Google will not let me load this photo, says it is too large)

Here is the same image, in the formats that Google let me load...

Image
RAW ORF file

Image
JPEG

Image
EXIF-TIFF (8 bit?)

Image
BMP

Strangely enough, in my personal opinion at least, the last BMP looks to be the best out of them all, followed by the TIFF, a close 3rd the jpeg, and the actually RAW looks terrible. I say strangely, because it is supposedly only 757kb compared to the largest of the files, the jpeg at 6.2mb.

What is going on here?!
-Chris
User avatar
cpb145
Posts: 75
Joined: 7/15/2014
14ers: 12 
13ers: 1
Trip Reports (0)
 

Re: RAW files... why didn't I do this sooner

Post by cpb145 »

And to further muddy the waters, I looked at the 'View image details' option by right clicking the Google BMP file. The image properties I receive that way don't even match up with what Google says they are. So now the 45mb BMP, loaded into Google at only 757kb, and when the image is inspected by right clicking, it is only 141kb! ARGGGG!!!!!

And worse yet, when I do side by side comparison on my laptop between the jpeg, exif-tiff, tiff, and bmp, I can't even tell the difference at all. I mean not in the slightest. Begging me to ask the question, "What's the point"? And in case anyone is interested, when the image properties are inspected directly from my laptop via a right click, all the files say 24 bit depth except for the true TIFF which says 48bit (not the exif-TIFF).
-Chris
User avatar
djkest
Posts: 1420
Joined: 9/7/2009
14ers: 58 
13ers: 19
Trip Reports (44)
 
Contact:

Re: RAW files... why didn't I do this sooner

Post by djkest »

I think you are missing the point somewhat.

If I take a photo that is RAW + JPEG

And then I look at both photos on my computer- they look identical. As they should.

RAW doesn't make your photo look better on it's own. With other formats, there are conversions going on that may effect how it looks. The RAW file is still superior, in that it is more true to the original photo taken. But the biggest difference you will notice when you start editing your photos in something like Lightroom. It's not that the pixels are better in RAW, well, they are actually. But, I digress.
Life is a mountain, not a beach.
Exploring and Wine, my personal blog
User avatar
jdorje
Posts: 1388
Joined: 6/16/2010
14ers: 12 
13ers: 27
Trip Reports (16)
 

Re: RAW files... why didn't I do this sooner

Post by jdorje »

cpb, some of what you say is hard to explain. But there are a few things to keep in mind -

Jpeg is lossy compression that is supposed to work well on photographs. I wouldn't expect differences visual to the casual eye. This is especially true when you have converted your 8000x2000 picture into a 600x300. Look at the jpeg versus bitmap zoomed all the way in and you can probably see some differences.

Google is going to automatically compress some files to be sizes they like. Probably into jpeg.
"I don't think about the past, and the future is a mystery. Only the present matters."
User avatar
mattpayne11
Posts: 992
Joined: 5/9/2009
14ers: 58 
13ers: 111
Trip Reports (48)
 
Contact:

Re: RAW files... why didn't I do this sooner

Post by mattpayne11 »

cpb145 wrote:Begging me to ask the question, "What's the point"?
Unless you do heavy editing to the image, there is no point at all. A camera-generated .jpg will apply all of the settings you put in the camera's back-end (sharpening, contrast, saturation, white balance, etc) and most of these are irreversible in post-processing on the .jpg format. With a RAW file, I can totally change the white balance, saturation, etc. I can even bring out details from the shadows or bring down highlights - which is why cameras like the D800 are used by most pros - the dynamic range on the files is tremendous - allowing a great deal of flexibility in post.

To further demonstrate... I could take an under-exposed RAW file of the Milky Way - bring it into LR or PS and then bring out the details of the stars. This is impossible with a .jpg file - the data is gone!
djkest wrote:If I take a photo that is RAW + JPEG

And then I look at both photos on my computer- they look identical. As they should.

RAW doesn't make your photo look better on it's own. With other formats, there are conversions going on that may effect how it looks. The RAW file is still superior, in that it is more true to the original photo taken. But the biggest difference you will notice when you start editing your photos in something like Lightroom. It's not that the pixels are better in RAW, well, they are actually. But, I digress.
Yes and no - you're right about the RAW file needing tweaking =) -- your comment about the files looking the same in RAW vs. JPG is not accurate. The JPG version will already have algorythms applied to it by the camera. The RAW file will be a blank slate, often looking flat.

In my opinion, jpg still has a use if you don't plan to do much editing and if you are a journalist wanting to get the images out quickly. Otherwise, if you plan to tweak a lot and make images have your personal look/feel, shoot RAW and use LR/PS/Camera Raw to edit. It takes FOREVER though.
User avatar
Derby Ale
Posts: 275
Joined: 7/29/2013
14ers: 26 
13ers: 1
Trip Reports (0)
 

Re: RAW files... why didn't I do this sooner

Post by Derby Ale »

mattpayne11 wrote:Yes and no - you're right about the RAW file needing tweaking =) -- your comment about the files looking the same in RAW vs. JPG is not accurate. The JPG version will already have algorythms applied to it by the camera. The RAW file will be a blank slate, often looking flat.
Not sure if it's my camera settings or something the way I have photoshop set up, but my .jpg files are often significantly different than the .raw files. Or it could just be that I tend to play with exposure a lot and I'm seeing the inherent differences/benefits of shooting in .raw compared to the comparatively static .jpg
In my opinion, jpg still has a use if you don't plan to do much editing and if you are a journalist wanting to get the images out quickly. Otherwise, if you plan to tweak a lot and make images have your personal look/feel, shoot RAW and use LR/PS/Camera Raw to edit. It takes FOREVER though.
I do think .jpg is great for some things. One is being able to publish quickly. Another I find is akin to the old exercise of shooting film slides versus film negatives. Negatives proved so much more forgiving than slides so photographers would sometimes shoot slides as a way to really practice the discipline of exposure control. I think it's good practice for photographers to occasionally challenge themselves to shoot something in .jpg only (pet project, not something you'd be doing for a client) and really practice your exposure control in that medium as well. I do find that when my exposure and composition is well thought out, my editing time in camera raw and Photoshop are not too bad.
User avatar
mattpayne11
Posts: 992
Joined: 5/9/2009
14ers: 58 
13ers: 111
Trip Reports (48)
 
Contact:

Re: RAW files... why didn't I do this sooner

Post by mattpayne11 »

Derby Ale wrote: Another I find is akin to the old exercise of shooting film slides versus film negatives. Negatives proved so much more forgiving than slides so photographers would sometimes shoot slides as a way to really practice the discipline of exposure control. I think it's good practice for photographers to occasionally challenge themselves to shoot something in .jpg only (pet project, not something you'd be doing for a client) and really practice your exposure control in that medium as well. I do find that when my exposure and composition is well thought out, my editing time in camera raw and Photoshop are not too bad.
Totally! It is indeed good practice, and you're right, should not take too long. I'm talking more about doing trip reports though, with 100+ photos that all need editing. I think it has taken me up to 12 hours to edit and post a trip report before.
Post Reply