Digital Negative

From Infogalactic: the planetary knowledge core
Jump to: navigation, search

<templatestyles src="Module:Hatnote/styles.css"></templatestyles>

Digital Negative (DNG)
DNG tm.svg
Filename extension .dng
Developed by Adobe Systems
Initial release September 27, 2004; 20 years ago (2004-09-27)
Latest release
1.4.0.0
(September 2012; 12 years ago (2012-09))
Type of format raw image format
Container for Metadata may be embedded in XMP, Exif or IPTC formats.
Extended from TIFF/EP
Open format? Yes
Website helpx.adobe.com/photoshop/digital-negative.html

Digital Negative (DNG) is a patented, open, non-free lossless raw image format written by Adobe used for digital photography. It was launched on September 27, 2004.[1] The launch was accompanied by the first version of the DNG specification,[2] plus various products, including a free-of-charge DNG converter utility. All Adobe photo manipulation software (such as Adobe Photoshop and Adobe Lightroom) released since the launch supports DNG.[3]

DNG is based on the TIFF/EP standard format, and mandates significant use of metadata. Use of the file format is royalty-free; Adobe has published a license allowing anyone to exploit DNG,[4] and has also stated that there are no known intellectual property encumbrances or license requirements for DNG.[5] Adobe stated that if there were a consensus that DNG should be controlled by a standards body, they were open to the idea.[6] Adobe has submitted DNG to ISO for incorporation into their revision of TIFF/EP.[7]

Rationale for DNG

Given the existence of other raw image formats, Adobe's creation of DNG as a competing format implies that DNG is unusual and satisfies objectives that other raw image formats do not. These objectives and the associated characteristics of DNG, as well as assessments of whether these objectives are met, are described below. Increasingly, professional archivists and conservationists, working for respectable organizations, variously suggest or recommend DNG for archival purposes.[8][9][10][11][12][13][14][15][16]

Objectives

These objectives are repeatedly emphasized in Adobe documents:[1][6][17][18]

  • Digital image preservation (sometimes known as "archiving"): to be suitable for the purpose of preserving digital images as an authentic resource for future generations.[19] Assessment: The US Library of Congress states that DNG is a recommended alternative to other raw image formats: "Less desirable file formats: RAW; Suggested alternatives: DNG".[20] The Digital Photography Best Practices and Workflow (dpBestflow) project, funded by the United States Library of Congress and run by the American Society of Media Photographers (ASMP), singles out DNG, and states "DNG files have proven to be significantly more useful than the proprietary raw files in our workflow".[21]
  • Easy and/or comprehensive exploitation by software developers: to enable software to be developed without the need for reverse engineering; and to avoid the need for frequent software upgrades and re-releases to cater for new cameras. Assessment: Software could support raw formats without DNG, by using reverse engineering and/or dcraw; DNG makes it easier, and many software products can handle, via DNG, images from cameras that they have no specific knowledge of.[22] An unresolved restriction is that any edit/development settings stored in the DNG file by a software product are unlikely to be recognized by a product from a different company. (This problem is not specific to DNG).
  • In-camera use by camera manufacturers: to be suitable for many camera manufacturers to use as a native or optional raw image format in many cameras. Assessment: About 12 camera manufacturers have used DNG in-camera. About 38 camera models have used DNG.[23] Raw image formats for more than 230 camera models can be converted to DNG.[24]
  • Multi-vendor interoperability: to be suitable for workflows where different hardware and software components share raw image files and/or transmit and receive them.

Characteristics

All of the above objectives are facilitated or enabled by most of these characteristics:[25]

  • Freely-available specification:[26] this can be downloaded from the Adobe website without negotiation or needing justification.[2]
  • Format based on open specifications and/or standards: DNG is compatible with TIFF/EP, and various open formats and/or standards are used, including Exif metadata, XMP metadata, IPTC metadata, CIE XYZ coordinates and JPEG.[2][27]
  • Self-contained file format: a DNG file contains the data (raw image data and metadata) needed to render an image without needing additional knowledge of the characteristics of the camera.[2][22]
  • Version control scheme: it has a version scheme built into it that allows the DNG specification, DNG writers, and DNG readers, to evolve at their own paces.[28][29]
  • Freely-available source-code-based software development kit (SDK): there are three aspects - there is an SDK; it is source-code-based (as can be verified by examination); and it can be downloaded from the Adobe website without needing justification.[30]
  • Documented to have no known intellectual property encumbrances or license requirements: there is both a "Digital Negative (DNG) Specification Patent License" which says that anyone can exploit DNG,[4] and a statement that there are no known intellectual property encumbrances or license requirements for DNG.[5]
  • Lossless and lossy compression (optional): DNG support optional lossless and (since version 1.4) also lossy compression.[31] The lossy compressions losses are practically indistinguishable in real world images.[32]

Technical summary

A DNG file always contains data for one main image, plus metadata, and optionally contains at least one JPEG preview.[2] It normally has the extension "dng" or "DNG".

DNG conforms to TIFF/EP and is structured according to TIFF. DNG supports various formats of metadata, (including Exif metadata, XMP metadata, IPTC metadata), and specifies a set of mandated metadata.[28]

DNG is both a raw image format and a format that supports "non-raw", or partly processed, images.[2] The latter (non-raw) format is known as "Linear DNG".[33] Linear DNG is still scene-referred[34] and can still benefit from many of the operations typically performed by a raw converter, such as white balance, the application of a camera color profile, HDR compositing, etc. All images that can be supported as raw images can also be supported as Linear DNG. Images from the Foveon X3 sensor or similar, hence especially Sigma cameras, can only be supported as Linear DNG.

DNG can contain raw image data from sensors with various configurations of color filter array (CFA). These include: conventional Bayer filters, using three colors and rectangular pixels; four-color CFAs, for example the RGBE filter used in the Sony Cyber-shot DSC-F828; rectangular (non-square) pixels, for example as used in the Nikon D1X; and offset sensors (for example with octagonal pixels) such as Super CCD sensors of various types, as used in various Fujifilm cameras. (Or combinations of these if necessary). DNG specifies metadata describing these individual parameters; this is one significant extension to TIFF/EP.

When used in a CinemaDNG movie clip, each frame is encoded using the above DNG image format. The clip's image stream can then be stored in one of two formats: either as video essence using frame-based wrapping in an MXF file, or as a sequence of DNG files in a specified file directory.

Timeline

This provides a mixture of the dates of significant events (such as "the first X") and various counts of usage at the anniversaries of the launch (each 27 September).

Counts of products and companies that use DNG in some way are provided primarily for illustration. They are approximate, and include products that are no longer sold. The purpose is mainly to demonstrate that such products and companies exist, and to show trends. Convertible raw image formats (camera models whose raw images can be converted to DNG) only include official support by Adobe DNG converters; not unofficial support by Adobe products (sometimes reaching about 30), nor support by other DNG converters.[24]

  • 2003, late: Adobe started work on the DNG specification.[6]
  • 2004, early: Adobe started talking to other companies about DNG.[6]
  • Launch, 2004, 27 September: Adobe launched DNG. Specification version 1.0.0.0 published.[1] Convertible raw image formats: 60+
  • 2005, February: Specification version 1.1.0.0 published.
  • 2005, June: First digital back to write DNG, the Leica DMR (Digital Modul R) back for the R8 and R9.
  • 2005, July: First camera manufacturer to provide a DNG converter for its own raw file formats - Hasselblad's FlexColor.
  • 2005, July: First genuine digital SLR camera to write DNG, Hasselblad H2D.
  • 1st anniversary, 2005, 27 September: Camera manufacturers: 4. Camera models: 7. Software products: 70+. Convertible raw image formats: 70+.
  • 2005, October: First compact camera to write DNG, Ricoh GR Digital.
  • 2006, July: First monochrome digital back to write DNG, MegaVision E Series MonoChrome.
  • 2006, September: First rangefinder camera to write DNG, Leica M8.
  • 2006, September: First camera to offer the user a choice of proprietary raw or DNG, Pentax K10D.
  • 2nd anniversary, 2006, 27 September: Camera manufacturers: 8. Camera models: 9. Software products: 120+. Convertible raw image formats: 110+.
  • 2007, May: First raw converter & photo-editor whose first raw-handling release only supported DNG, Pixelmator.
  • 2007, July: First underwater camera to write DNG, Sea&Sea DX-1G. (Based on Ricoh Caplio GX100).
  • 3rd anniversary, 2007, 27 September: Camera manufacturers: 10. Camera models: 13. Software products: 170+. Convertible raw image formats: 160+.
  • 2007, October: First digital scan back system and first 360-degree panorama system to write DNG, Seitz 6x17 Digital and Seitz Roundshot D3 with D3 digital scan back.[35][36]
  • 2008, February: First software on a mobile phone to write DNG, Tea Vui Huang's "DNG Phone Camera" for Nokia.
  • 2008, April: Adobe announces CinemaDNG initiative, using DNG as the basis for the individual raw images of a movie.
  • 2008, May: Specification version 1.2.0.0 published.
  • 2008, September: First movie camera to use DNG as a raw image format, Ikonoskop A-cam dII.[37]
  • 4th anniversary, 2008, 27 September: Camera manufacturers: 13. Camera models: 29. Software products: 200+. Convertible raw image formats: 180+.
  • 2008, September: First DNG converter running on Linux (among several other things), digiKam.
  • 2009, spring/summer: First digiscope with built-in camera to write DNG, Zeiss Photoscope 85 T* FL.[38]
  • 2009, June: Specification version 1.3.0.0 published.
  • 5th anniversary, 2009, 27 September: Camera manufacturers: 14. Camera models: 38. Software products: 220+. Convertible raw image formats: 230+.
  • 2009, November: First "interchangeable unit" camera to write DNG, Ricoh GXR.[39]
  • 2010, February: First 3D movie camera to write DNG, Ikonoskop A-cam3D.[40]
  • 2010, March: First medium format camera to offer the user a choice of proprietary raw or DNG, Pentax 645D.
  • 6th anniversary, 2010, 27 September: Camera manufacturers: 14. Camera models: 47. Software products: 240+. Convertible raw image formats: 290+.
  • 2012, September: Specification version 1.4.0.0 published.

During the first 5 years when about 38 camera models were launched that wrote DNG, Adobe software added support for about 21 Canon models, about 20 Nikon models, and about 22 Olympus models.

Reception

The reaction to DNG has been mixed.[24] A few camera manufacturers stated their intention to use DNG at launch. The first supported DNG about 9 months after launch. Several more niche and minority camera manufacturers added support after this (e.g. Leica). The largest camera manufacturers have apparently never indicated an intention to use DNG (e.g. Nikon and Canon).[citation needed]

Some software products supported DNG within 5 months of launch, with many more following. Some only support DNG from cameras writing DNG, or from cameras supported via native raw image formats.[citation needed]

OpenRAW was an advocacy and lobby group with the motto "Digital Image Preservation Through Open Documentation". They became opposed to DNG.

Some photographic competitions do not accept converted files, and some do not accept DNG.[41]

DNG conversion

"DNG conversion" refers to the process of generating a DNG file from a non-DNG image. (This is in contrast to "raw conversion", which typically refers to reading and processing a raw file, which might be a DNG file, and generating some other type of output from it). DNG conversion is one of the sources of DNG files, the other being direct output from cameras and digital backs.

Several software products are able to do DNG conversion. The original such product is Adobe DNG Converter or DNG Converter, a freely available stand-alone utility from Adobe.[17] Other Adobe products such as the ACR plugin to Photoshop or Lightroom can also generate DNG files from other image files.

Most DNG converters are supplied by companies other than Adobe. For example:

  • The software that Pentax supplies with all their dSLR cameras can convert PEF raw image files from Pentax cameras into DNG files.
  • Flexcolor and Phocus from Hasselblad can convert 3FR raw image files from Hasselblad cameras and digital backs into DNG files.
  • Capture One from Phase One is a raw converter that can process not only raw image files from Phase One digital backs, but also raw image files from many other cameras too. Capture One can save images from many of those cameras to DNG.
  • KDE Image Plugin Interface is an API, that can save the images it is processing to DNG. It can be used standalone or with any image processing applications of the KDE desktop environment under Linux and Windows.
  • A number of DNG converters have been developed by "amateurs" to enable raw images from their favored camera or digital back to be processed in a large range of raw converters. These include cases where cameras have been hacked to output raw images that have then been converted to DNG.

The process of DNG conversion involves extracting raw image data from the source file and assembling it according to the DNG specification into the required TIFF format. This optionally involves compressing it. Metadata as defined in the DNG specification is also put into that TIFF assembly. Some of this metadata is based on the characteristics of the camera, and especially of its sensor. Other metadata may be image-dependent or camera-setting dependent. So a DNG converter must have knowledge of the camera model concerned, and be able to process the source raw image file including key metadata. Optionally a JPEG preview is obtained and added. Finally, all of this is written as a DNG file.

DNG conversion typically leaves the original raw image file intact. For safety, many photographers retain the original raw image file on one medium while using the DNG file on another, enabling them to recover from a range of hardware, software, and human, failures and errors. For example, it has been reported in user forums that some versions of the Adobe DNG Converter don't preserve all the raw data from raw images from some camera models.[42][43]

Summary of products that support DNG in some way

This section summarizes other more comprehensive lists.[44][45]

Adobe products

All raw image file handling products from Adobe now support DNG.[3] Adobe DNG Converter was published by Adobe Systems on September 27, 2004. It converts different camera raw format files into the Digital Negative (DNG) standard. It also supports lossless data compression when converting. The program is free of charge. It can be downloaded at Adobe's site (for Microsoft Windows[46] and the Apple Macintosh[47]).

Digital cameras and related software

Use by camera manufacturers varies; there are about 15 camera manufacturers that use DNG, including a few that specialize in movie cameras:[23]

  • Niche camera manufacturers typically use DNG in new cameras (including a digiscope, panorama cameras, and at least one movie camera). The article on raw image formats illustrates the complicated relationship between new raw image formats and third-party software developers. Using DNG provides immediate support for these cameras by a large range of software products.
  • High-end smartphone cameras using Android 5 or Windows Mobile that are capable to shoot RAW images typically use DNG (e.g. Samsung and Nokia).
  • Some low market share but conventional camera manufacturers use DNG in new cameras. Camera manufacturers that do not supply their own software for processing raw images typically, but not always, use DNG.[citation needed]
  • Pentax typically offers users the option of whether to use Pentax's own raw image format (PEF) or DNG, but some, for example Pentax Q, Q10 and Q7, do not support PEF. For example, the digital SLR Camera Pentax K-x does offer the ability to save PEF or DNG or even DNG+ which saves two files, a DNG and a separate JPEG file at the same time.
  • If a camera uses DNG, and that camera manufacturer supplies software, it will support DNG. It may support DNG only from their own cameras, or support it more generically.
  • Canon, Nikon, Sony, Panasonic, Olympus, Fuji, and Sigma do not use DNG in their cameras. If a camera manufacturer's cameras do not use DNG, their software is unlikely to support DNG unless that software is also sold independently of the cameras.

Some digital cameras that support DNG.:[23]

Some of the Canon cameras can shoot as DNG using additional free software CHDK.

Third-party software

Support by software suppliers varies; there are of the order of 200 software products that use DNG.[44][50]

The majority of raw handling software products support DNG. Most provide generic support, while a few support it only if it is output directly from a camera. The type of support varies considerably. There appear to be very few third party software products that process raw images but don't support DNG. This may reflect the difficulty of discovering all of those that do not.[51]

Versions of the specification

All versions of the specification remain valid, in the sense that DNG files conforming to old versions should still be read and processed by DNG readers capable of processing later versions. DNG has a version scheme built into it that allows the DNG specification, DNG writers, and DNG readers, to evolve at their own paces.[29] Each version of the specification describes its compatibility with previous versions.[2]

1.0.0.0, published September, 2004
This version accompanied the launch of DNG and related products. It was a rare, possibly unique, example of a raw image format specification published by its owner. It was adequate for representing typical images, but it had a few errors and deficiencies (specifically the lack of support of "masked pixels" and an inadvertent deviation from the JPEG specification) that required it soon to be replaced by the next version.
1.1.0.0, published February, 2005
This version corrected the flaws in the first version. It has proved capable of representing raw images for a large variety of cameras (both when written in-camera or via conversion from other raw image formats) for a few years, and it is the version still typically written in-camera.
1.2.0.0, published May 2008
This version was based on experience and feedback from other companies about DNG since its launch. It introduced many new features, especially several new options for color specification under the general heading of "Camera Profiles". These are mainly of value to software products wanting their own flavor of color handling. This version permits administrative control of Camera Profiles, including calibration signatures and copyright information.
1.3.0.0, published June, 2009
This version added various improvements, but the major change was to introduce "Opcodes". A Opcode is an algorithm to be applied to some or all of the image data, described in the specification, and (optionally) implemented in the product that reads and processes the DNG file. The DNG file itself holds lists of Opcodes to be executed, together with the parameters to be applied on execution. In effect, the DNG file can contain lists of "function calls" to be executed at various stages in the raw conversion process. For example, the WarpRectilinear Opcode "applies a warp to an image and can be used to correct geometric distortion and lateral (transverse) chromatic aberration for rectilinear lenses". This is an example of an algorithm that cannot be applied to the raw image data itself before it is placed into the DNG file, because it should be executed after demosaicing. There are 13 Opcodes described in this version, and each Opcode is accompanied by a specification version so that more can be added in future.
1.4.0.0, published Sept., 2012
This version added Floating Point Image Data, Transparent Pixels, Proxy DNG Files, and additional tags. It also added SampleFormat and Predictor.
CinemaDNG, published September 2009
CinemaDNG uses DNG for each frame of a movie clip. There are additional tags specifically for movies: TimeCodes and FrameRate.[52] It is not clear whether these tags will be added to a later version of the DNG specification, or will remain separately described only in the CinemaDNG specification.

Standardization

DNG is not (yet) a standard format, but is based on several open formats or standards and is being used by ISO in its revision of TIFF/EP. A timeline:

  • 2001: The ISO standard raw image format, ISO 12234-2, better known as TIFF/EP, was ratified and published. It also supports "non-raw", or "processed", images. TIFF/EP provided a basis for the raw image formats of a number of cameras, but they typically added their own proprietary data. Some cameras have sensors that cannot be described by that version of TIFF/EP.
  • 2004, September: Adobe launched DNG. Its specification states it "is compatible with the TIFF-EP standard".[2] It is a TIFF/EP extension with considerably more specified metadata, brought up-to-date and made fit for purpose. DNG also exploits various other open formats and standards, including Exif metadata, XMP metadata, IPTC metadata, CIE XYZ coordinates, ICC profiles, and JPEG.[27][2] Although DNG supports more sensor configurations than TIFF/EP (for example, cameras from Fujifilm using Super CCD sensors), it still doesn't support all sensor types as raw images, especially those using the Foveon X3 sensor or similar, hence especially Sigma cameras.
  • 2006: TIFF/EP began its 5-year revision cycle.
  • 2006 to 2007: Adobe offered the DNG specification to ISO to be part of ISO's revised TIFF/EP standard.[7][53]
  • 2008, September & October: Minutes of ISO/TC 130/WG2 — Prepress Data Exchange, 37th Meeting: "WG 18 is revising the two-part standard (ISO 12234), which addresses digital camera removable memory. The revision of... Part 2 will add DNG into TIFF/EP." A progress report from ISO about the revision of TIFF/EP stated that the revision "...currently includes two "interoperability-profiles," "IP 1" for processed image data, using ".TIF" extension, and "IP 2" for "raw" image data, ".DNG" extension".[54]
  • 2009, September: Minutes of ISO/TC 130/WG2 — Prepress Data Exchange, 39th Meeting: the revision of TIFF/EP "is comprehensive to support many different use cases, including backward compatibility with current TIFF readers and support of Adobe DNG... Profile 2 (proposed extension .dng, if Adobe is in agreement) is intended for camera raw images, including un-demosaiced images... This format will be similar to DNG 1.3, which serves as the starting point for development."[55]

See also

References

  1. 1.0 1.1 1.2 Lua error in package.lua at line 80: module 'strict' not found.
  2. 2.0 2.1 2.2 2.3 2.4 2.5 2.6 2.7 2.8 Lua error in package.lua at line 80: module 'strict' not found..
  3. 3.0 3.1 Lua error in package.lua at line 80: module 'strict' not found..
  4. 4.0 4.1 Lua error in package.lua at line 80: module 'strict' not found..
  5. 5.0 5.1 Lua error in package.lua at line 80: module 'strict' not found..
  6. 6.0 6.1 6.2 6.3 Lua error in package.lua at line 80: module 'strict' not found..
  7. 7.0 7.1 Lua error in package.lua at line 80: module 'strict' not found..
  8. universal photographic digital imaging guidelines (UPDIG): File formats - the raw file issue
  9. Archaeology Data Service / Digital Antiquity: Guides to Good Practice - Section 3 Archiving Raster Images - File Formats
  10. University of Connecticut: "Raw as Archival Still Image Format: A Consideration" by Michael J. Bennett and F. Barry Wheeler
  11. Inter-University Consortium for Political and Social Research: Obsolescence - File Formats and Software
  12. JISC Digital Media - Still Images: Choosing a File Format for Digital Still Images - File formats for master archive
  13. International Digital Enterprise Alliance, Digital Image Submission Criteria (DISC) Guidelines & Specifications 2007 (PDF)
  14. The J. Paul Getty Museum - Department of Photographs: Rapid Capture Backlog Project - Presentation
  15. American Institute for Conservation - Electronic Media Group: Digital Image File Formats
  16. Archives Association of British Columbia: Born Digital Photographs: Acquisition and Preservation Strategies (Rosaleen Hill)
  17. 17.0 17.1 Adobe: Digital Negative (DNG) - The public, archival format for digital camera raw data
  18. Adobe: Introducing the Digital Negative Specification: Information for manufacturers
  19. Planning for US Library of Congress Collections: Sustainability Factors
  20. Planning for US Library of Congress Collections: Preferences in Summary
  21. dpBestflow: Raw File Formats
  22. 22.0 22.1 Barry Pearson: Support via DNG but not native raws
  23. 23.0 23.1 23.2 Barry Pearson: Products from Camera Manufacturers that support DNG in some way.
  24. 24.0 24.1 24.2 Barry Pearson: A brief history of DNG
  25. Planning for US Library of Congress Collections: Adobe Digital Negative (DNG), Version 1.1
  26. The Luminous Landscape: The RAW Flaw by Michael Reichmann and Juergen Specht
  27. 27.0 27.1 Barry Pearson: DNG's relationship to standards
  28. 28.0 28.1 Adobe: DNG Specification (Section 4)
  29. 29.0 29.1 Barry Pearson: Version control in DNG files
  30. Adobe: Adobe DNG Software Development Kit (SDK)
  31. Adobe
  32. Lossy DNG comparison
  33. Barry Pearson: Linear DNG
  34. CIE: "Scene-referred": image state, scene-referred
  35. Seitz Phototechnik: Roundshot D3 with Seitz D3 digital scan back
  36. Seitz Phototechnik: Seitz 6x17 Digital with Seitz D3 digital scan back
  37. Ikonoskop: Ikonoskop A-cam dII
  38. Zeiss: PhotoScope 85 T* FL
  39. Ricoh: Ricoh GXR
  40. Ikonoskop: Ikonoskop A-cam3D
  41. Lua error in package.lua at line 80: module 'strict' not found.
  42. Adobe-hosted User Forum: Reconvert with new DNG Converter?
  43. DPReview Forum: PROGRESS: version 0.9.0.0 now supports PEF files
  44. 44.0 44.1 Adobe: DNG hardware and software support
  45. Barry Pearson: Products that support DNG in some way
  46. Lua error in package.lua at line 80: module 'strict' not found..
  47. Lua error in package.lua at line 80: module 'strict' not found..
  48. Lua error in package.lua at line 80: module 'strict' not found..
  49. Lua error in package.lua at line 80: module 'strict' not found..
  50. Barry Pearson: Software products that support DNG in some way
  51. Barry Pearson: Products without explicit DNG support
  52. Adobe Labs: CinemaDNG - Image Data Format Specification (Version 1.0.0.0) (PDF)
  53. Lua error in package.lua at line 80: module 'strict' not found..
  54. Lua error in package.lua at line 80: module 'strict' not found..
  55. Lua error in package.lua at line 80: module 'strict' not found..