BMP — 没有压缩的童年

BMP — A Childhood Without Compression

把像素原样写进硬盘,这就够了。

Just write the pixels straight to disk; that's enough.

1980 年代末,Windows 需要一个不依赖任何压缩库、可以从显存里直接 dump 出来、又能直接 load 回去的位图容器。设计目标根本不是体积,而是"零依赖、零解码、零思考"。BMP 因此把当年显存的扫描方向、字节顺序、行对齐规则一并写进了文件头,并固定下来——三十多年后,这些 80 年代显存的影子仍然活在 .bmp 里。

In the late 1980s Windows needed a bitmap container that depended on no compression library, could be dumped straight from video memory, and loaded straight back in. The design goal was not file size; it was zero dependencies, zero decoding, zero thinking. BMP froze the scan direction, byte order, and row alignment of the era's video memory into the file header. Three decades later, those 1980s VRAM ghosts still live inside every .bmp.

技术内核

Technical core

BMP 由两段定长头组成:14 字节 BITMAPFILEHEADER(magic "BM" + 文件大小 + 像素数据偏移)和 40 字节 BITMAPINFOHEADER(宽、高、位深、压缩方式、调色板大小等)。像素阵列自下而上排列——origin 在左下角,这直接对应 80 年代 CRT 显存的扫描方向。颜色通道顺序是 BGR 而不是 RGB,同样源自当年 Windows 显存的字节排列;打开任何一个 .bmp,前三个像素字节读出来都是蓝绿红。每一行的字节数必须是 4 字节的整数倍——不足的尾部用 0 padding,目的是让 32 位 CPU 一次取一个像素时不需要做对齐计算。后期的 BMPv4 / BMPv5 加了 RLE-4 / RLE-8 行程编码、bitfield 自定义通道掩码、ICC profile、alpha 通道,但生态并没有真的跟进——大多数解码器只认那 40 字节头。

A BMP file is two fixed-size headers: a 14-byte BITMAPFILEHEADER (magic "BM", file size, pixel-data offset) and a 40-byte BITMAPINFOHEADER (width, height, bit depth, compression, palette size). The pixel array is stored bottom-up: origin in the lower-left corner, mirroring the scan direction of 1980s CRT VRAM. Channel order is BGR, not RGB — again copied from how Windows laid out video memory. Every row must be padded with zeros to a multiple of 4 bytes, so a 32-bit CPU can read one pixel per fetch with no alignment math. Later BMPv4 / BMPv5 added RLE-4 / RLE-8 run-length encoding, bitfield channel masks, ICC profiles, and a real alpha channel — but the ecosystem never caught up; most decoders still only recognise the original 40-byte info header.

GIF — 1987 与 LZW 专利往事

GIF — 1987 and the LZW Patent Saga

用 256 个颜色坚持了 39 年。

Held the line with 256 colors for 39 years.

1987 年是拨号上网的年代——一张 100 KB 图片要传整整一分钟。CompuServe 需要一种比 BMP 小得多、跨平台、还能拼成动图的格式。Wilhite 拿了刚发表不久的 LZW 字典压缩,加上 256 色调色板,做出了 GIF87a。两年后 89a 又补上透明色与动画扩展,从此一锤定音——并且谁也没想到,它会撑过 GeoCities、撑过宽带、撑过 Flash,最后被 Twitter 时代的"表情包"二次激活。

1987 was the dial-up era — a 100 KB image took a full minute to download. CompuServe needed something far smaller than BMP, cross-platform, and capable of stitching frames into a loop. Wilhite combined freshly published LZW dictionary compression with a 256-colour palette and shipped GIF87a. Two years later 89a added transparency and animation extensions, locking the format in. No one expected it to outlive GeoCities, broadband, and Flash — only to be re-ignited by the Twitter-era reaction-meme.

技术内核

Technical core

GIF 由四件事拼出来:① 调色板——一张全局调色板(GCT),最多 256 个 RGB888 entry,每帧也可以再覆盖一张局部调色板(LCT);② LZW 压缩——变长 9 至 12 bit 字典,字典随像素索引流动态扩展,字典满了就清空重来;③ 多帧 + disposal——每帧带一个 Graphic Control Extension,disposal method 决定下一帧绘制前如何处理当前帧(保留 / 还原背景 / 还原前一帧);④ 89a 扩展——透明色索引(让其中一个 palette 槽位变透明,所以 alpha 永远是 1 bit)、Comment / Plain Text Extension、以及最关键的 NETSCAPE2.0 Application Extension——后者带一个 16-bit 循环计数,从此 GIF 可以无限循环。这个扩展不是 GIF 标准的一部分,是 Netscape 1995 年自己加的——但今天每一张循环 GIF 都欠 Netscape 一个 credit。

GIF is four things stitched together. ① Palette — one global colour table (GCT) of up to 256 RGB888 entries, optionally overridden per frame by a local table (LCT). ② LZW compression — a variable-width 9-to-12-bit dictionary that grows with the pixel-index stream and resets when full. ③ Frames + disposal — each frame carries a Graphic Control Extension whose disposal method tells the decoder how to wipe the previous frame (keep / restore-background / restore-previous). ④ 89a extensions — a transparent-colour index (one palette slot becomes "transparent", which is why alpha is forever 1-bit), Comment and Plain-Text extensions, and the all-important NETSCAPE2.0 Application Extension that carries a 16-bit loop counter. That last one isn't part of any standard — Netscape just added it in 1995 — yet every looping GIF on Earth still owes Netscape a credit.

PNG — DEFLATE、scanline filter、与一场弑父

PNG — DEFLATE, Scanline Filters, and a Patricide

GIF 收钱的那天,自由工程师写了一个免费的弑父者。

The day GIF started charging, free engineers wrote its successor.

1995 年初 Unisys 开始执行 LZW 专利,所有 GIF 编码器都要付费——包括 CompuServe 自己。Usenet comp.graphics 上 Thomas Boutell 在两周内拉起一支 30 人志愿团队,目标四条:(a) 完全无专利;(b) 比 GIF 更小;(c) 真正的 alpha 通道,而不是 1 bit 透明色;(d) 16 bit/channel + ICC profile + gamma 校正,为下一个十年的设备做准备。九个月后 PNG 1.0 + zlib + DEFLATE 三个 RFC 同时发布——这是互联网历史上最快、最干净的一场技术弑父。

In early 1995 Unisys began enforcing its LZW patent: every GIF encoder, including CompuServe's own, now owed money. Within two weeks a thirty-person volunteer crew on Usenet's comp.graphics rallied around Thomas Boutell with four goals: (a) wholly patent-free; (b) smaller than GIF; (c) real alpha, not a single transparent palette slot; (d) 16 bit / channel plus ICC profiles and gamma, ready for the next decade of hardware. Nine months later PNG 1.0, zlib, and DEFLATE shipped as three simultaneous RFCs — possibly the cleanest, fastest patricide in internet history.

技术内核

Technical core

PNG 的五个支柱:① DEFLATE = LZ77 + Huffman——和 zip / gzip 完全同款,1996 年的 RFC 1951,从设计第一天起就保证无专利。② 5 种 scanline filter(None / Sub / Up / Average / Paeth):每行像素独立选最优 filter——filter 不压缩,而是把数据预测成残差,让后面的 DEFLATE 更容易找到重复。Paeth filter 用左、上、左上三像素做方向预测,在自然图像上几乎总赢。③ chunks 体系:IHDR / PLTE / IDAT / IEND 是必须的,后面可以追加 tRNS(调色板透明)、gAMA / cHRM / iCCP(色彩管理)、tEXt / iTXt(metadata)、acTL / fcTL(APNG 扩展)等等;每个 chunk 一个 CRC32 校验,未知 chunk 解码器必须安全跳过——因此 PNG 的扩展性几乎是无限的。④ 真 alpha:8 或 16 bit 的独立 alpha 通道,不再借调色板槽位伪装;PNG-32 = RGBA 8-bit。⑤ Adam7 interlace:7 趟扫描的渐进显示——20 年前在 56 K 调制解调器上极其有用,今天不再常用。

Five pillars hold PNG up. ① DEFLATE = LZ77 + Huffman — the exact stack used by zip and gzip, RFC 1951, patent-free by construction. ② Five scanline filters (None / Sub / Up / Average / Paeth): each row picks its best filter independently — the filter doesn't compress, it predicts residuals so DEFLATE can spot repetitions. Paeth, which predicts from the left, upper and upper-left neighbours, almost always wins on natural images. ③ The chunks system: IHDR / PLTE / IDAT / IEND are mandatory; everything else (tRNS for palette transparency, gAMA / cHRM / iCCP for colour management, tEXt / iTXt for metadata, acTL / fcTL for APNG, …) is optional and CRC-checked, and decoders are required to ignore chunks they don't recognise — so PNG can grow forever without breaking old readers. ④ Real alpha: an independent 8- or 16-bit alpha channel, no longer disguised as a palette slot. PNG-32 is plain RGBA 8-bit. ⑤ Adam7 interlace: a 7-pass progressive scan — invaluable on 56 K modems twenty years ago, mostly obsolete today.

$ oxipng -o6 in.png                  # brute-force re-pack, 5–30% smaller
$ pngcrush -reduce -brute in.png out.png   # classic, slower but still useful
$ convert in.png -strip out.png      # ImageMagick — drop metadata chunks
$ pngquant --quality=70-90 in.png    # lossy palette quantisation → PNG-8
$ zopflipng -m in.png out.png        # Google's zopfli, max DEFLATE compression

APNG — PNG 偷偷做了动图

APNG — PNG Secretly Grew Frames

PNG 工作组说不要,Mozilla 偷偷加了。

PNG WG said no. Mozilla shipped it anyway.

2004 年 Mozilla 想给 Firefox 加一类"加载中"的动态图标:GIF 只剩 256 色看起来很丑,而 PNG 工作组官方动图方案 MNG 又庞大复杂,几乎没人实现。两个 Mozilla 工程师 Stuart Parmenter 和 Vladimir Vukićević 干脆在 PNG 上塞了三个新 chunk:acTL(动画控制)、fcTL(每帧控制)、fdAT(帧数据)。提案发到 PNG 邮件列表,工作组明确拒绝接收——理由是"会破坏 PNG 简洁性"。Mozilla 不为所动,2008 年随 Firefox 3 直接发布;十年后 Apple、Google 跟进,2017 年 W3C 终于回头把它收编为标准。一个被官方拒绝的扩展,反过来被市场和标准追认。

In 2004 Mozilla wanted lightweight loading animations in Firefox: GIF's 256 colours looked ugly and the PNG working group's official MNG was so vast that almost no one implemented it. Two Mozilla engineers, Stuart Parmenter and Vladimir Vukićević, simply added three new chunks to PNG — acTL (animation control), fcTL (per-frame control), fdAT (frame data). They sent the proposal to the PNG mailing list; the working group flatly refused, citing damage to "PNG's simplicity". Mozilla shipped it anyway in Firefox 3 (2008). A decade later Apple and Google followed, and in 2017 the W3C finally adopted APNG as a standard. A rejected extension, ratified later by the market and the spec.

技术内核

Technical core

APNG 在 PNG 上做的改动只有三件事:① 三个新 chunk——acTL 携带帧数和循环次数;fcTL 是每帧控制块,描述偏移、宽高、显示时长(分子/分母两个 16-bit)、blend mode 与 disposal mode;fdAT 本质是带 4 字节 sequence 编号的 IDAT,数据段格式完全相同。② 第一帧仍是合法 PNG——把首帧仍写成 IDAT,意味着不支持 APNG 的解码器(早期 Safari、ImageMagick 旧版本)看到的就是一张静态图,向后兼容性极佳。这是 APNG 比 MNG 成功的最大原因之一。③ blend / disposal 模式——blend mode 有 SOURCE(直接覆盖)与 OVER(alpha 合成)两种;disposal mode 有 NONE(保留)、BACKGROUND(清空)、PREVIOUS(还原前一帧)三种,跟 GIF 89a 的 disposal 完全是同一套语义。除此之外,APNG 对色彩空间、滤波、压缩(DEFLATE)的处理与 PNG 一字不差。

APNG only adds three things to PNG. ① Three new chunks: acTL carries frame count and loop count; fcTL is a per-frame control block describing offset, width/height, delay (a 16-bit numerator and denominator), blend mode and disposal mode; fdAT is essentially an IDAT prefixed with a 4-byte sequence number — its data payload format is identical. ② The first frame is still a valid PNG: keeping frame 0 as an IDAT means a decoder that doesn't understand APNG (old Safari, older ImageMagick) just sees a static image. This backward-compatibility trick is the biggest reason APNG won where MNG failed. ③ Blend / disposal modes: blend modes are SOURCE (overwrite) and OVER (alpha composite); disposal modes are NONE (keep), BACKGROUND (clear), PREVIOUS (restore prior frame) — exact same semantics as GIF 89a. For everything else (colour space, filters, DEFLATE), APNG inherits PNG byte for byte.

animated WebP — WebP 的动图分身

animated WebP — WebP's Multi-Frame Twin

WebP 在容器里偷偷塞了多帧——比 PNG 优雅,比 GIF 漂亮。

WebP slipped multiple frames into one container — neater than PNG, prettier than GIF.

静态 WebP 已经在体积上把 GIF 摁在地上摩擦——同一张表情包,WebP 通常只要 GIF 的 1/3。Google 接下来要做的事很自然:把 RIFF 容器从"装一帧"扩展成"装多帧",加一个 VP8X 扩展头声明 alpha / animation / ICC 等 feature flags,再加一个 ANIM 全局动画块和若干 ANMF 帧块。这套扩展在 2012 年附近随 libwebp 0.2 公开,WebP 一夜之间从静图格式变成"比 GIF 小 30%、画质好一个数量级、还有真 alpha"的动图格式。今天 Telegram、WhatsApp 的"高级表情包"几乎都是 animated WebP。

Static WebP already crushed GIF on bytes — the same sticker is typically a third the size in WebP. The next step was obvious: extend the RIFF container from one frame to many. Google added a VP8X extended header to declare feature flags (alpha / animation / ICC), an ANIM global animation block, and a stream of ANMF per-frame blocks. The extension landed around 2012 with libwebp 0.2, and overnight WebP went from a still-image format to one that beats GIF by ~30 % in size, an order of magnitude in quality, and finally adds real alpha. Today's "premium" stickers on Telegram and WhatsApp are almost all animated WebP.

技术内核

Technical core

animated WebP 的扩展逻辑可以用三句话概括:① RIFF 容器 + VP8X 扩展头——RIFF 是 Microsoft 1991 年发明的"分块容器"标准(用过 .wav / .avi 都见过它),WebP 直接复用,VP8X 是 WebP 自己加的扩展头,8 字节里第一字节是 feature flags(ICC profile / alpha / EXIF / XMP / animation 各占一位),其余字节给出 24-bit 的画布宽高。② ANIM + ANMF——ANIM 块在文件级别声明背景色和循环次数,ANMF 块在帧级别给出偏移、宽高、duration、blend mode、disposal mode,跟 APNG / GIF 是同一套语义。③ 每帧可独立选编码——WebP 内置两套编码器 VP8(有损,基于运动补偿和 DCT)与 VP8L(无损,基于 LZ77 + 颜色变换 + Huffman),animated WebP 允许逐帧切换:贴纸的纯色背景一帧用 VP8L 无损,人物动画一帧用 VP8 有损,同一文件里混排。

Three sentences cover the entire mechanism. ① RIFF + VP8X header: RIFF is Microsoft's 1991 chunk container (anyone who's opened a .wav or .avi has met it). WebP reuses it verbatim and adds an 8-byte VP8X header — the first byte is a bitfield of feature flags (ICC profile / alpha / EXIF / XMP / animation), and the remainder encodes a 24-bit canvas width and height. ② ANIM + ANMF: ANIM sits at file scope and declares background colour plus loop count; each ANMF then carries per-frame offset, dimensions, duration, blend mode and disposal mode — exact same semantics as APNG and GIF. ③ Per-frame codec choice: WebP ships two encoders, VP8 (lossy, motion-compensation + DCT) and VP8L (lossless, LZ77 + colour transform + Huffman). An animated WebP can switch encoders frame by frame — a sticker's flat background uses lossless VP8L, the character animation uses lossy VP8, all in one file.

JPEG — 8×8 DCT 三十年统治

JPEG — Three Decades of the 8×8 DCT

8×8 的格子,装下了三十年的人类视觉。

An 8×8 grid that held three decades of human vision.

1980 年代末,扫描仪、数码相机、传真机几乎同时崛起,所有人都需要一个能把"自然图像"压到 1/10 体积、人眼又看不太出来的标准。JPEG 委员会用三个事实搭了一条压缩流水线:人眼对亮度比对色度敏感、对低频比对高频敏感、对能量集中的信号(自然图像)有极强冗余可挖。把这三件事翻译成代码,就是 YCbCr + 4:2:0 + 8×8 DCT + 量化——JPEG 因此能在 Q85 这个挡位上把 5 MB 的照片压到 250 KB,而你几乎看不出差。

By the late 1980s scanners, digital cameras and fax machines were arriving in parallel — everyone needed a way to crunch "natural images" to a tenth of their size while the human eye barely noticed. The JPEG committee built a pipeline around three facts: the eye is more sensitive to luma than to chroma, more sensitive to low frequencies than to high, and natural images carry enormous redundancy in their energy distribution. Translated into code, that becomes YCbCr + 4:2:0 + 8×8 DCT + quantisation — and lets JPEG turn a 5 MB photo into 250 KB at Q85 with practically no visible loss.

技术内核

Technical core

JPEG 的压缩流水线有六环:① RGB → YCbCr——分离亮度与色度,为后续区别对待打基础。② 4:2:0 子采样——色度面 Cb / Cr 在水平和垂直方向各降一半采样率,体积立刻砍掉 50%,人眼几乎无感。③ 切成 8×8 块,每块做 DCT-II——空域换频域,自然图像的能量集中在左上(低频),右下大量系数趋近 0。④ 量化表——亮度表 + 色度表(色度表更激进),把不重要的高频系数除以一个较大的整数后取整,大量系数被清零。这一步是 JPEG 唯一的有损步骤,所有"画质损失"都在此发生。⑤ zig-zag 扫描 + RLE + Huffman——把 64 个系数按"之"字形排成一维序列,尾部一长串零交给 RLE 压缩,剩下的字面量做 Huffman 熵编码,无损。⑥ JFIF / Exif 容器封装——JPEG 标准本身只规定 codec 流(SOI / APPn / DQT / DHT / SOF / SOS / EOI marker),文件格式是另一层:JFIF 1.02(1992)规定了标准的 APP0 元数据,Exif(1995)在 APP1 里塞相机参数。今天你看到的每张 .jpg 几乎都是 "JFIF + Exif 包了一段 JPEG codec 流"。

Six stages make up the JPEG pipeline. ① RGB → YCbCr: split luma from chroma so the rest of the pipeline can treat them differently. ② 4:2:0 chroma subsampling: halve Cb and Cr horizontally and vertically — instantly drops 50 % of the data with virtually no perceptual cost. ③ Split into 8×8 blocks; run DCT-II per block: spatial → frequency domain. Natural-image energy clusters in the top-left (low frequency); the bottom-right is mostly near-zero. ④ Quantisation tables (luma + chroma, chroma being more aggressive): each coefficient is divided by the matching integer and rounded, killing huge swathes of high-frequency information. This is the only lossy step — every visible artefact JPEG ever produces comes from here. ⑤ Zig-zag scan + RLE + Huffman: unroll the 64 coefficients into a 1-D stream so the long zero-tail compresses cleanly under RLE, then Huffman-encode the remaining literals. Lossless. ⑥ JFIF / Exif container: the JPEG spec only defines the codec stream (SOI / APPn / DQT / DHT / SOF / SOS / EOI markers); the file format is a separate layer. JFIF 1.02 (1992) standardised an APP0 metadata segment, Exif (1995) tucked camera metadata into APP1. Almost every .jpg you've ever seen is "JFIF + Exif wrapping a JPEG codec stream".

$ cjpeg -quality 85 -optimize -progressive in.ppm > out.jpg   # reference libjpeg encoder, progressive scan
$ jpegoptim --max=85 --strip-all in.jpg                       # cap quality at 85, drop all metadata in place
$ mozjpeg cjpeg -quality 85 in.png > out.jpg                  # Mozilla's encoder — 5–10% smaller at same Q
$ exiftool -all= -overwrite_original in.jpg                   # nuke all Exif / GPS / thumbnail metadata

JPEG-LS — 你没听说过的无损 JPEG

JPEG-LS — The Lossless JPEG You Never Heard Of

比 PNG 快 3 倍,但因为不带容器和颜色管理,谁都没记住它。

3× faster than PNG, but no container, no colour management — and no one remembered.

1990 年代中期,医学影像的需求摆在桌面上:CT 和 MRI 一帧就是 12-bit 灰度大图,一次扫描几百帧——必须无损,但要比 PNG 简单、要比 JPEG 的 lossless mode 实用。HP Labs 的 Marcelo Weinberger 团队拿出了 LOCO-I(LOw COmplexity LOssless COmpression for Images)算法:用三像素中位数预测器(MED)估算下一个像素,把残差送进 Golomb-Rice 编码,在长平滑区域切到 RLE。1997 年 ISO/IEC 14495-1 正式发布,实测比 PNG 压缩率略好、解码速度快 3-5 倍——但它没有自带容器(只是裸 bitstream + 极简 marker),没有 ICC profile,没有元数据,没有 alpha,Web 浏览器全部当它不存在。最后只有医学影像活到了今天:DICOM 把 JPEG-LS 当成它的标准内嵌编码之一。

By the mid-1990s the medical-imaging world had a clear ask: CT and MRI frames were 12-bit greyscale, hundreds of slices per scan, and they had to be lossless — but simpler than PNG and more usable than JPEG's lossless mode. Marcelo Weinberger and team at HP Labs produced LOCO-I (LOw COmplexity LOssless COmpression for Images): a median-of-three predictor (MED) estimates each pixel, the residual goes into Golomb-Rice coding, and long flat runs switch to RLE. ISO/IEC 14495-1 shipped in 1997, beating PNG slightly on ratio and decoding 3–5× faster — but JPEG-LS arrived as a bare bitstream with minimal markers: no container, no ICC profile, no metadata, no alpha. Browsers ignored it entirely. Only medical imaging kept it alive — DICOM still embeds JPEG-LS as one of its standard encodings.

技术内核

Technical core

JPEG-LS 的精彩之处在于"用三件极简的事打败了 PNG"。① MED 预测器——只用左、上、左上三个像素就能判断当前像素位置在水平边、垂直边还是平坦区:c ≥ max(a,b) 说明这是从右上往下的边,预测取 min(a,b);c ≤ min(a,b) 反向,取 max(a,b);否则就是平坦区,预测取 a+b-c(即沿梯度延伸)。预测准了,残差就接近 0。② Golomb-Rice 熵编码——预测残差大致服从拉普拉斯/几何分布,Golomb-Rice 是这种分布的最优前缀码:把残差除以 2^k 拆成商和余数,商用 unary 码(若干个 1 加一个 0),余数用 k 位定长。参数 k 在编码过程中根据上下文自适应,完全跳过了 Huffman 表的构造,无需多遍扫描。③ Run-length mode——当解码器探测到连续多个像素被相同的上下文预测、且残差全部为 0 时,自动切换到 RLE 模式直接编码 run length——这是它在医学灰度图(大量黑底)和文档扫描上完胜 PNG 的关键。整个 codec 没有 DCT、没有变换、没有量化(无损模式),数学上接近"算术替代变换"。

JPEG-LS beats PNG with three tiny ideas. ① MED predictor: just three neighbouring pixels (left, above, top-left) decide whether the current pixel sits on a horizontal edge, vertical edge or smooth surface. c ≥ max(a,b) picks min(a,b); c ≤ min(a,b) picks max(a,b); otherwise a + b − c (planar extrapolation). When the predictor is right, the residual is near zero. ② Golomb-Rice entropy coding: residuals roughly follow a Laplacian / geometric distribution, and Golomb-Rice is the optimal prefix code for it — divide the residual by 2^k, encode the quotient in unary (k ones plus a terminating zero) and the remainder in k bits flat. The parameter k adapts per context during encoding, so there's no Huffman table to construct and no extra pass over the data. ③ Run-length mode: when the decoder sees consecutive pixels predicted by the same context with zero residuals, it switches to RLE and encodes the run length directly — the move that destroys PNG on medical greyscales (mostly black background) and document scans. The whole codec has no DCT, no transform, no quantisation (in lossless mode); it's almost pure arithmetic replacing transforms.

JPEG 2000 — 小波变换的悲壮失败

JPEG 2000 — The Tragic Defeat of the Wavelet

技术比 JPEG 强,专利让它寸步难行。

Technically beats JPEG. Patents tied its feet.

90 年代末 JPEG 的痛点是清楚的:8×8 块边界肉眼可见、没有 alpha、压缩等级单一、metadata 设计落后。JPEG 工作组想用一个全新算法一次性解决——结果就是 JPEG 2000:用整图 DWT(离散小波变换) 取代 8×8 DCT,无块边界、天然多分辨率;用 EBCOT(Embedded Block Coding with Optimized Truncation) 做编码,可以按"质量层、分辨率层、组件层、空间区域"任意子集解码——同一个 .jp2 文件,你可以只取低分辨率缩略,也可以只取一个画面区域。技术上完胜 JPEG。但有两件事压垮了它:① 解码复杂度比 JPEG 高 10 倍以上,移动设备根本跑不动;② 标准里嵌了几十项专利(虽然多数 RAND 免费),浏览器厂商出于法律风险拒绝实现。Mozilla 和 Google 多次明确说"不"。最后只在三个不在意延迟和算力的领域活下来:数字影院(DCI 强制使用)、卫星图像、医学影像。Safari 是唯一原生支持的浏览器——这是 Apple ImageIO 框架顺带带的,Apple 自己也不主推。

JPEG's 1990s pain points were obvious: visible 8×8 block boundaries, no alpha, only one compression curve, dated metadata. The JPEG WG tried to fix all of it with a clean-sheet algorithm — JPEG 2000. Replace the 8×8 DCT with a whole-image discrete wavelet transform (no block edges, naturally multi-resolution). Replace the entropy coder with EBCOT (Embedded Block Coding with Optimised Truncation), which lets a decoder grab any subset of quality / resolution / component / region from the same .jp2 file — pull just a thumbnail, or just one ROI. Technically it crushes JPEG. Two things broke it. ① Decoding cost is 10× JPEG or more — mobile silicon could not keep up. ② The standard sits on dozens of patents (most RAND-free, but the legal cloud was real), and browser vendors refused to implement it. Mozilla and Google both said no on record. JPEG 2000 survived only in three latency-insensitive, compute-rich worlds: digital cinema (DCI mandates it), satellite imagery, and medical imaging. Safari is the only browser that ships native support — and even that came along for free with Apple's ImageIO framework. Apple never promoted it.

技术内核

Technical core

JPEG 2000 的技术结构有四件事值得记住。① DWT 小波变换——替代 DCT。无损模式用 5/3 整数小波(可逆),有损模式用 9/7 浮点小波(更高效)。整图变换没有 8×8 块边界,所以彻底消除了 JPEG 那种"打格子"的 artifact;同时小波天然多分辨率——见上图。② tile + code-block + EBCOT 三层切分——大图先按 tile(典型 256×256 或 1024×1024)分块独立处理,tile 内部 DWT 后的每个子带再切成 code-block(典型 64×64),EBCOT 对每个 code-block 做位平面编码 + 算术编码,最后 R-D 优化决定哪些位平面截断。③ quality / resolution / component / position progression——同一个 .jp2 文件可以按四种顺序组织码流,解码器拿到任意前缀就能解出对应的一份"低质量但完整"或"高质量但单一分辨率"或"单一区域"的图像。这是 IIIF(图书馆/博物馆高分辨率扫描)的核心能力。④ 同算法既无损又有损——只通过量化步长切换,不像 JPEG / JPEG-LS 是两个独立标准。

Four pieces are worth remembering. ① Discrete wavelet transform replaces the DCT. Lossless mode uses the reversible 5/3 integer wavelet; lossy mode uses the 9/7 floating-point wavelet (higher efficiency). Whole-image transform = no 8×8 block edges = no JPEG-style tiling artefacts. The wavelet is also naturally multi-resolution (see figure above). ② tile + code-block + EBCOT three-level partitioning. Large images are first split into tiles (typically 256×256 or 1024×1024), each tile is wavelet-transformed, each subband is split into code-blocks (typically 64×64), and EBCOT bit-plane codes each block with arithmetic coding before R-D optimisation decides which bit-planes to truncate. ③ Quality / resolution / component / position progression: a single .jp2 can order its codestream four different ways, and any prefix the decoder receives yields either a "low-quality but complete" or "high-quality but single-resolution" or "single-region" image. This is the core capability behind IIIF (the library / museum high-resolution scan protocol). ④ One algorithm, both lossless and lossy — switching is just a matter of the quantisation step, not a separate standard like JPEG vs JPEG-LS.

JPEG XR — 微软的最后一次努力

JPEG XR — Microsoft's Last Attempt

微软第一个支持 HDR 32-bit float 的 web 格式,但 Chrome 没要它。

Microsoft's first 32-bit float HDR web format. Chrome said no.

2006 年微软看着 Web 上的图片格式仍然是 JPEG / GIF / PNG 三件套,觉得机会来了:推一个比 JPEG 强、带 alpha、支持 HDR 浮点、解码比 JPEG 2000 快的"下一代"web 图片格式。原名 HD Photo / Windows Media Photo,2009 年通过 ISO/IEC 29199 标准化为 JPEG XR(XR = eXtended Range)。技术上确实漂亮:16×16 大块 PCT 变换比 JPEG 的 8×8 DCT artifact 更不可见、原生支持 RGBE 和 scRGB 的 32-bit float HDR、无损与有损共用算法。微软在 Internet Explorer 9 / Edge Legacy 里直接内置了原生支持。但是——Chromium 拒绝实现,Mozilla 拒绝实现。理由很直白:"我们已经在押注 WebP / AVIF,不想为一个微软推的格式增加攻击面"。Edge 在 2018 年放弃自己的渲染引擎转 Chromium 后,JPEG XR 的最后一个原生支持者也消失了。讽刺的是,这套"微软推格式 - Chrome 拒绝实现 - 格式死亡"的剧本,后来被 Google 反过来用在了 WebP 推广上——你推什么我接什么,我推什么你最好接。

By 2006 Microsoft surveyed the web and saw JPEG / GIF / PNG still ruling the field. They saw a gap: ship a next-generation format that beats JPEG, adds alpha, supports HDR floats, and decodes faster than JPEG 2000. Originally HD Photo / Windows Media Photo, it was standardised in 2009 as JPEG XR ("eXtended Range") under ISO/IEC 29199. The technology was genuinely good: a 16×16 photo core transform (PCT) replaces JPEG's 8×8 DCT, with much less visible blocking; native support for RGBE and scRGB 32-bit float HDR; lossless and lossy sharing one algorithm. Microsoft baked native support into Internet Explorer 9 and Edge Legacy. But Chromium refused. Mozilla refused. The reasoning was blunt: "we're already betting on WebP / AVIF; we don't want extra attack surface for a Microsoft-pushed format." When Edge gave up its own rendering engine and switched to Chromium in 2018, the last browser with native JPEG XR support vanished. The painful irony: the "Microsoft pushes a format → Chrome refuses → format dies" playbook was later inverted by Google for WebP — what you push, I'll accept; what I push, you'd better accept.

技术内核

Technical core

JPEG XR 的技术设计有三个亮点。① 整数 16×16 PCT(Photo Core Transform)——本质上是一个类 DCT 的整数变换,但块更大、内部还有一层 4×4 子变换做"重叠"(lapped transform),让块与块之间不再有硬边界。同等质量下,JPEG XR 的 blocking artifact 比 JPEG 弱得多,但解码复杂度只比 JPEG 高一点点(远低于 JPEG 2000 的 10×)。② 原生 HDR float 支持——这是 JPEG XR 最超前的部分。它直接编码 RGBE(共享指数 32-bit)和 scRGB 浮点,不需要色调映射就能存高动态范围内容。这比 HEIC / AVIF 推广 HDR 早了将近十年——但当时显示器和操作系统都没准备好,没人用得上。③ 共享熵编码思路——熵编码部分仍然用类 JPEG 的"块+扫描+游程+熵"路径,所以软件实现成本低,微软自己的参考实现一千多行 C 就够了。这跟 JPEG 2000 几万行的复杂度相比,工程上确实"够轻"——但终究敌不过浏览器厂商的政治意愿。

JPEG XR has three technical strengths. ① Integer 16×16 PCT (Photo Core Transform) — essentially a DCT-like integer transform with a larger block, plus an inner 4×4 sub-transform that does a lapped overlap, killing hard block edges between adjacent macro-blocks. At equal quality JPEG XR shows much weaker blocking than JPEG, while costing only marginally more to decode (nowhere near JPEG 2000's 10×). ② Native HDR float support — the most forward-looking piece. It encodes RGBE (shared-exponent 32-bit) and scRGB floating-point directly, storing high-dynamic-range content without tone-mapping. This predated HEIC's and AVIF's HDR push by nearly a decade — but in 2006 neither displays nor operating systems were ready, and nobody had a workflow for it. ③ Shared entropy-coding lineage — the entropy back end is still a JPEG-style "block + scan + run-length + entropy" pipeline, so implementations are small. Microsoft's own reference implementation is barely a thousand lines of C — far lighter than JPEG 2000's tens of thousands. Engineering cost wasn't the problem. Browser-vendor politics was.

WebP — Google 把 VP8 帧内拿来做图

WebP — Google Carved an Image Format Out of a Video Frame

把 VP8 视频的一帧抠出来当图片,体积砍掉 30%。

Took one frame out of a VP8 video, shaved 30% off image size.

2010 年的 Google 看着 web 图片世界,觉得三件套(JPEG / PNG / GIF)中间还有一道明显的"裂缝":没有一种格式能同时满足"比 JPEG 小 30%、比 PNG 小 26%、还能动图 + alpha"。Google 当时刚刚在 2009 年用 1.246 亿美元收购了视频编码公司 On2 Technologies,手里握着一颗刚开源的 VP8 视频 codec——VP8 的 intra-frame(I 帧) 已经具备完整的图像帧内编码能力。Google 工程师的算盘很直接:与其重新发明轮子,不如直接把 VP8 的一帧拿出来,套一层 RIFF 容器,就是一种新的图片格式。WebP 由此诞生——它是历史上第一个"视频 codec 直接派生为图片格式"的工业级例子,后来 HEIC / AVIF 都走了完全相同的路线。

In 2010 Google looked at the web's image landscape and saw a clear gap in the JPEG / PNG / GIF triumvirate: nothing was simultaneously "30% smaller than JPEG, 26% smaller than PNG, and capable of both animation and alpha." Having just paid $124.6 million in 2009 to acquire the video-codec company On2 Technologies, Google now owned the VP8 video codec — and a VP8 intra-frame (I-frame) is already a complete still-image encoding pipeline. The Googlers did the obvious thing: pull out a single VP8 frame, wrap it in a RIFF container, ship it as a new image format. WebP was born — historically the first industrial-scale example of "video codec directly repurposed into still-image format". HEIC and AVIF later took the exact same playbook.

技术内核

Technical core

WebP 内部其实是两个完全独立的格式,共用一个 .webp 后缀和一个 RIFF 外壳。① VP8 intra-frame(有损):4×4 / 16×16 块预测(共 10 + 4 种 intra mode)→ 类 DCT 整数变换 → 量化 → boolean arithmetic coding(算术编码)。预测让"猜得准的部分不用传",算术编码比 Huffman 多挤出 5-15% 体积——这是 WebP 比 JPEG 小 30% 的两大功臣。② VP8L(无损):跟 VP8 一点关系都没有,是 Google 自己写的一套独立无损算法——14 种 spatial predictor + color cache(用 hash table 缓存最近用过的颜色)+ LZ77 + Huffman。在自然图像上比 PNG 小 26%,但编码慢 5-10×。③ RIFF 容器:借用微软 Wave 音频用过的 RIFF 格式——文件头是 RIFF<size>WEBP,后面跟 chunk 序列:VP8X(全局信息)/ VP8(有损主帧)/ VP8L(无损主帧)/ ALPH(独立 alpha 通道)/ ANIM + ANMF(动图)/ ICCP(色彩配置)/ EXIF / XMP。④ 独立 alpha:lossy 主帧不带 alpha,alpha 走单独的 ALPH chunk,可以选择无损 lossless 或有损 lossy 编 alpha——这是 WebP 比 JPEG + PNG 拼凑方案精巧的地方。⑤ animated WebP:ANIM 设全局参数(背景色 / 循环次数), ANMF 每帧带 disposal / blend / xy offset,逻辑跟 GIF 完全同源,但每帧用 VP8 / VP8L 编。

WebP is, in fact, two unrelated formats sharing a .webp extension and a RIFF wrapper. ① VP8 intra-frame (lossy): 4×4 / 16×16 block prediction (10 + 4 intra modes) → DCT-like integer transform → quantise → boolean arithmetic coding. Prediction means "the easy-to-guess parts don't need to ship" and arithmetic coding squeezes out another 5–15 % over Huffman — together those are why WebP runs ~30 % smaller than JPEG. ② VP8L (lossless): unrelated to VP8 — a separate lossless codec Google wrote from scratch — 14 spatial predictors + a color cache (hash-tabling recently-used colours) + LZ77 + Huffman. ~26 % smaller than PNG on natural images but 5–10 × slower to encode. ③ RIFF container: borrowed from Microsoft's Wave audio — the file starts with RIFF<size>WEBP, then a sequence of chunks: VP8X (global info) / VP8 (lossy main frame) / VP8L (lossless main frame) / ALPH (separate alpha channel) / ANIM + ANMF (animation) / ICCP (color profile) / EXIF / XMP. ④ Separate alpha: lossy main frames don't carry alpha; alpha lives in a dedicated ALPH chunk that can itself be encoded losslessly or lossily — much cleaner than JPEG + PNG patchwork. ⑤ animated WebP: ANIM sets the globals (background colour, loop count), each ANMF frame carries disposal / blend / xy-offset just like GIF, but each frame is itself VP8 or VP8L.

$ cwebp -q 75 in.png -o out.webp                        # lossy default · Q 75 ≈ JPEG Q85 quality
$ cwebp -lossless in.png -o out.webp                    # VP8L lossless path · 5–10× slower
$ cwebp -near_lossless 60 in.png -o out.webp            # lossy preprocessing then lossless encode
$ cwebp -q 80 -alpha_q 100 in.png -o out.webp           # keep alpha lossless even with lossy RGB
$ webpmux -frame f1.webp +100 -frame f2.webp +100 \
          -loop 0 -o anim.webp                          # build animated WebP from frames
$ dwebp out.webp -o decoded.png                         # decode back to PNG for inspection

HEIC / HEIF — 苹果与专利墙

HEIC / HEIF — Apple and the Patent Wall

技术上是 AVIF 的爸爸,专利上是 AVIF 的反例。

Technically the parent of AVIF; legally the cautionary tale.

2015 年 MPEG 把 HEVC(H.265 视频)的帧内编码能力封装成一个图像容器规范,叫 HEIF(High Efficiency Image File Format),标准号 ISO/IEC 23008-12。思路与 WebP 完全同源:用现代视频 codec 的 intra-frame 做静态图像编码,用 ISOBMFF(MP4 同根的容器)装。HEIF 是个"容器规范",真正的像素 codec 由 payload 决定——用 HEVC 装就叫 HEIC(.heic),用 AVC/H.264 装就叫 HEIF AVCI;Apple 选了前者。2017 年 9 月 iOS 11 把相机默认存储格式从 JPEG 改成 HEIC——一夜之间,全球数亿台 iPhone 开始产生 HEIC 文件。比 JPEG 体积小一半、支持 10-bit HDR、支持 alpha、支持多对象嵌套——技术上没毛病,问题全在专利。

In 2015 MPEG wrapped HEVC's (H.265 video) intra-frame coding into an image-container spec called HEIF — High Efficiency Image File Format, ISO/IEC 23008-12. Same thinking as WebP: take a modern video codec's intra-frame, use it as a still-image codec, package it in ISOBMFF (the same container family as MP4). HEIF itself is just a container spec; the actual pixel codec depends on the payload — HEVC-payloaded HEIF is HEIC (.heic), AVC/H.264-payloaded HEIF is HEIF AVCI. Apple picked HEVC. In September 2017, iOS 11 switched the camera's default capture format from JPEG to HEIC — overnight, hundreds of millions of iPhones started producing HEIC files. Half the size of JPEG, 10-bit HDR support, alpha, nested multi-image objects — technically flawless. All the problems are in the patents.

技术内核

Technical core

HEIF / HEIC 的技术构造分四层。① HEIF 容器 = ISOBMFF box 系——跟 MP4 / MOV / 3GP 同根的"box-in-box"二进制格式,每个 box 4-byte size + 4-byte FourCC type + payload。这套格式过去 20 年被全球视频行业打磨得极其成熟,标准库一抓一大把,Apple 自然顺手。② HEVC intra-frame payload——CTU(Coding Tree Unit)最大可达 64×64,远大于 JPEG 的 8×8 / WebP 的 16×16,同样质量下 macroblock artifact 几乎肉眼不可见;intra prediction 有 35 种方向(DC + Planar + 33 angular),比 VP8 的 10 种细得多;后处理还有 SAO(Sample Adaptive Offset)和 deblocking filter,把块边界进一步抹平。这是 HEIC 能比 JPEG 小 50% 的核心。③ 多对象 / 派生项 / 网格——HEIF 不止能存"一张图",它能存"主图 + 缩略图 + 多视角图 + 派生编辑(裁剪 / 旋转 / 网格拼接)",每个对象一个 item,iloc 表查偏移。Apple 利用这个特性做"突发拍照"(把一个 burst session 的 10 张图打包成 1 个 .heic)。④ Live Photo 混合容器——iPhone 的 Live Photo 不是单文件,它是 1 张 .heic 静图(主关键帧)+ 1 段 .mov 视频(前后 1.5 秒 + 音频)的组合,iCloud 同步时把它们绑在一起作为"一个资产"管理——这是 HEIF 最被低估的工程贡献。

HEIF / HEIC has four technical layers. ① HEIF container = ISOBMFF box family — the same "box-in-box" binary format as MP4 / MOV / 3GP, every box is 4-byte size + 4-byte FourCC type + payload. Twenty years of video-industry tooling makes the spec battle-tested and trivial for Apple to adopt. ② HEVC intra-frame payload — the Coding Tree Unit can reach 64×64, much larger than JPEG's 8×8 or WebP's 16×16, so macroblock artefacts are practically invisible at the same quality; intra prediction has 35 directions (DC + Planar + 33 angular) versus VP8's 10; post-processing adds SAO (Sample Adaptive Offset) and a deblocking filter that further smooth block boundaries. That's the core reason HEIC weighs ~50 % less than JPEG. ③ Multi-item, derived items, grids — HEIF doesn't store "one image"; it stores "main image + thumbnails + multi-view images + derived edits (crop / rotate / grid-tile composition)". Each object is its own item, addressed via the iloc table. Apple uses this to pack a burst-photo session of ten images into a single .heic file. ④ Live Photo as a hybrid container — iPhone's Live Photo isn't a single file; it's a .heic still (the keyframe) + a .mov video (1.5 s before + 1.5 s after, with audio). iCloud syncs them as a bound pair, treating the combo as a single asset — HEIF's most underappreciated engineering contribution.

AVIF — AV1 的副产品成了王

AVIF — A Video Codec's Side-Effect Became King

为视频生的 codec,顺手把图片格式革命了一遍。

A video codec by birth — and it casually rewrote image formats.

2018 年 3 月 AOMedia 发布 AV1 视频编码 1.0,目标是做"完全免专利费的 HEVC 替代品"——背后是 Google / Mozilla / Cisco / Apple / Netflix / Microsoft / Intel / Amazon / Nvidia / Samsung 三十多家公司组成的联盟,带着各自的专利池交叉许可。AV1 走的是同一条"视频帧内 → 静态图片"路径(WebP / HEIC 都是这条路),把 intra-frame 编码能力套进 HEIF 容器(ISOBMFF),就拿到了 AVIF (AV1 Image File Format)——体积比 HEIC 略小、专利免费、跨厂商共识、Chrome 与 Firefox 与 Safari 三大引擎都点头。AVIF 2019 年 2 月发布标准,Chrome 85 (2020 年 8 月) 落地,Firefox 93 (2021 年 10 月) 跟进,Safari 16.4 (2023 年 3 月) 收尾——HEIC 阵营在 Web 上正式退场。

In March 2018 AOMedia shipped AV1 1.0 — the goal was a "completely royalty-free HEVC alternative". The alliance behind it is 30+ companies (Google, Mozilla, Cisco, Apple, Netflix, Microsoft, Intel, Amazon, Nvidia, Samsung…) cross-licensing their patent pools to make it stick. AV1 took the same "video intra-frame → still image" route as WebP and HEIC, wrapped its intra-frame encoder in HEIF (ISOBMFF), and out came AVIF (AV1 Image File Format) — smaller than HEIC, patent-free, cross-vendor, with all three big browser engines on board. The spec landed in February 2019, Chrome 85 shipped it in August 2020, Firefox 93 in October 2021, Safari 16.4 in March 2023. On the open web HEIC was officially out.

技术内核

Technical core

AVIF 的技术深度全在 AV1 这一侧——容器只是 HEIF 的复用。① AV1 intra prediction:56 种角度方向(粗扇 9° 步 + 细扇 3° 步)+ 4 种特殊模式(DC / Planar / Smooth / Paeth)+ CfL(Chroma from Luma,色度从亮度推导)+ Palette mode(每块独立小调色板,适合 UI 截图)+ Intra Block Copy(块内自指,跟视频的"运动补偿"对偶)——比 HEVC 的 35 方向、VP8 的 10 方向都细很多。② Superblock 128×128 + 多种切分:递归切到最小 4×4,还允许 2:1 / 1:2 / 4:1 矩形,平坦区整块保留、纹理区切细。③ 变换块 16 种组合:DCT-2 / ADST(非对称离散正弦)/ WHT(Walsh-Hadamard)/ IDTX(恒等)四种变换在 H/V 两个方向独立选择,共 4×4 = 16 种组合——纹理方向不同,选不同变换效率最优。④ HEIF 容器:跟 HEIC 完全同根的 ISOBMFF box 树(ftyp 'avif' · meta · iloc · iprp · mdat),thumbnail / alpha / depth map 都是独立 item。⑤ 专利策略是它最大的非技术杀招:AOMedia 的核心承诺是"会员单位互相 royalty-free 交叉许可,且对所有人 patent non-assert"——Google / Cisco 把已有专利池贡献进来,把"做一个免费 codec"从技术问题变成了行业政治问题,并赢了。

AVIF's technical depth lives on the AV1 side; the container is just HEIF reused. ① AV1 intra prediction: 56 angular directions (coarse 9° + fine 3° steps) + 4 special modes (DC / Planar / Smooth / Paeth) + CfL (Chroma-from-Luma) + Palette mode (per-block tiny palette, great for UI screenshots) + Intra Block Copy (intra-frame self-reference, the still-image dual of motion compensation). HEVC has 35 directions; VP8 had 10. ② Superblocks at 128×128 recursively split down to 4×4, with 2:1 / 1:2 / 4:1 rectangular partitions — flat regions stay whole, textured regions split. ③ Sixteen transform-block combinations: DCT-2 / ADST (asymmetric discrete sine) / WHT (Walsh-Hadamard) / IDTX (identity) chosen independently for H and V — 4×4 = 16 combos, different texture orientations get different transforms. ④ HEIF container: the same ISOBMFF box tree as HEIC (ftyp 'avif', meta · iloc · iprp, mdat); thumbnails, alpha and depth maps live as independent items. ⑤ Patent strategy is the real masterstroke: AOMedia's binding promise is "members cross-license royalty-free; every patent the alliance touches is non-asserted against the world". Google and Cisco committed their pools, and the question of "can a free codec exist?" turned from a technical one into an industry-politics one — which they won.

$ avifenc --min 0 --max 63 -a end-usage=q -a cq-level=23 in.png out.avif   # typical Q23 — visually near-lossless
$ avifenc -j 8 -s 6 in.png out.avif                # speed 0–10 (lower=better/slower); -j threads
$ avifenc -d 10 --yuv 444 in.png out.avif          # 10-bit + 4:4:4 chroma — for HDR / design assets
$ avifdec out.avif decoded.png                     # reference decode via libavif
$ cavif --quality 80 in.png -o out.avif            # Rust CLI built on rav1e; faster preset

JPEG XL — 被 Chrome 砍掉的"完美"格式

JPEG XL — The "Perfect" Format Chrome Killed

技术上吊打所有人,被 Chrome 团队以"兴趣不足"砍掉。

Technically beats everyone. Chrome killed it citing "insufficient interest".

2017 年 AOMedia 已经在猛推 AVIF,但有一群人不满足:HDR 摄影师、印刷出版业、漫画 / 插画家、需要无损归档的博物馆、还有手里握着几十亿张 JPEG 资产没法迁移的所有人——AVIF 解决不了他们的问题。Cloudinary 与 Google Research 把两个独立项目(Cloudinary 的 FUIF + Google 的 PIK)合并,推出 JPEG XL,目标是做"一个能同时干完所有事的下一代格式":(a) 把现存 JPEG 文件 无损 transcode 成 JXL,体积省 ~20%,任何时候可逆向恢复原 byte-exact JPEG;(b) 现代 VarDCT lossy 编码,质量比 AVIF 略好;(c) Modular 模式做无损,比 PNG / WebP-LL 都小;(d) 真正的渐进式解码——第一段 ~1/64 数据就能显示完整的"像素化粗略图",随后几段越来越清晰;(e) 8–32 bit + float、HDR、宽色域、CMYK、高位深 alpha 全套原生。技术上几乎是"现代格式应该有的样子"的完整集成,2021 年 2 月以 ISO/IEC 18181 标准化通过——但落地之路比技术艰难得多。

By 2017 AOMedia was already pushing AVIF hard, but a constituency wasn't satisfied: HDR photographers, the print and publishing industry, comic/manga artists, archival museums, and anyone holding billions of legacy JPEGs they couldn't migrate — AVIF solved none of their problems. Cloudinary and Google Research merged two independent projects (Cloudinary's FUIF and Google's PIK) into JPEG XL with the explicit ambition of "doing all of it at once": (a) losslessly transcode existing JPEGs into JXL, ~20 % smaller, reversible to byte-exact original JPEG; (b) modern VarDCT lossy with quality slightly above AVIF; (c) Modular mode for lossless, smaller than both PNG and WebP-LL; (d) real progressive decoding — the first ≈1/64 of the bitstream already displays a complete coarse image, with subsequent segments adding detail; (e) native 8–32 bit + float, HDR, wide gamut, CMYK and high-bit-depth alpha. Technically it's the complete integration of "what a modern format should look like". ISO/IEC 18181 was published in February 2021. The path to adoption proved much harder than the engineering.

技术内核

Technical core

JXL 的技术广度是当代图像格式里最大的——它把"现代图像格式应该有的所有能力"打包进同一个容器,六个核心点:① VarDCT(可变块 DCT)——块大小可在 2×2 到 256×256 之间自由变化,远比 AVIF 的 4×4–128×128 灵活;搭配 XYB(感知分离的色彩空间,JXL 自创)+ 自适应量化矩阵(可按图像内容定制),lossy 模式直接对标 AVIF。② Modular 模式——meta-adaptive 预测器(WP / Gradient / Self-correcting,可学习权重)+ 通道变换链(Squeeze / RCT / 自定义 transform),做无损或 near-lossless,小于 PNG / WebP-LL 30–50%。③ JPEG 无损 transcode(最革命性):任意 JPEG 文件解码到 DCT 系数,不再变换、不再量化,直接用 JXL 的熵编码重新打包,体积省 ~20%;djxl 反向时 byte-exact 恢复原 JPEG——这是其它 codec 全都做不到的事。④ 真渐进式解码——比特流头部就是低分辨率版本,解码器收到前 ~1/64 字节就能渲染一张完整的低分辨率图(不像 progressive JPEG 是按频率扫描,中途看起来糊);非常适合慢网。⑤ HDR / 32-bit float / wide gamut / CMYK 全原生——无需 ICC profile hack,XYB 色空间内部就支持 HDR;打印行业的高位深 + CMYK 也是一等公民。⑥ Patch 系统——对图片中重复出现的 pattern(同一个表情、漫画里反复出现的角色脸)单独编码一次,在出现位置插入引用,极大压缩漫画 / 表情包 / 截图。技术上几乎是"现代图像格式应该有的样子"的完整集成。

JXL has the broadest technical surface area of any current image format — it bundles every capability "a modern image format ought to have" into one container. Six pillars: ① VarDCT — block sizes range freely from 2×2 to 256×256, far more flexible than AVIF's 4×4–128×128. Combined with XYB (a perceptually separated colour space JXL invented) and content-adaptive quantisation matrices, lossy mode trades blow-for-blow with AVIF. ② Modular mode — meta-adaptive predictors (WP / Gradient / Self-correcting, weights learnable) plus channel-transform chains (Squeeze / RCT / custom) deliver lossless or near-lossless that's 30–50 % smaller than PNG and WebP-lossless. ③ JPEG lossless transcode (the revolutionary one): decode any JPEG into its DCT coefficients, skip requantising and re-transforming, and just re-encode with JXL's entropy coder — about 20 % smaller. djxl recovers the original JPEG byte for byte. No other codec offers this. ④ True progressive decoding — the bitstream's head is the low-resolution version. Receive the first ~1/64 of bytes and the decoder renders a complete coarse image (unlike progressive JPEG, which scans by frequency and stays blurry mid-load). Excellent for slow networks. ⑤ HDR, 32-bit float, wide gamut, CMYK all native — no ICC-profile hacks; XYB supports HDR internally; high-bit-depth + CMYK are first-class for print. ⑥ Patch system — encode a repeating pattern (an emoji, a recurring character face in a comic) once, then place references at every occurrence. Comics, sticker sheets and screenshots compress dramatically.

$ cjxl in.png out.jxl --quality 90       # quality 0–100 (≈90 visually lossless)
$ cjxl in.png out.jxl --distance 1.0     # distance: 0=lossless, ~1=Q90, ~3=Q75
$ cjxl in.jpg out.jxl --lossless_jpeg 1  # JPEG → JXL lossless transcode (~20% smaller)
$ djxl out.jxl roundtrip.jpg             # reverse transcode — byte-exact original .jpg
$ cjxl in.png out.jxl -d 0 -e 9          # lossless, max effort (smallest, slowest)

KTX / KTX2 — 容器与 payload 的分离

KTX / KTX2 — separating container from payload

"它本身不是格式,是装格式的盒子。"

"Not a format itself — a box that holds formats."

GPU 块压缩格式(BCn / ETC2 / ASTC)的规范只规定了"4×4 像素块怎么编成几个字节",但没规定一个完整的纹理资产文件要怎么组织——mipmap 链怎么排?cubemap 的六个面怎么放?array layer 怎么索引?ICC color profile 放哪?Khronos 看不下去,做了 KTX(Khronos TeXture)当通用容器:头部 + key-value metadata + level/layer/face 的 byte-offset 索引表 + 真正的像素 payload。KTX 不关心 payload 是 BC7 还是 ASTC,只负责"把它装好、运行时一次性 upload 到 GPU"。2019 年 KTX2 加上 supercompression(用 Zstd 或 Basis Universal 把已经 GPU 压过的 payload 再压一遍),并把 mip 顺序改成 smallest-first 便于流式加载——成了 glTF 2.0 / WebGPU / Babylon / three.js 的资产事实标准。

GPU block-compression specs (BCn / ETC2 / ASTC) only define "how a 4×4 pixel block is encoded into a few bytes" — they say nothing about how a complete texture asset is laid out: how the mip chain is ordered, how the six faces of a cubemap sit together, how array layers are indexed, where the ICC colour profile lives. Khronos picked up the slack with KTX (Khronos TeXture): header + key-value metadata + a byte-offset index table for every level/layer/face + the actual pixel payload. KTX is payload-agnostic — it doesn't care whether the payload is BC7 or ASTC, it just packs the asset and lets the runtime upload it to the GPU in one go. KTX2 (2019) added supercompression — running the already-GPU-compressed payload through Zstd or Basis Universal a second time — and reversed the mip order to smallest-first so streaming loaders can swap in a low-res placeholder immediately. It is now the de-facto asset format for glTF 2.0, WebGPU, Babylon.js and three.js.

技术内核

Technical core

KTX 的设计有四个支点。① header + index 表——文件头 80 字节,描述纹理的逻辑维度(width / height / depth / mip levels / array layers / faces);后面跟一张 level index 表,告诉 loader 第 N 级 mip 在文件内的 byte offset 和 byte length。这种"先索引后数据"的布局让 loader 不用扫整个文件就能跳读任意 level。② 每 mip level 内有 padding——GPU 上传时纹理需要按硬件对齐(通常 4 字节或 8 字节边界),KTX 直接在 file format 层面加 padding,运行时 memcpy 一行就能直接交给 glCompressedTexImage2D。③ KTX2 supercompression——这是 KTX2 相对 KTX1 最大的进化。GPU 块压缩(BC7 / ASTC)在 GPU 端是不能再压的——它们必须保持"硬件能直接 sample"的格式。但传输时(网络下载、磁盘存储)可以再用 Zstd 把字节流压一遍,运行时解压回原样再 upload。Basis Universal 更激进:它在 KTX2 里存的是一种"中间表示",运行时按目标设备转码成 BC7(桌面 D3D12 / Vulkan)、ETC2(老移动)或 ASTC(现代移动)——一个文件,所有平台。④ 多对象类型——同一份 KTX2 可以装单 2D 纹理、cubemap(6 face)、texture array(N layer)、3D 体积纹理,甚至带 mipmap 的 cubemap array(常用于 IBL 反射探针)。glTF 2.0 用 KHR_texture_basisu 扩展把 KTX2 + Basis 钉成 PBR 资产的官方携带格式。

KTX rests on four pillars. ① Header + level index — an 80-byte header describes the texture's logical dimensions (width / height / depth / mip levels / array layers / faces); then a level index lists the byte offset and byte length of every mip level. With "index first, data later" a loader can seek straight to any level without scanning the whole file. ② Padding inside each mip level — GPUs require texture rows to land on hardware-aligned boundaries (typically 4- or 8-byte). KTX bakes the padding into the file so the runtime can memcpy a row straight into glCompressedTexImage2D. ③ KTX2 supercompression — the headline upgrade over KTX1. GPU block compression (BC7 / ASTC) cannot be re-compressed on the GPU — the format has to stay "hardware-sampleable". But for transit (download, disk) the byte stream can be Zstd'd once and decompressed at load time before upload. Basis Universal goes further: KTX2 stores an intermediate representation that the runtime transcodes per-device into BC7 (desktop D3D12 / Vulkan), ETC2 (older mobile) or ASTC (modern mobile). One file, every platform. ④ Multi-object payload — a single KTX2 can carry a 2D texture, a cubemap (6 faces), a texture array (N layers), a 3D volume texture, even a mipmapped cubemap array for IBL reflection probes. glTF 2.0's KHR_texture_basisu extension nails KTX2 + Basis as the official carrier for PBR assets.

DDS — DirectDraw Surface 容器

DDS — the DirectDraw Surface container

"D3D 时代的 KTX,只是没人记得它先来。"

"The KTX of the D3D era — except few remember it came first."

1999 年 Direct3D 7.0 推出的时候,游戏行业急需一个"硬件能直接 sample 的纹理容器"——你不能用 BMP / TGA,因为它们是 CPU 端 RGBA,显卡读到要先解压再上传,带宽吃不住。微软干脆把 DirectDraw Surface(.dds)定义成纹理资产的标准磁盘格式:头部 124 字节描述维度 / mip 数 / pixel format / cubemap 标记,后面直接是 DXT(后来的 BCn)块或未压缩 RGBA8 字节流。Khronos 的 KTX 要 6 年后(2005)才出来。所以严格讲,"GPU 纹理容器"这个范式是微软先做的——KTX 是开放生态对它的回应。Bethesda 时代的 PC 游戏 mod 圈,几乎所有纹理替换包都是 .dds——这就是它的护城河。

When Direct3D 7.0 shipped in 1999, the games industry urgently needed "a texture container the hardware could sample directly". BMP and TGA were CPU-side RGBA — the GPU would have to decompress and re-upload before sampling, and the bus simply couldn't take it. Microsoft defined DirectDraw Surface (.dds) as the standard on-disk texture asset: a 124-byte header describing dimensions / mip count / pixel format / cubemap flags, followed by raw DXT (later BCn) blocks or uncompressed RGBA8. Khronos's KTX wouldn't appear for another six years. Strictly speaking, the "GPU texture container" idea was Microsoft's first — KTX is the open-ecosystem reply. The Bethesda-era PC modding scene (Skyrim / Fallout) shipped texture replacements almost exclusively as .dds — that's the moat that keeps DDS relevant.

技术内核

Technical core

DDS 的结构简单到几乎没什么可讲的——这是它的优点。① 头部 124 字节 DDS_HEADER:固定字段描述 width / height / depth(volume 纹理用)/ mipMapCount / pitch(每行 byte 数)/ PixelFormat(老的 FourCC 字段:DXT1/DXT3/DXT5/...);加上 dwCaps / dwCaps2 标记(cubemap / volume / mip)。② DX10 扩展头 20 字节(可选):DirectX 10+ 引入的现代头,用 DXGI_FORMAT 枚举(DXGI_FORMAT_BC7_UNORM / DXGI_FORMAT_BC6H_UF16 / ...)替代 FourCC——因为新的块压缩格式(BC6H、BC7)的 FourCC 名字位不够用了。③ payload 直接是块压缩字节流——没有 padding 设计、没有 supercompression、没有 key-value metadata,只有最直接的 mip + face + layer 字节拼接。这是它跟 KTX2 最大的差距:DDS 是"足够好"的工程容器,KTX2 是"考虑到 Web / 跨平台 / Basis 转码"的现代容器。但对于 Windows / D3D 闭环,DDS 已经够用 25 年。

DDS's structure is almost embarrassingly simple — and that's its strength. ① The 124-byte DDS_HEADER: fixed fields for width / height / depth (for volume textures) / mipMapCount / pitch (bytes per row) / PixelFormat (the old FourCC field — DXT1 / DXT3 / DXT5 / …); plus dwCaps / dwCaps2 flags (cubemap / volume / mip). ② The optional 20-byte DX10 extension header: a modern header introduced in DirectX 10+ that swaps FourCC for the DXGI_FORMAT enum (DXGI_FORMAT_BC7_UNORM / DXGI_FORMAT_BC6H_UF16 / …) — necessary because newer block formats (BC6H, BC7) ran out of FourCC bits. ③ The payload is just block-compressed bytes — no padding scheme, no supercompression, no key-value metadata, just the most direct possible concatenation of mip × face × layer bytes. That's the gap with KTX2: DDS is a "good enough" engineering container, KTX2 is a modern container that thinks about Web, cross-platform delivery and Basis transcoding. For a Windows / D3D walled garden, though, DDS has been sufficient for 25 years.

BC1 (DXT1) — 4×4 块、4 bpp 的祖宗

BC1 (DXT1) — the 4×4-block, 4-bpp ancestor

"4 个像素压成 8 字节,显存砍掉 8 倍,从此再也回不去。"

"Four pixels squeezed into eight bytes — VRAM cut 8×, no going back."

1998 年的 GPU 显存极其稀缺——NVIDIA Riva TNT 旗舰 16 MB,普通卡 8 MB,而一张 256×256 的 RGBA 纹理就要 256 KB。一个游戏关卡要几十张纹理,显存装不下,带宽更扛不住(显存带宽要支撑帧缓冲、Z-buffer、纹理 sample 三路并发)。S3 Graphics 提出 S3TC(S3 Texture Compression):把 4×4 = 16 个像素打包成 8 字节,体积压到 1/8(原 64 字节),GPU 纹理单元在 sample 时硬件解块——不需要 CPU 全图解压上传,显存里存的就是块数据。一夜之间,同样显存能装 8 倍的纹理,带宽吃掉 1/8。这是 GPU 块压缩的开山之作,定义了往后 25 年所有 BCn / ETC / ASTC 的基础范式:固定大小块 + 端点 + 内插 + 索引。

In 1998, GPU VRAM was scarce — NVIDIA's flagship Riva TNT had 16 MB, mid-range cards 8 MB. A single 256×256 RGBA texture cost 256 KB. A game level needed dozens; the VRAM couldn't hold them and the bus couldn't feed them (memory bandwidth had to serve framebuffer, Z-buffer and texture sampling at the same time). S3 Graphics proposed S3TC (S3 Texture Compression): pack 4×4 = 16 pixels into 8 bytes, an 8× shrink from the original 64 bytes; the texture unit decodes a block on the fly during sampling, so VRAM stores the compressed blocks directly without any CPU-side full-image decompression. Overnight, the same VRAM could hold 8× as many textures and the bus had to move 1⁄8 the bytes. This is the founding act of GPU block compression and it set the template every later BCn / ETC / ASTC variant follows: fixed-size block + endpoints + interpolation + per-pixel index.

技术内核

Technical core

BC1 的"4×4 块 + 端点 + 内插 + 索引"四件套是它的全部技术内核,也是后面所有 BCn / ETC / ASTC 都在改进的同一个范式。① 固定大小块——4×4,绝不可变。这是为了让 GPU 纹理单元能直接通过坐标计算定位到块,不需要扫表;sample 一个像素只需要"算块号 → 加载 8 字节 → 解端点 → 查 index → 输出颜色"四步,完全硬件实现。② 端点 + 内插——只存两个端点 c0/c1(RGB565,各 16-bit),内插出 c2/c3 让块能表达 4 种颜色。这是个赌博:它假设一个 4×4 块内的颜色变化是"沿着色空间一条直线"的,适用于大多数自然纹理(草地、石头、皮肤)但对锯齿状颜色边缘会糊。③ 2-bit/像素 index——每像素只需要 2 bit 选 4 选 1,16 像素共 32 bit = 4 byte,跟 endpoints 的 4 byte 加一起正好 8 byte 一块。④ 1-bit alpha 隐藏档——如果 c0 ≤ c1(数值上),BC1 进入"alpha 模式":c3 变成"完全透明",c2 = (c0 + c1)/2 只有一种内插;每像素的 index = 3 表示透明。这就是 BC1 的"穷人 alpha"——只有透/不透,但不占额外字节。需要平滑 alpha 必须升级 BC2 / BC3。

BC1's "4×4 block + endpoints + interpolation + index" combo is the entire technical core — every later BCn / ETC / ASTC just iterates on this same template. ① Fixed-size blocks — 4×4, immutable. This lets the GPU's texture unit address a block directly via coordinate arithmetic, no lookup needed; sampling one pixel reduces to "compute block id → load 8 bytes → decode endpoints → read index → emit colour", four steps, all hardware. ② Endpoints + interpolation — only two endpoints c0/c1 (RGB565, 16 bits each) are stored; c2/c3 are interpolated so the block expresses four colours. It's a bet: BC1 assumes the colour variation in any 4×4 block lies along a straight line in colour space. True enough for most natural textures (grass, stone, skin), but jagged colour edges blur. ③ 2 bits per pixel of index — each pixel just needs 2 bits to choose one of four colours; 16 pixels × 2 bits = 32 bits = 4 bytes, which combined with the 4 bytes of endpoints lands exactly at 8 bytes per block. ④ 1-bit alpha hidden mode — if c0 ≤ c1 numerically, BC1 enters "alpha mode": c3 becomes fully transparent, c2 = (c0 + c1)/2 is the only interpolated colour, and an index of 3 means transparent. That's BC1's "poor man's alpha" — opaque/transparent only, no extra bytes. For smooth alpha you have to step up to BC2 / BC3.

BC2 / BC3 (DXT3 / DXT5) — alpha 处理两条路

BC2 / BC3 (DXT3 / DXT5) — two ways to handle alpha

"BC2 给你显式 4-bit alpha,BC3 让 alpha 也学 BC1 的内插。"

"BC2 gives explicit 4-bit alpha; BC3 lets alpha use the BC1 trick too."

BC1 的 1-bit alpha(透 / 不透)对游戏 UI 圆角、粒子边缘、烟雾、玻璃、毛发都不够——这些都需要平滑的 alpha 渐变(0 到 255 中间的值)。S3 / Microsoft 在 1998-1999 同时提出 DXT3 和 DXT5 两条路:DXT3(BC2)粗暴,每像素直接给 4-bit alpha,16 像素共 64 bit = 8 byte;再加 BC1 的 8 byte 颜色块,共 16 byte/块,8 bpp。DXT5(BC3)聪明,把 alpha 也当成"端点 + 内插"块——存 2 个 8-bit alpha 端点 + 6 个内插值(共 8 种 alpha) + 每像素 3-bit index;颜色块仍用 BC1 那套。两者体积一样(16 byte/块),但 BC3 在平滑 alpha 渐变(粒子、烟雾)上明显好,BC2 在锐利 alpha 边缘(UI 图标的 1-bit-like alpha)上略好——但实践中 BC3 几乎全胜。所以游戏圈 BC3 / DXT5 才是事实主流。

BC1's 1-bit alpha (opaque or transparent, nothing in between) wasn't enough for game UI rounded corners, particle edges, smoke, glass or hair — all of those need smooth alpha gradients (values between 0 and 255). S3 / Microsoft proposed DXT3 and DXT5 in 1998-1999, two roads. DXT3 (BC2) is brute force: store an explicit 4-bit alpha per pixel; 16 pixels × 4 bits = 64 bits = 8 bytes; plus the 8-byte BC1 colour block, total 16 bytes per block at 8 bpp. DXT5 (BC3) is clever: treat alpha as an "endpoints + interpolation" block too — 2 × 8-bit alpha endpoints + 6 interpolated values (8 alpha levels in total) + a 3-bit index per pixel; the colour block still uses BC1. Both occupy the same 16 bytes per block, but BC3 clearly wins on smooth alpha gradients (particles, smoke); BC2 has a slight edge on razor-sharp alpha edges (UI icons that are basically 1-bit alpha). In practice BC3 wins almost everywhere — so the games industry treats BC3 / DXT5 as the de-facto default.

技术内核

Technical core

两个格式的核心差异全在 alpha 块。① 都是 16 byte/块,8 bpp——BC1 的颜色块 8 byte 不变,各加 8 byte 的 alpha 块。颜色端点和 BC1 一样:c0/c1 RGB565 + 内插 c2/c3 + 2-bit index——没区别。② BC2 的 alpha 块 = 16 个 4-bit 直接值——每像素 0-15 表示 alpha 量化到 16 阶。优点:对锐利 alpha 边界(UI 图标、纹理掩码)无量化误差;缺点:平滑渐变只有 16 阶,会出 banding。BC2 在 1999 年被一些早期 UI 系统用过,后来逐渐让位给 BC3。③ BC3 的 alpha 块 = BC1 alpha 化——存 2 个 8-bit alpha 端点 a0/a1(各 1 byte = 2 byte),如果 a0 > a1 用 6 个 1/7 步长内插值(共 8 阶),如果 a0 ≤ a1 用 4 个内插值 + 2 个保留(0 和 255 的硬端点)= 8 阶里有 2 个固定;每像素 3-bit index(16 px × 3 bit = 48 bit = 6 byte)。共 2+6 = 8 byte。BC3 在平滑 alpha(粒子、烟雾、毛发)上明显优于 BC2,代价是锐利 alpha 边缘会有轻微模糊。④ 命名混乱:游戏圈一般叫 DXT3 / DXT5(D3D 老命名),Khronos / Vulkan / Metal 一般叫 BC2 / BC3——同一个东西两套名字,是 OpenGL 和 D3D 命名分歧的活化石。

The whole difference between the two lives in the alpha block. ① Both are 16 bytes per block, 8 bpp — the BC1 colour block (8 bytes) is unchanged; each format adds an 8-byte alpha block. Colour endpoints, c2/c3 interpolation and 2-bit indices are identical to BC1 — no surprises there. ② BC2's alpha block = 16 explicit 4-bit values — each pixel quantises alpha to one of 16 levels. Pro: zero quantisation error on sharp alpha edges (UI icons, masks). Con: only 16 levels, so smooth gradients band. BC2 saw use in some early-2000s UI systems and then quietly handed the baton to BC3. ③ BC3's alpha block = BC1, applied to alpha — store 2 × 8-bit alpha endpoints a0/a1 (1 byte each = 2 bytes); if a0 > a1, interpolate 6 values at 1/7 steps (8 levels total); if a0 ≤ a1, interpolate 4 values + reserve two slots for hard 0 and 255 (2 of the 8 levels are fixed); 3-bit index per pixel (16 × 3 = 48 bits = 6 bytes). Total 2 + 6 = 8 bytes. BC3 clearly beats BC2 on smooth alpha (particles, smoke, hair), at the cost of slightly fuzzier sharp alpha edges. ④ Naming chaos: the games industry says DXT3 / DXT5 (D3D legacy); Khronos / Vulkan / Metal say BC2 / BC3 — same thing, two name systems, a living fossil of the OpenGL-vs-D3D naming split.

BC4 / BC5 — 单/双通道,法线贴图省一通道

BC4 / BC5 — single / dual channel, dropping a channel from normal maps

"法线贴图省一通道,显存再砍一半。"

"Drop a channel from normal maps; halve the VRAM again."

游戏图形里,法线贴图是仅次于 albedo 的第二大显存消耗——每个像素一个法线向量(X, Y, Z)。直觉上要 RGB 三通道,但法线是单位向量(长度 = 1),所以 Z 可以由 X / Y 推导出来:Z = sqrt(1 - X² - Y²)。这意味着实际只需要存 X / Y 两个通道,Z 在 fragment shader 里现算。BC5 就是为这个场景设计的——只存 R / G 两通道,每个通道用 BC3 的 alpha 块法(端点 + 内插 + 3-bit index),共 16 byte/块、8 bpp。BC4 是 BC5 的"半个版本",只存一个通道,用于灰度纹理:高度图、roughness 图、AO 遮罩、metallic 通道。BC4 / BC5 的本质是"把 BC3 的 alpha 块单独拎出来当颜色通道用"——这种"通道拆分 + 几何内插"的思路把法线贴图的显存占用从 BC3 RGB 的 8 bpp 直接砍到 BC5 RG 的 8 bpp 但质量提升 3-5×(因为不浪费 bits 在不需要的通道上)。

In game graphics, normal maps are the second-largest VRAM hog after albedo — every pixel stores a normal vector (X, Y, Z). Intuitively that means three RGB channels, but a normal is a unit vector (length 1), so Z can be derived: Z = sqrt(1 − X² − Y²). You really only need to store X / Y; the fragment shader recomputes Z. BC5 is built for exactly that — store just R / G, each compressed with the BC3 alpha-block trick (endpoints + interpolation + 3-bit index), 16 bytes per block at 8 bpp. BC4 is the "half-version" of BC5: just one channel, for greyscale textures — height maps, roughness maps, AO masks, the metallic channel. BC4 / BC5 are essentially "BC3's alpha block lifted out and used as a colour channel". This "channel split + geometric interpolation" trick keeps normal maps at 8 bpp (same as BC3 RGB) but bumps quality 3-5× because no bits are wasted on a channel you don't need.

技术内核

Technical core

BC4 / BC5 的设计思路简洁到一句话:把 BC3 的 alpha 块当成"通用的单通道压缩块"用。① BC4 = BC3 的 alpha 块独立——4×4 块,8 byte;存 2 个 8-bit 端点 r0 / r1(2 byte)+ 6 个内插值(隐含,不占字节,运行时算)+ 每像素 3-bit index(16 × 3 = 48 bit = 6 byte);共 8 byte / 16 像素 = 4 bpp。每像素只有一个 8-bit 通道(原数据的 R)。② BC5 = 两个 BC4 块叠加——一个 BC4 块存 R(法线 X),一个 BC4 块存 G(法线 Y);共 16 byte / 块 = 8 bpp。Z 不存,fragment shader 里算 z = sqrt(1 - x*x - y*y)——单 sqrt + 2 mul + 1 sub,GPU 一周期完成。③ BC4 的"unsigned"和"signed"两种模式:BC4_UNORM(0-255)和 BC4_SNORM(-128 到 127),后者专门给法线分量这种"中心对称"信号用,避免 0.5 偏置。BC5 同理。④ 命名又分裂:Khronos 叫 BC4 / BC5,Microsoft 老命名叫 ATI1 / ATI2(AMD 提出的格式名),OpenGL ARB 扩展叫 RGTC1 / RGTC2(Red-Green Texture Compression)——三套名,一个东西。游戏引擎源码里三种叫法都能见到。

BC4 / BC5's design boils down to one sentence: take BC3's alpha block and reuse it as a generic single-channel compression block. ① BC4 = BC3's alpha block, standalone — 4×4 block, 8 bytes; 2 × 8-bit endpoints r0 / r1 (2 bytes) + 6 implicit interpolated values (computed at runtime, no bytes spent) + 3-bit per-pixel index (16 × 3 = 48 bits = 6 bytes); total 8 bytes / 16 pixels = 4 bpp. Each pixel carries one 8-bit channel (the input's R). ② BC5 = two BC4 blocks stacked — one BC4 block for R (normal X), one for G (normal Y); 16 bytes per block = 8 bpp. Z isn't stored — the fragment shader computes z = sqrt(1 − x*x − y*y), one sqrt + two muls + one sub, retired in a single GPU cycle. ③ BC4 has UNORM and SNORM modes — BC4_UNORM (0-255) and BC4_SNORM (−128 to 127); the signed variant is specifically for centre-symmetric signals like normal components, avoiding a 0.5 bias. BC5 mirrors this. ④ Naming forks again: Khronos says BC4 / BC5; Microsoft's legacy names are ATI1 / ATI2 (AMD-coined names); the OpenGL ARB extension calls them RGTC1 / RGTC2 (Red-Green Texture Compression). Three names, one thing — and you'll see all three in any sufficiently old engine source tree.

BC6H — HDR 块压缩

BC6H — HDR block compression

"显存里的 HDR — 反射探针、cubemap 全靠它。"

"HDR in VRAM — reflection probes and cubemaps depend on it."

PBR(基于物理的渲染)需要 HDR 环境贴图——天空、室内 IBL 反射探针、自发光场景全是。问题是 BC1-5 都基于 8-bit/通道 端点 + 内插,根本无法表达 float16 的 [-65504, 65504] 范围。如果用未压缩 RGBA16F,一张 1024×1024 的 cubemap(6 面)要 1024×1024×6×8 = 48 MB。一个室外场景几张 cubemap 几百 MB 就没了。BC6H 是 D3D11 时代专门为 HDR 设计的块压缩:4×4 块、16 byte/块、8 bpp(跟 BC7 同尺寸),但 payload 直接是 float16 RGB(无 alpha)。它用 14 种块模式来权衡精度——根据这块的颜色分布选最合适的模式。BC6H 让 HDR cubemap 体积从 RGBA16F 的 64 bpp 砍到 8 bpp(8× 压缩),同时保持 float16 的动态范围——这是 PBR 渲染管线得以普及的硬件基础。Unreal Engine 4 / 5、Unity HDRP 默认对 cubemap 的 HDR 资产用 BC6H。

PBR (physically based rendering) needs HDR environment maps — skies, indoor IBL reflection probes, emissive scenes all live in HDR. The trouble is that BC1-5 all rely on 8-bit-per-channel endpoints + interpolation, so they simply cannot express float16's [−65504, 65504] range. Uncompressed RGBA16F would cost 1024 × 1024 × 6 × 8 = 48 MB for a single 1024² cubemap (six faces); an outdoor scene with a handful of cubemaps blows past hundreds of MB. BC6H is the D3D11-era block format built specifically for HDR: 4×4 block, 16 bytes per block, 8 bpp (same size as BC7), but the payload is float16 RGB (no alpha). Its trick is 14 block modes that trade off precision differently — the encoder picks the mode best suited to that block's colour distribution. BC6H takes HDR cubemaps from RGBA16F's 64 bpp down to 8 bpp (8× compression) while keeping float16's dynamic range. That's the hardware foundation that lets PBR pipelines exist at scale today. Unreal Engine 4 / 5 and Unity HDRP default to BC6H for HDR cubemap assets.

技术内核

Technical core

BC6H 跟 BC1-5 不是同一类设计——它没有"统一 4 端点 + 内插"的简洁结构,而是 14 种块模式让编码器按块的颜色分布挑最优解。① 14 种块模式——每种模式给端点不同 bit 数(如 10-10-10、7-6-6-6、11-5-4-4 等三/四个分量)、是否启用 2 分区(把 4×4 块拆成两组,每组独立端点 + 内插,适用于块内有明显颜色边界的情况)、index 用 3-bit 还是 4-bit。编码器对每个块尝试多种模式,挑 PSNR 最高那个塞进 16 byte。② 端点用 float16 表示——这是 BC6H 区别于所有其他 BC 的核心。BC1-5 的端点是定点整数(RGB565 或 8-bit),只能表示 0-1;BC6H 的端点是浮点,可以表示 [-65504, 65504]——HDR 高光、太阳直射、自发光物体的真实数值都能装进去。③ UF16 (unsigned) vs SF16 (signed)——UF16 范围 [0, 65504],适合不会有负值的 HDR 颜色;SF16 范围 [-65504, 65504],适合可能有负值的 HDR 法线或其他工程数据。④ 4×4 块仍只 16 byte——这是工程上最重要的一点:BC6H 跟 BC7 一样是 8 bpp,HDR 的体积成本只比 LDR 多 1×(BC1 是 4 bpp,BC7 / BC6H 都是 8 bpp)。这个"HDR 不贵"的承诺让 IBL 反射探针 / cubemap 的大规模使用成为可能——Unreal Engine 默认每个室外场景烘焙几十张 BC6H cubemap。

BC6H isn't built like BC1-5 — there's no clean "two endpoints + interpolation" template. Instead, 14 block modes let the encoder pick the best fit for that block's colour distribution. ① 14 block modes — each mode allocates different bit counts to the endpoints (e.g. 10-10-10, 7-6-6-6, 11-5-4-4, three or four components), optionally enables 2-partition mode (split the 4×4 block into two regions, each with its own endpoints + interpolation, which helps when a block has a sharp colour boundary), and uses 3- or 4-bit indices. The encoder tries multiple modes per block and packs whichever maximises PSNR into the 16-byte block. ② Endpoints expressed as float16 — this is the one thing that sets BC6H apart from every other BCn. BC1-5 endpoints are fixed-point integers (RGB565 or 8-bit) capped at 0-1; BC6H endpoints are floating point and can express [−65504, 65504] — the actual numerical range of HDR highlights, direct sun, emissive surfaces. ③ UF16 (unsigned) vs SF16 (signed) — UF16's range is [0, 65504], suitable for non-negative HDR colour; SF16's is [−65504, 65504], suitable for HDR normals or other engineering data that may go negative. ④ 4×4 block, still just 16 bytes — and this is the most important engineering fact: BC6H is 8 bpp, the same as BC7. HDR costs only 1× more bytes than LDR (BC1 is 4 bpp, BC7 / BC6H are 8 bpp). That "HDR isn't expensive" promise is what makes large-scale IBL reflection probes and HDR cubemaps practical — Unreal Engine routinely bakes dozens of BC6H cubemaps per outdoor scene.

BC7 — 现代 BCn 的集大成

BC7 — the synthesis of modern BCn

"一种格式,八种块模式,自动挑最合适那种。"

"One format, eight block modes — pick whichever fits best."

BC1-5 各自只擅长一种场景:BC1 是 RGB 无 alpha、BC2 是 RGB + 锐利 alpha、BC3 是 RGB + 平滑 alpha、BC4 是单通道、BC5 是双通道。游戏纹理混合场景多——一张角色贴图可能同时有平滑 RGB 渐变 + 锐利 alpha 边缘 + 高频金属反光,任何单一 BCn 都解释不了整张。美术希望"一种格式覆盖所有"——不用每张图手动挑 BCn。BC7 的解法是 8 种内部块模式 + 编码器为每个 4×4 块自动挑最合适那种:同样 8 bpp(跟 BC2 / BC3 一样),但同图视觉质量比它们好 5-10×,几乎追上未压缩。BC7 因此成为 D3D11 时代之后桌面游戏纹理的事实唯一选择——AAA 游戏 90% 桌面贴图都用 BC7。

BC1-5 each excel at exactly one scenario: BC1 is RGB without alpha, BC2 is RGB + sharp alpha, BC3 is RGB + smooth alpha, BC4 is single-channel, BC5 is dual-channel. Real game textures mix scenarios — a single character map can carry smooth RGB gradients, sharp alpha edges and high-frequency metallic specular all at once, and no single BCn explains the whole thing. Artists want "one format that covers everything" without per-texture format picking. BC7's answer: 8 internal block modes plus an encoder that picks the best mode per 4×4 block. At the same 8 bpp as BC2 / BC3, BC7 looks 5-10× better visually — close to uncompressed. That's why, post-D3D11, BC7 became the de-facto only choice for desktop game textures: 90 % of AAA desktop textures are BC7.

技术内核

Technical core

BC7 的设计哲学跟 BC1-5 完全相反——BC1-5 是"一种结构覆盖一类场景",BC7 是"八种结构都做出来,让编码器临时挑"。① 8 种 mode (mode 0-7):每种 mode 内部不同的 (a) 区块切分(1 / 2 / 3 个子区,subsets——把 4×4 块拆成多组,每组独立端点 + 内插,适用于块内有明显颜色边界);(b) endpoint bit 分配(如 mode 1 给端点 6·6·6 高精度,mode 2 给 5·5·5 留更多 bit 给 index);(c) index bit width(2 或 3 或 4 bit,索引位越多越能精细内插);(d) 可选 p-bit(端点末位补一位精度)与 rotation(把 alpha 跟某个颜色通道交换,提升 alpha 精度)。② mode 0-3 偏 RGB 高质量,mode 4-7 偏 RGBA——RGB 模式给颜色更多 bit 但不要 alpha;RGBA 模式拨一些 bit 给 alpha 通道。这种"分工"让 BC7 既能当 BC1 的 RGB 升级,又能当 BC3 的 RGBA 升级,完全覆盖。③ 编码器枚举所有 mode 选最优——每个 4×4 块要对 8 mode × 几十种分区组合 × 端点优化跑一遍,计算 SSE(平方误差和),选 SSE 最低那个塞进 16 byte。这就是 BC7 编码慢的根本原因——典型 8K 纹理用 naive brute-force 要 40 分钟,Intel ISPC SIMD 后降到几秒。④ 8 bpp(同 BC2 / BC3,但视觉质量好 5-10×)——BC1 / BC4 是 4 bpp,BC7 / BC2 / BC3 / BC5 / BC6H 都是 8 bpp。BC7 跟 BC2 / BC3 同 bpp,胜在 mode 选择灵活,典型纹理 PSNR 高 +8-12 dB。⑤ 解码硬件原生——D3D11+ / GL 4.2+ / Vulkan / Metal 全平台支持,GPU sample 一个 BC7 texel 跟 sample 一个 RGBA8 一样快。这是 BC7 比"软件解码 + 上传"格式(如 KTX 装 zlib)的根本优势。

BC7's design philosophy inverts BC1-5: BC1-5 use one structure per scenario, BC7 ships eight structures and lets the encoder pick at runtime. ① 8 modes (mode 0-7), each varying along (a) partitioning (1, 2 or 3 subsets — splitting the 4×4 block into independent regions, useful when there's a sharp colour boundary inside the block); (b) endpoint bit allocation (mode 1 gives endpoints 6·6·6 high precision; mode 2 gives 5·5·5 and donates the saved bits to the index); (c) index bit width (2, 3 or 4 bits — more bits means finer interpolation); (d) optional p-bits (one extra LSB on the endpoints) and rotation (swap alpha with one of the colour channels to boost alpha precision when warranted). ② Mode 0-3 lean toward high-quality RGB; mode 4-7 lean toward RGBA — RGB modes give colour more bits with no alpha, RGBA modes shave bits off colour to fund alpha. That division of labour is what lets BC7 simultaneously upgrade BC1 (RGB) and BC3 (RGBA). ③ The encoder enumerates all modes and picks the optimum — for every 4×4 block it tries 8 modes × tens of partition combinations × endpoint optimisations, scores them by SSE (sum of squared error), and writes the best one into 16 bytes. This is the core reason BC7 encoding is slow: a typical 8K texture needs ~40 minutes with naive brute-force, dropping to seconds with Intel's ISPC SIMD encoder. ④ 8 bpp (the same as BC2 / BC3) with 5-10× better visual quality — BC1 / BC4 are 4 bpp; BC7 / BC2 / BC3 / BC5 / BC6H are all 8 bpp. At equal bpp BC7's mode-selection flexibility wins +8-12 dB PSNR over BC2 / BC3 on typical textures. ⑤ Hardware-native decoding — D3D11+ / GL 4.2+ / Vulkan / Metal all decode BC7 in silicon; sampling a BC7 texel costs the same as sampling RGBA8. That hardware-native sampling is BC7's fundamental advantage over "software-decode + upload" formats like KTX-with-zlib payloads.

$ nvtt_export --bc7 in.png -o out.dds            # NVIDIA Texture Tools, GPU-accelerated
$ ispc_texcomp -bc7 in.png out.dds               # Intel SIMD encoder, ~10× faster than naive
$ toktx --encode bc7 out.ktx2 in.png             # wrap into KTX2 (web / WebGPU friendly)
$ texconv -f BC7_UNORM in.png                    # Microsoft DirectXTex CLI
$ Compressonator.exe -fd BC7 in.png out.dds      # AMD Compressonator

ETC1 — Android 早期标准

ETC1 — the early Android standard

"OpenGL ES 时代第一个免专利的块压缩。"

"The first patent-free block codec of the OpenGL ES era."

2005 年 Khronos 在为 OpenGL ES 标准化纹理压缩时遇到一个棘手问题——S3TC(BC1-3)效果好但被 S3 Graphics 申请了一堆专利,Khronos 不可能把"必须授权才能用"的格式塞进开放标准。Ericsson Research 提了 ETC1(Ericsson Texture Compression),声明免专利,正好填上空缺,跟着 OpenGL ES 2.0(2007)一起进入 Android 强制基线。Android 从此在游戏纹理上有了统一格式——美术不必为不同 GPU 厂商分别打包,Mali / Adreno / PowerVR / Tegra 全都能解 ETC1。代价是 ETC1 没有 alpha 通道,任何带透明度的资产(UI 图标、粒子、角色边缘)都要拆成"RGB 用 ETC1 + alpha 用 8-bit 灰度图"两份纹理上传——显存和带宽都要付双份钱。这是 ETC2 在 2013 年出生的根本原因。但回到 2005,免专利 + GLES 2.0 强制 = ETC1 一夜之间成了 Android 游戏纹理事实标准。Angry Birds(2009)、Cut the Rope(2010)这一代手机游戏的纹理资产几乎全是 ETC1。

In 2005, while Khronos was standardising texture compression for OpenGL ES, it ran into a thorny problem — S3TC (BC1-3) worked beautifully but was wrapped in patents owned by S3 Graphics, and an open standard couldn't mandate "must license to use" formats. Ericsson Research proposed ETC1 (Ericsson Texture Compression), declared it patent-free, and it slotted neatly into the gap, riding alongside OpenGL ES 2.0 (2007) into the Android mandatory baseline. Suddenly Android had a single texture format every artist could ship — no need to repackage per vendor, since Mali, Adreno, PowerVR and Tegra all decoded ETC1. The price was that ETC1 had no alpha channel, so anything translucent (UI icons, particles, character edges) had to be split into "RGB as ETC1 + alpha as an 8-bit greyscale map" — two texture uploads, double the VRAM and bandwidth. That is exactly why ETC2 was born in 2013. But back in 2005, patent-free + GLES 2.0 mandatory equals ETC1 becoming the de-facto Android texture standard overnight. Angry Birds (2009) and Cut the Rope (2010) — that generation of mobile games — shipped almost their entire texture base in ETC1.

技术内核

Technical core

ETC1 的设计是"把 BC1 的思路换一种几何切分,绕开专利"。① 4×4 块切两半——不像 BC1 把 4×4 当整体处理,ETC1 把块切成上下 2×4 或左右 4×2 两半(块头有 1 bit 标记 flip 方向),每半独立有自己的颜色 base + modifier。这是 ETC1 跟 BCn 最大的几何差异——BCn 块是统一的 16 像素插值,ETC1 是两组 8 像素插值。② RGB444 base + 16 行 modifier 表——每半的 base color 只有 12 bit(RGB444),精度比 BC1 的 RGB565 还低;但靠 modifier 表补救——3 bit 选 16 行预设里的一行,每行给出 4 个亮度偏移值(如 ±2 / ±8 这种"小幅"组,或 ±42 / ±183 这种"大幅"组),覆盖从平滑渐变到硬边缘的不同需求。③ 2-bit/像素 index——每像素再用 2 bit 选 modifier 行里的 4 个偏移值之一,加到 base color 上得到最终颜色。换言之 ETC1 的颜色计算是"base ± modifier",只在亮度方向上调,色相不变——这意味着 ETC1 处理彩色高频细节(花布、彩色噪点)很差,但处理"单色平滑+亮度变化"(皮肤、墙面、地形)很好。④ 没有 alpha——这是 ETC1 最致命的局限。Android 游戏的解决方案是"双纹理上传":RGB 用 ETC1,alpha 用单通道 8-bit 灰度图(或 ETC1 的另一个块当 alpha 用,叫 ETC1+A 的 hack)。⑤ 每块 8 byte / 16 像素 = 4 bpp——跟 BC1 同体积。质量略差于 BC1(因为色相方向死板),但免专利 = 能强制进 GLES 标准,这是 BC1 做不到的。

ETC1's design is "use a different geometric split from BC1 to dodge the patents". ① 4×4 block split into two halves — unlike BC1, which treats the 4×4 as one unit, ETC1 splits the block into two 2×4 halves (or two 4×2 halves; the block header carries a single flip bit). Each half independently owns its colour base + modifier. That's the biggest geometric difference from BCn: BCn's block is one 16-pixel interpolation; ETC1's is two 8-pixel interpolations. ② RGB444 base + a 16-row modifier table — each half's base colour is only 12 bits (RGB444), even less precise than BC1's RGB565; the modifier table makes up the difference. Three bits pick one of 16 preset rows, each row carrying four brightness offsets (a "fine" set like ±2 / ±8, a "coarse" set like ±42 / ±183), covering everything from smooth gradients to hard edges. ③ 2 bits per pixel for the index — each pixel picks one of the four offsets in the chosen modifier row and adds it to the base colour, producing the final value. ETC1's colour math is therefore "base ± modifier" — adjustment only along brightness, never along hue. That makes ETC1 poor on coloured high-frequency detail (patterned cloth, coloured noise) and excellent on monochrome-plus-brightness signals (skin, walls, terrain). ④ No alpha — ETC1's most fatal limitation. The Android workaround was the "two-texture upload": RGB as ETC1, alpha as a single-channel 8-bit greyscale map (or a second ETC1 block reused as alpha — the "ETC1+A" hack). ⑤ 8 bytes per 16-pixel block = 4 bpp, the same footprint as BC1. Quality lags BC1 slightly (because hue can't move) but ETC1 is patent-free, which lets it become a mandatory part of the GLES standard — something BC1 could never be.

ETC2 / EAC — alpha 加成

ETC2 / EAC — adding alpha

"ETC1 加上 alpha 通道,正好赶上 OpenGL ES 3.0。"

"ETC1 with alpha — just in time for OpenGL ES 3.0."

ETC1 在 Android 上跑了 6 年(2007-2013),但"没有 alpha"这个缺陷越用越疼。任何带透明度的资产——UI 图标、HUD、粒子系统、抠图角色——都要拆成两份纹理上传:RGB 用 ETC1(4 bpp),alpha 用 8-bit 灰度图(8 bpp),合计 12 bpp,显存和带宽是单纹理的 3 倍。手机游戏的 UI 又特别多透明元素,这个负担实打实地让中低端 Android 设备跑不动。2013 年 Khronos 正式推 ETC2 / EAC——保持向下兼容(老的 ETC1 块在 ETC2 解码器里能直接用),同时加入 RGBA 模式(ETC2 RGB 块 + EAC alpha 块,共 16 byte = 8 bpp)。ETC2 还顺手补齐了 R11 / RG11 单/双通道格式(对应桌面的 BC4 / BC5,用于法线贴图、roughness 等),让移动端也有了完整的"通道拆分"工具箱。最重要的政治决定:Khronos 把 ETC2 定成 OpenGL ES 3.0 的强制基线——任何宣称支持 GLES 3.0 的 GPU 都必须解码 ETC2。这意味着 2014 年之后的 Android 游戏可以放心地"全资产 ETC2",不再需要为"老设备没 ETC2"留 fallback。Unity / Unreal 在 2014 年都把 Android 默认纹理改成了 ETC2。

ETC1 ran on Android for six years (2007-2013), but "no alpha" hurt more every year. Anything translucent — UI icons, HUDs, particle systems, alpha-masked characters — had to upload two textures: RGB as ETC1 (4 bpp) plus alpha as an 8-bit greyscale map (8 bpp), 12 bpp combined and roughly 3× the bandwidth of a single texture. Mobile UI is unusually heavy on translucent elements, and that overhead measurably broke mid- to low-end Android devices. In 2013 Khronos shipped ETC2 / EAC: keep ETC1 backward compatibility (legacy ETC1 blocks decode unchanged in an ETC2 decoder) and add an RGBA mode (an ETC2 RGB block + an EAC alpha block, 16 bytes = 8 bpp total). ETC2 also rounded out single- and dual-channel formats with R11 / RG11 (the mobile counterparts to desktop BC4 / BC5 — normals, roughness, etc.), giving mobile its own full "channel-split" toolbox. The crucial political decision: Khronos made ETC2 a mandatory baseline for OpenGL ES 3.0. Any GPU that claims GLES 3.0 support must decode ETC2. Post-2014 Android games could finally ship all-ETC2 with no "device might not have ETC2" fallback. Unity and Unreal both flipped their Android default to ETC2 in 2014.

技术内核

Technical core

ETC2 的设计哲学是"在 ETC1 上做加法,不做减法"——所有 ETC1 块在 ETC2 解码器里都能正常工作(向下兼容),新增的能力都通过"block 头部模式位"切换。① RGB 块 4 种模式:(a) ETC1 兼容模式(老的"两半 base + modifier" 结构,8 byte);(b) T-mode(把 4×4 块按 T 形分成两个颜色区,适合块内有 L 形 / T 形硬边);(c) H-mode(把块按 H 形分两区,适合垂直硬边);(d) Planar mode(用三个角点的颜色定义平面,块内每像素从平面采样,适合平滑渐变如皮肤、天空)。每块的头部 1 bit 指明用哪种模式,编码器为每块挑最优。② RGBA8 = ETC2 RGB block + EAC alpha block——一块 16 byte,前 8 byte 是 ETC2 RGB,后 8 byte 是 EAC(Ericsson Alpha Compression)alpha 块。EAC alpha 块用 8 个端点 + 内插值 + 3-bit/像素 index,提供 11-bit 等效精度,远高于 BC3 alpha 块的 8-bit。③ R11 / RG11——独立的单/双 11-bit 通道格式,对应桌面的 BC4 / BC5,用于法线贴图(RG11)、高度图 / roughness(R11)等。R11 是 8 byte/块 = 4 bpp,RG11 是 16 byte/块 = 8 bpp。④ punch-through alpha——一种特殊模式叫 ETC2 RGBA1(RGB8_PUNCHTHROUGH_ALPHA1_ETC2),只允许 alpha = 0 或 255 的硬切边(像 BC1 的 1-bit alpha),用于树叶、栅栏这种"完全透明 / 完全不透"的资产,体积仍是 4 bpp。⑤ OpenGL ES 3.0 强制 = 不需要 fallback——这是 ETC2 最大的工程优势。BC1-7 在桌面是"硬件支持但要查 capability",ETC2 在 Android GLES 3.0+ 是"必然存在"。Unity / Unreal 因此在 2014 年果断把 Android 默认纹理改成 ETC2。

ETC2's design philosophy is "add to ETC1, never subtract" — every ETC1 block decodes correctly in an ETC2 decoder (backward compatibility), and new capabilities are gated behind block-header mode bits. ① The RGB block has four modes: (a) ETC1-compatible (the legacy "two halves, base + modifier" structure, 8 bytes); (b) T-mode (the 4×4 block split into two colour regions in a T shape — handy for blocks with L- or T-shaped hard edges); (c) H-mode (split into two regions in an H shape — for vertical hard edges); (d) Planar mode (three corner colours define a plane, every pixel is sampled from that plane — for smooth gradients like skin and sky). One bit in the block header chooses the mode; the encoder picks per-block. ② RGBA8 = ETC2 RGB block + EAC alpha block — 16 bytes per block: the first 8 are ETC2 RGB, the next 8 are EAC (Ericsson Alpha Compression). The EAC alpha block carries 8 endpoints + interpolated values + 3-bit per-pixel index, delivering an effective 11-bit precision — far above BC3 alpha's 8-bit. ③ R11 / RG11 — standalone single- and dual-channel 11-bit formats, the mobile counterparts to desktop BC4 / BC5, used for normal maps (RG11), height / roughness maps (R11), etc. R11 is 8 bytes per block = 4 bpp; RG11 is 16 bytes = 8 bpp. ④ Punch-through alpha — a special mode called ETC2 RGBA1 (RGB8_PUNCHTHROUGH_ALPHA1_ETC2) only allows alpha = 0 or 255 hard cut-outs (like BC1's 1-bit alpha), targeted at foliage / fences / "fully on or fully off" assets at 4 bpp. ⑤ OpenGL ES 3.0 mandatory = no fallback needed — and that is ETC2's biggest engineering advantage. On desktop, BC1-7 are "hardware-supported but capability-checked"; on Android with GLES 3.0+, ETC2 is guaranteed to exist. That is exactly why Unity and Unreal flipped the Android default to ETC2 in 2014.

PVRTC — Apple 早期独占

PVRTC — Apple's early proprietary lock-in

"PowerVR 的私有方案,iPhone 一代到 7 代的纹理本命。"

"PowerVR's proprietary scheme — texture-of-life for iPhone 1 through 7."

PVRTC 的诞生跟一个非常特定的硬件架构绑定:Imagination Technologies 的 PowerVR GPU 用的是 TBDR(Tile Based Deferred Rendering,基于瓦片的延迟渲染)——把屏幕切成小瓦片(典型 32×32 像素),每个瓦片独立渲染、合成,显著省功耗(手机的核心需求)。问题是 TBDR 处理瓦片时,纹理 sample 经常跨瓦片边界,如果纹理压缩格式是"块独立"的(像 BC1 / ETC1 那种,每个 4×4 块独立解码),瓦片边界处会出现明显的"块状不连续"(blocky artifact)。Imagination 在 2003 年提出 PVRTC 解决这个问题:不存"每块独立的颜色",而是存两层"低分辨率的颜色信号" + 一个"调制信号"——运行时 GPU 在采样点对两层信号做双线性插值,然后用调制信号在两个插值结果之间混合。这样块之间天然连续,没有边界 artifact——完美适配 TBDR。代价是 PVRTC 是私有格式,只有 PowerVR GPU 能解。但 Apple iPhone 1(2007)到 iPhone 7(2016)全部用 PowerVR GPU,所以 PVRTC 是 iOS 游戏的唯一标准纹理格式近十年。Infinity Blade、Real Racing、Monument Valley 一代游戏的纹理资产基本全是 PVRTC。iPhone 8 / iPad Pro A10(2017)改用 Apple 自研 GPU,默认 ASTC,PVRTC 进入历史。

PVRTC's birth is tied to one very specific hardware architecture: Imagination Technologies' PowerVR GPUs use TBDR (Tile Based Deferred Rendering), which slices the screen into small tiles (typically 32×32 pixels), renders and composites each tile independently, and saves significant power — the core mobile requirement. The trouble is that during tile processing, texture samples regularly cross tile boundaries; if the texture format is "block independent" (like BC1 / ETC1, each 4×4 block decoded in isolation), tile boundaries grow visible "blocky" artifacts. In 2003 Imagination proposed PVRTC to solve this. Instead of storing "independent colour per block", PVRTC stores two layers of low-resolution colour signals plus a modulation signal — at sample time the GPU bilinearly interpolates both colour layers, then blends the two interpolated results using the modulation signal. Blocks are naturally continuous across boundaries — no block artifacts, a perfect TBDR fit. The price is that PVRTC is proprietary, decodable only on PowerVR GPUs. But every iPhone from the iPhone 1 (2007) through the iPhone 7 (2016) shipped with a PowerVR GPU, so PVRTC was the de facto sole texture standard on iOS for nearly a decade. Infinity Blade, Real Racing and Monument Valley — that generation of iOS games — basically shipped their entire texture base as PVRTC. The iPhone 8 / iPad Pro A10 (2017) switched to Apple's own GPU, defaulting to ASTC, and PVRTC slid into history.

技术内核

Technical core

PVRTC 的技术结构跟 BCn / ETCn 完全是另一条思路——它不做"每块独立解码",而是用"全图低分辨率信号 + 调制图"的方案。① 两个低分辨率 RGB 层 + 一个调制层——记原图分辨率 W×H,PVRTC 把它编码为:(a) 信号 A,分辨率 (W/4)×(H/4)(2 bpp 模式)或 (W/4)×(H/2)(4 bpp 模式),每个采样点存 RGB 端点;(b) 信号 B,跟 A 同分辨率,存另一组 RGB 端点;(c) 调制信号 mod,跟原图同分辨率,每像素 1 bit(2 bpp 模式)或 2 bit(4 bpp 模式)指明 A 和 B 的混合比例。② 采样时的实际运算:GPU 对 A、B 各自做双线性插值得到 colorA、colorB,再用 mod 在两者之间混合。这不是块独立——同一个像素的颜色受周围 4 个 A 端点 / 4 个 B 端点的影响,块边界因此天然平滑过渡。③ 块尺寸 8×4(2 bpp)或 4×4(4 bpp)——两种码率档:2 bpp 是 8×4 块 / 8 byte = 2 bpp(注意是 8 byte/块,跟 BC1 同 byte 数但块更大,所以 bpp 减半),4 bpp 是 4×4 块 / 8 byte。④ 原生 alpha——比 ETC1 强,能直接装 RGBA 数据(虽然质量略差于 BC3 / BC7)。⑤ "分辨率必须是 2 的幂 + 正方形 + ≥8×8"——PVRTC v1 的硬限制。这个限制源于"信号 A、B 必须能均匀采样到原图所有像素"。PVRTC2(2009)放宽了这个限制(支持任意宽高 + punch-through alpha),但 PVRTC2 的硬件支持远不如 v1 普及。⑥ PowerVR 独占解码硬件——这是 PVRTC 同时是它的优势和坟墓。优势:iPhone 1-7 全部 PowerVR,PVRTC 在 iOS 游戏里是"必然支持";坟墓:其他 GPU 不解 PVRTC,Android 设备完全用不了,跨平台游戏要分别打包 PVRTC(iOS)+ ETC2(Android)两份纹理。Apple 在 A10(2017)改用自研 GPU(基于 Imagination 但有改造),iOS 11+ 推荐 ASTC 后,PVRTC 就停止发展了。Imagination Technologies 也在 2017 年因 Apple 流失被收购,PVRTC 实际上跟着公司一起进入历史。

PVRTC's technical structure is on a completely different track from BCn / ETCn — it doesn't do "decode each block in isolation"; it uses "global low-resolution signals + a modulation map." ① Two low-resolution RGB layers plus one modulation layer — given source resolution W×H, PVRTC encodes: (a) signal A at (W/4)×(H/4) (2 bpp mode) or (W/4)×(H/2) (4 bpp mode), each sample storing an RGB endpoint; (b) signal B, same resolution as A, holding another RGB endpoint set; (c) modulation signal mod, at the source's full resolution, with 1 bit/pixel (2 bpp mode) or 2 bits/pixel (4 bpp mode) specifying the blend ratio between A and B. ② Sample-time arithmetic: the GPU bilinearly interpolates A and B independently to produce colourA and colourB, then uses mod to blend them. This is not block-independent — a single pixel's colour depends on the surrounding 4 A endpoints + 4 B endpoints, so block boundaries transition smoothly by construction. ③ Block size 8×4 (2 bpp) or 4×4 (4 bpp) — two bitrate tiers. The 2 bpp variant uses 8×4 blocks at 8 bytes per block (note: same bytes-per-block as BC1, but the block is larger, so bpp halves); the 4 bpp variant is 4×4 blocks at 8 bytes. ④ Native alpha — stronger than ETC1, can carry RGBA directly (though with somewhat lower quality than BC3 / BC7). ⑤ "Power-of-two, square, ≥ 8×8" — PVRTC v1's hard requirement, rooted in the need for signals A and B to sample uniformly onto every source pixel. PVRTC2 (2009) relaxed this (arbitrary aspect ratios + punch-through alpha), but PVRTC2 hardware support never reached v1's ubiquity. ⑥ PowerVR-exclusive decode hardware — both PVRTC's strength and its tomb. The strength: every iPhone 1-7 had a PowerVR GPU, so PVRTC was guaranteed-supported on iOS. The tomb: no other GPU decodes PVRTC, so Android couldn't use it at all, and cross-platform games had to ship two texture builds — PVRTC (iOS) + ETC2 (Android). When Apple's A10 (2017) moved to in-house GPUs (originally Imagination-derived, but heavily modified), and iOS 11+ recommended ASTC, PVRTC stopped evolving. Imagination Technologies itself was acquired in 2017 after losing the Apple business; PVRTC effectively went into the history books with the company.

ASTC — 可变块大小的现代之王

ASTC — the modern king of variable block size

"4×4 还是 12×12?同一个格式,你自己挑。"

"4×4 or 12×12? Same format — you choose."

BCn / ETC2 都是固定 4×4 块,bpp 永远 4 或 8——只能"全图统一档位"。但游戏里的纹理质量需求从来不是一档的:UI 图标、角色脸需要高质量(密集块、高 bpp);远处地形、天空盒可以低质量(稀疏块、低 bpp)。美术希望一种格式同时支持"质量/体积"光谱滑块——同样一个文件结构,从 8 bpp 一路滑到 1 bpp。ARM 主导设计 ASTC(Adaptive Scalable Texture Compression),提供 14 种块大小(4×4 至 12×12),bpp 从 8 降到 0.89——同一格式覆盖近 9× 体积范围。Khronos 在 2012 年通过标准化(GLES 3.2 强制 + Vulkan 默认 + Apple A8 起原生支持),ASTC 成为现代移动 + WebGPU 的事实之王。BC7 守桌面、ASTC 守移动——这是 GPU 纹理压缩 2010 年代后的两强格局。

BCn / ETC2 are fixed at 4×4 blocks; bpp is locked at 4 or 8 — every texture in a project must pick one tier for the whole image. But real game textures need a spectrum: UI icons and character faces want high quality (dense blocks, high bpp), while distant terrain and skyboxes can run low quality (sparse blocks, low bpp). Artists want one format that exposes a quality / size dial — the same file structure sliding from 8 bpp down to 1 bpp. ARM led the design of ASTC (Adaptive Scalable Texture Compression), shipping 14 block sizes from 4×4 to 12×12, with bpp dropping from 8 to 0.89 — one format spanning nearly a 9× size range. Khronos standardised it in 2012 (mandatory in GLES 3.2, default in Vulkan, native on Apple from A8 onward), and ASTC became the de-facto king of modern mobile and WebGPU. BC7 owns desktop, ASTC owns mobile — that's the post-2010s duopoly of GPU texture compression.

技术内核

Technical core

ASTC 的设计哲学是"一格框架,无限档位"——所有块共用 16 byte 容器,但内部组件按块大小重新分配比例,让格式从 8 bpp 一路滑到 0.89 bpp。① 14 种块大小:4×4 / 5×4 / 5×5 / 6×5 / 6×6 / 8×5 / 8×6 / 8×8 / 10×5 / 10×6 / 10×8 / 10×10 / 12×10 / 12×12——LDR profile 全部支持,HDR profile 仅前 8 种。还有 3D 体素扩展(用 3×3×3 等 11 种 3D 块)。② 每块固定 16 byte——这是 ASTC "档位光谱"的根本机制:容器不变,块越大(像素更多)→ 每像素分到的 bit 越少 → bpp 越低。比如 4×4 块 = 16 px / 16 byte = 8 bpp;12×12 块 = 144 px / 16 byte = 0.89 bpp。同样的解码硬件、同样的文件结构,档位却覆盖近 9× 体积差。③ 13 种 endpoint 编码格式 (CEM):LDR int 8/16/24-bit、HDR float、单/双/三平面(单平面 = RGB 共享一组端点;三平面 = RGB 各自独立端点,适合高频彩色细节)。每块在 endpoint mode 字段内挑一种,精确匹配局部像素分布。④ 权重平面 + 双权重平面:基本 ASTC 用一张权重图控制所有通道的内插;双权重平面(dual-plane)模式让 alpha 或某一颜色通道走独立权重——类比 BC7 的 "rotation",但更通用,在彩色 + 高频 alpha 混合贴图上质量明显更好。⑤ HDR + 3D 双扩展——LDR profile(主流硬件全支持)给颜色 0-1 范围;HDR profile(部分硬件)给 float 范围,直接当移动版 BC6H 用;3D profile(更小众)给体素纹理(医疗影像、烟雾模拟、地形 3D 噪声)。⑥ 权重网格大小可独立于块大小——一个 12×12 块的权重网格可以是 4×4(更稀疏,更节省 bit 给 endpoint),也可以是 8×8(更密,牺牲 endpoint 精度换插值精度)。这是 ASTC 比 BC7 更灵活的核心,编码器要在"块大小 × endpoint mode × 权重网格"三维空间搜索最优。⑦ 编码极慢——astcenc 参考编码器是 brute-force 搜全部组合,单图 6×6 thorough 模式可能要几分钟。但解码硬件原生,sample 一个 ASTC texel 跟 sample BC7 一样快。

ASTC's design philosophy is "one frame, infinite tiers" — every block shares a 16-byte container, but the internal allocation re-balances by block size, sliding the format from 8 bpp all the way to 0.89 bpp. ① 14 block sizes: 4×4 / 5×4 / 5×5 / 6×5 / 6×6 / 8×5 / 8×6 / 8×8 / 10×5 / 10×6 / 10×8 / 10×10 / 12×10 / 12×12 — all supported by the LDR profile; HDR is limited to the first 8. A 3D extension also defines 11 voxel block sizes. ② Every block is exactly 16 bytes — the mechanism behind ASTC's tier spectrum. The container stays constant; the bigger the block (more pixels packed in), the fewer bits per pixel, and the lower the bpp. 4×4 = 16 px / 16 bytes = 8 bpp; 12×12 = 144 px / 16 bytes = 0.89 bpp. Same decode hardware, same file layout, nearly 9× size difference between extremes. ③ 13 endpoint encodings (CEM): LDR int at 8 / 16 / 24-bit, HDR float, with one / two / three planes (one-plane = RGB share endpoints; three-plane = each colour channel has independent endpoints, ideal for high-frequency colour). Each block picks one CEM in its endpoint-mode field to match the local pixel distribution. ④ Weight plane + dual weight plane: basic ASTC uses one weight grid controlling all channels' interpolation; dual-plane mode lets alpha or one colour channel travel on an independent weight grid — analogous to BC7's "rotation" but more general, and visibly better on mixed colour + high-frequency-alpha maps. ⑤ HDR + 3D extensions — the LDR profile (universally supported) covers colour in [0, 1]; the HDR profile (partial hardware support) gives float range and effectively serves as mobile BC6H; the 3D profile (more niche) targets voxel textures (medical imaging, smoke simulation, 3D terrain noise). ⑥ Weight grid size independent of block size — a 12×12 block can use a 4×4 weight grid (sparser, donating bits to endpoints) or an 8×8 grid (denser, trading endpoint precision for interpolation precision). This is what makes ASTC more flexible than BC7: the encoder searches a three-dimensional space of "block size × endpoint mode × weight grid." ⑦ Encoding is brutally slow — astcenc, the reference encoder, brute-forces the whole combination space; a single image at 6×6 with the thorough preset can take minutes. But decoding is hardware-native — sampling an ASTC texel costs the same as sampling BC7.

$ astcenc -cl in.png out.astc 6x6 -medium         # LDR profile, 6×6 block, medium preset
$ astcenc -cs in.png out.astc 6x6 -thorough       # thorough = brute-force search, much slower / better
$ astcenc -ch in.exr out.astc 6x6 -medium         # HDR profile (input EXR float)
$ toktx --astc 6x6 out.ktx2 in.png                # wrap into KTX2 (web / WebGPU friendly)
$ toktx --encode astc --astc_blk_d 6x6 out.ktx2 in.png  # explicit block-size flag

Basis Universal — 一次编码、多平台转码

Basis Universal — encode once, transcode anywhere

"一份 .basis,运行时按设备转 BC7、ETC2 或 ASTC。"

"One .basis file — transcoded at runtime to BC7, ETC2 or ASTC depending on the device."

Web 和跨平台游戏一直有个尴尬:桌面要 BC7、Android 要 ETC2、现代移动要 ASTC、老 iOS 要 PVRTC——同一张纹理要打四份,资产包体积爆炸,CDN 流量翻倍,管理痛苦不堪。Rich Geldreich(前 Valve、Crunch 作者、桌面纹理压缩领域的活字典)在 2018 年提出"中间格式"思路:编码时存为 Basis Universal(一种紧凑的 IR——intermediate representation),运行时用 JS 或 Wasm 解码到目标设备的块格式。一份资产 → 任意设备。Khronos 接受捐赠后,Basis 成为 KTX2 supercompression scheme 的事实标准,glTF 2.0 把 KTX2 + Basis 列为推荐的纹理 payload。three.js / Babylon.js / Unity WebGL / godot Web 全都内置 Basis transcoder。Web 端从此告别"打四份纹理"的时代。

Web and cross-platform games long suffered an awkward problem: desktop needs BC7, Android needs ETC2, modern mobile wants ASTC, legacy iOS wants PVRTC — the same texture has to be packed four ways, asset bundles balloon, CDN traffic doubles, and asset management becomes a nightmare. In 2018 Rich Geldreich (ex-Valve, author of Crunch, a living encyclopedia of desktop texture compression) proposed an "intermediate format" approach: encode once into Basis Universal (a compact IR — intermediate representation), then at runtime use JS or Wasm to transcode to whatever block format the target device wants. One asset → any device. After Geldreich donated the project to Khronos, Basis became the de-facto KTX2 supercompression scheme; glTF 2.0 lists KTX2 + Basis as the recommended texture payload. three.js / Babylon.js / Unity WebGL / godot Web all ship the Basis transcoder. The "pack four textures" era of the Web ended here.

技术内核

Technical core

Basis 的核心是"中间表示 + 运行时转码"——既不像 BCn/ASTC 那样直接是 GPU 块格式,也不像 PNG/JPEG 那样是 CPU 像素流,而是一种专门设计来"几乎零成本转码到任何块格式"的紧凑中间形态。① 两个 profile:ETC1S 基于 ETC1 的色彩端点结构,每块 ~2 bpp,体积极小,质量约相当于 JPEG 中等;UASTC("Universal ASTC")基于 ASTC 4×4 的子集,每块 8 bpp,质量约等于 ASTC 4×4 / BC7。两档对应"小贴图随便堆"和"重要纹理用高质量"。② 编码后再用 supercompression 压一遍——ETC1S 用 LZ-style + RDO(rate-distortion optimisation)再压缩 30-50%,UASTC 用 Zstd 压缩 ~30%。最终 .basis / KTX2 文件比裸 BCn 还小,比 PNG/JPEG 慢一点解码但能直接送给 GPU。③ 运行时转码极快——transcoder 是设计成 O(blocks) 的简单查表 + 位重排,Wasm 实现单核能跑 几百 MB/s,比 PNG 解码快一个数量级。这是 Basis 跟传统"在线解码 PNG → CPU RGBA → uploadTexture"路径的根本区别——后者占 CPU + 占带宽 + 占显存,前者一步到位送 GPU 块格式。④ 支持目标:BC1 / BC3 / BC4 / BC5 / BC7 / ETC1 / ETC2 / ASTC 4×4 / PVRTC1 / PVRTC2 / RGBA32(无块格式硬件兜底)——基本覆盖现役所有 GPU。

Basis's core idea is "intermediate representation + runtime transcode" — it is neither a direct GPU block format like BCn / ASTC, nor a CPU pixel stream like PNG / JPEG, but a compact intermediate form deliberately designed to transcode to any block format at almost zero cost. ① Two profiles: ETC1S is built on ETC1's colour-endpoint structure, ~2 bpp per block, extremely small, with quality roughly on par with mid-quality JPEG; UASTC ("Universal ASTC") is built on a subset of ASTC 4×4, 8 bpp per block, with quality close to ASTC 4×4 / BC7. The two tiers map to "stack lots of small textures" vs "use high quality on important textures." ② The encode is then run through supercompression — ETC1S uses an LZ-style codec plus RDO (rate-distortion optimisation) and shrinks another 30–50 %; UASTC uses Zstd for about 30 %. The resulting .basis / KTX2 file is smaller than raw BCn, decodes a touch slower than PNG / JPEG, but ships straight to the GPU. ③ Runtime transcoding is blazing fast — the transcoder is engineered as O(blocks) with simple table lookups and bit re-shuffling; the Wasm build hits hundreds of MB/s on a single core, an order of magnitude faster than PNG decoding. That's the fundamental difference between Basis and the traditional "decode PNG → CPU RGBA → uploadTexture" path: the latter eats CPU + bandwidth + VRAM, while the former hands the GPU a block format in one step. ④ Supported targets: BC1 / BC3 / BC4 / BC5 / BC7 / ETC1 / ETC2 / ASTC 4×4 / PVRTC1 / PVRTC2 / RGBA32 (an uncompressed fallback for hardware without block formats) — effectively every GPU in service.

Crunch — 在 BC 体积上再砍一半

Crunch — halving BC's size with a second pass

"先 cluster 再 BC1 — 在 BC 体积上再砍一半。"

"Cluster first, then BC1 — halve the BC size."

2010s 初期移动 + 主机游戏的纹理资产包动辄几百 MB,主要是 BC1/BC3 块的累积——iOS App Store 限制单包 < 2 GB,主机光盘也是有限介质。Rich Geldreich 在 2012 年观察到一个事实:大量 4×4 块其实彼此相似——同一张草地纹理里成千上万个块都是"绿色为主、轻微噪点变化",同一面墙的砖块色调几乎一致。如果这些块共用一个码本(codebook),只存"指向码本的索引 + 微小偏移",体积可以再砍一半。Crunch 把这个想法落地:对所有块做 k-means 聚类,然后用 RC(range coder)+ Huffman 二次熵编码 BC 字典——磁盘体积比裸 BCn 再小 30-50%。运行时只需在 CPU 上花几十毫秒解回普通 BCn,再上传给 GPU。这是"BC 之上还能再压"的第一个工程化实践。后来同作者 6 年后用同样思路做了 Basis Universal,覆盖更广 GPU 块格式 + 加上 transcode 维度——Crunch 进入历史。

By the early 2010s, mobile and console game texture bundles had ballooned to hundreds of MB, mostly accumulated BC1/BC3 blocks — the iOS App Store capped single bundles at 2 GB, and console discs are finite media. In 2012 Rich Geldreich noticed an obvious truth: most 4×4 blocks are similar to each other — a grass texture has thousands of blocks that are all "mostly green with mild noise"; the bricks on a wall share an almost identical palette. If those blocks shared a single codebook and we only stored "codebook index + small offset," size could be halved again. Crunch put that idea into practice: run k-means clustering across all blocks, then run RC (range coder) + Huffman as a second-pass entropy code over the BC dictionary — on disk the result is 30–50 % smaller than raw BCn. At runtime the CPU spends tens of milliseconds decoding back to ordinary BCn, then uploads it to the GPU. It was the first engineering-grade demonstration of "compressing on top of BC." Six years later the same author took the idea further, covering more GPU block formats plus an extra transcode dimension — that became Basis Universal, and Crunch quietly walked off into history.

技术内核

Technical core

Crunch 的工程实现只有两步,但每步都精妙。① k-means cluster:把所有 4×4 块当成一个 64-bit 高维向量样本(BC1 块结构 = 2 个 16-bit 端点 + 32-bit 4-color 索引),用 k-means 在 BC 块空间内聚类成 N 个代表块(典型 N = 1024);每块只存"代表块索引 + 局部偏移量"。这一步把"每块 64 bit 独立"变成"每块 ~10 bit 索引 + 小残差",体积压缩比通常 4-6 倍,但因为 BCn 本身已经是有损,残差很小,质量损失可控。② RC + Huffman 二次熵编码:对 codebook 自身(1024 × 64 bit = 8 KB)和 index 流(~10 bit × 块数)再用 range coder + Huffman 树压缩——index 流通常有强自相关(同一区域的相邻块很可能落在同一 cluster),熵很低,Huffman 能再砍 30-50%。最终 .crn 文件平均比裸 BCn 小 50%,跟 PNG 体积差不多但能直接送 GPU(还要先在 CPU 上 swizzle 回 BCn,有几十 ms 解码延迟)。运行时解码是 streaming 的——可以一边读文件一边解块,不需要一次加载整张图——这是 Crunch 设计上对 mmap 友好的一个细节。

Crunch's engineering implementation has just two steps, but each is delicately tuned. ① k-means clustering: treat every 4×4 block as a 64-bit high-dimensional vector (a BC1 block = two 16-bit endpoints + a 32-bit 4-colour index), then run k-means in BC-block space to find N representative blocks (typically N = 1024); each block stores "representative-block index + local offset." This step turns "64 independent bits per block" into "about 10 bits of index + a tiny residual," giving a 4–6× size compression — and because BCn is already lossy, the residual is small and quality loss stays controlled. ② Second-pass RC + Huffman entropy coding: the codebook itself (1024 × 64 bits = 8 KB) and the index stream (~10 bits × block count) go through a range coder plus Huffman tree — the index stream is strongly auto-correlated (adjacent blocks in the same region almost always fall in the same cluster), entropy is low, and Huffman shaves another 30–50 %. The resulting .crn file averages 50 % smaller than raw BCn, comparable in size to PNG but ready to ship to the GPU (you do still need a CPU swizzle back to BCn first, costing tens of ms of decode latency). Runtime decode is streaming — blocks can be decoded as the file streams in, no need to load the whole image at once — a deliberate design choice that makes Crunch friendly to mmap.

Mipmap — 一张纹理八张分辨率

Mipmap — one texture, eight resolutions

"一张纹理八张分辨率,采样时按距离自动选。"

"One texture, eight resolutions — auto-picked by distance at sample time."

1983 年 Lance Williams(NYIT,纽约理工学院计算机图形实验室)在 SIGGRAPH 发表 'Pyramidal Parametrics',第一次系统提出 mipmap 概念。它解决的问题是 3D 场景里最古老也最折磨人的视觉 bug:aliasing(走样/摩尔纹/闪烁)。当一个有纹理的多边形(墙、地面、远处地形)远离相机,屏幕上一个像素就会覆盖纹理上多个 texel——简单的"取最近 texel"采样会随机丢掉大部分信息,产生移动时的闪烁、网格图案上的摩尔纹、远处瓦片纹理的"沸腾"效果。Williams 的洞察:预先存好 N 个降采样层(每层是上一层的 2× 缩放,带 box filter 平均),采样时按"屏幕像素覆盖纹理多大"(LOD,Level of Detail)自动选合适那一层。代价是显存 +33%(几何级数 1+1/4+1/16+...→4/3),收益是无 aliasing + 缓存命中率提升(远处 mip 是小图,容易留在 GPU L2)。所有现代 GPU 纹理默认都带 mipmap,几乎所有引擎都强制开启——这是 GPU 时代的"一旦学会就回不去"的基础设施。

In 1983 Lance Williams (NYIT — the New York Institute of Technology Computer Graphics Lab) published "Pyramidal Parametrics" at SIGGRAPH, the first systematic proposal of the mipmap concept. It solved one of the oldest and most maddening visual bugs in 3D rendering: aliasing (moiré patterns, shimmering, sparkle). When a textured polygon (a wall, the ground, distant terrain) recedes from the camera, a single screen pixel covers many texels — naive "nearest-texel" sampling randomly throws away most of the information, producing shimmering as the camera moves, moiré on grid textures, and "boiling" on distant tiled surfaces. Williams's insight: pre-store N down-sampled layers (each is the previous one box-filtered to 2× smaller), and at sample time pick the right layer based on how much texture area each screen pixel covers (LOD — Level of Detail). The cost is +33 % VRAM (a geometric series 1 + 1/4 + 1/16 + … → 4/3); the payoff is zero aliasing plus better cache hits (distant mips are small and fit in GPU L2). Every modern GPU texture defaults to having mipmaps, almost every engine enforces them — once you know how it feels, you never go back. It's the infrastructure of the GPU era.

技术内核

Technical core

Mipmap 的工程实现非常直接,但每个细节都暗含思想。① 每张 base 图额外存 log₂(N) 个降采样层:mip 0 = base 原始图;mip 1 = box-filtered 2× 缩放;mip 2 = 又 2×;直到 1×1。一张 1024×1024 base 共 11 个 mip(0~10)。降采样可以用 box filter(简单平均)、Lanczos(更锐利但更贵)、或在 sRGB 空间需要先 gamma decode 再 filter 再 encode 回去——很多老引擎在 sRGB 纹理上没做 gamma-correct mip 生成,导致远处纹理看起来"灰蒙蒙"。② 显存额外 +33%:几何级数 1 + 1/4 + 1/16 + ... = 4/3,极限是基础体积的 4/3。这是个固定开销,大概率值得——除非你的纹理永远只在近距离用(UI、屏幕特效)。③ GPU 采样时按 LOD 自动选 mip level:GPU 在像素着色器里能算出当前像素的 dPdx/dPdy(纹理坐标在屏幕水平/垂直方向的偏导数),据此估出"一个屏幕像素覆盖纹理多大",对数运算后得到 LOD 浮点数。整数部分选 mip level,小数部分用于trilinear filtering——在两个 mip 之间双线性插值,避免 mip 跳变可见的"接缝"。④ 各向异性过滤(anisotropic filtering)是 mipmap 的延伸——当视角倾斜时(比如往远处看的地面),屏幕像素在纹理上覆盖的不是正方形而是细长的矩形,简单 trilinear 会过模糊。aniso filtering 沿主轴方向多采样几次再加权,质量更好但带宽更大,通常给"开 16x aniso"档位。⑤ 容器内置 mip chain:KTX/KTX2/DDS 都把 mip 0 → mip N 顺序拼接进 payload,加载时一次 mmap 全部入显存——这就是为什么 GPU 容器格式天生跟 mipmap 绑定的设计。

The engineering of mipmap is straightforward, but every detail hides a small lesson. ① Each base texture stores log₂(N) extra down-sampled layers: mip 0 = the original base; mip 1 = box-filtered 2× smaller; mip 2 = another 2×; … down to 1×1. A 1024×1024 base has 11 total mips (0–10). The down-sample filter can be box (simple averaging), Lanczos (sharper but more expensive), or — for sRGB textures — must gamma-decode, filter in linear, then re-encode; many older engines skipped gamma-correct mip generation, which is why distant textures looked "washed out" in their games. ② +33 % VRAM: the geometric series 1 + 1/4 + 1/16 + … = 4/3, fixed extra cost. Almost always worth it, unless the texture is only ever used up close (UI, screen FX). ③ GPU picks the mip level by LOD at sample time: in a pixel shader the GPU can compute dPdx / dPdy (the partial derivatives of the texture coordinate in screen X / Y), use that to estimate "how much texture one screen pixel covers," and take the log to get a floating-point LOD. The integer part chooses the mip; the fractional part feeds trilinear filtering — bilinear blending between two adjacent mips to mask the visible "seams" of a mip transition. ④ Anisotropic filtering is an extension of mipmap — at oblique viewing angles (looking down at distant ground, say), one screen pixel covers a long thin rectangle on the texture, not a square, and plain trilinear over-blurs. Aniso filtering takes multiple samples along the major axis and weights them, giving better quality at the cost of bandwidth — usually exposed as a "16× aniso" toggle. ⑤ Containers embed the mip chain: KTX / KTX2 / DDS all concatenate mip 0 → mip N into the payload so a load can mmap the whole pyramid into VRAM at once — which is why GPU container formats have always been designed hand-in-glove with mipmaps.

OpenEXR — 影视行业标准

OpenEXR — the film industry standard

"星战幕后用了 30 年的格式,你做合成第一个学的就是它。"

"30 years of Star Wars VFX runs on this. The first format you learn in compositing."

1999 年 ILM 在做《珍珠港》《魔戒》前期合成时,发现手上没有一个合用的中间格式:16-bit TIFF 不够动态范围(镜头闪光、火焰、HDRI 环境贴图很容易超过 1.0),Radiance HDR(C29)只有 RGBE 三通道、不能装 Z-depth / motion vector / object ID。VFX 合成流程的真实需求是:(a) 真 HDR float——亮度无上限,可正可负;(b) 任意 channel——一张文件能塞 RGBA + Z + Normal + Motion + Object ID + UV pass + 几十层灯光分层;(c) tile-based 部分加载——一个 8K EXR 可能 2 GB,Nuke / Houdini 经常只读视口看得到的那一小块;(d) 多分辨率 mip——给 IBL 环境贴图直接拿不同 LOD 采样。OpenEXR 就是为这四件事设计的,30 年没出过第二个对手。

In 1999 ILM was deep in pre-production on Pearl Harbor and The Lord of the Rings, and discovered that no existing intermediate format fit their pipeline: 16-bit TIFF lacked dynamic range (lens flares, explosions and HDRI environment maps easily exceed 1.0), and Radiance HDR (C29) only carried three RGBE channels — no Z-depth, motion vector or object ID. The real VFX-compositing requirements were: (a) true HDR float — unbounded brightness, possibly negative; (b) arbitrary channels — one file holding RGBA + Z + Normal + Motion + Object ID + UV pass + dozens of light groups; (c) tile-based partial loading — an 8K EXR can be 2 GB, and Nuke / Houdini routinely read only the viewport tile; (d) multi-resolution mips — sampling IBL environment maps at the right LOD. OpenEXR was designed for those four needs, and in 30 years no rival has emerged.

技术内核

Technical core

OpenEXR 的设计跟 PNG / JPEG / TIFF 走的不是一条路——它不是"把一张 RGBA 图存好",而是"为合成流水线提供一个可流式部分加载、可任意配 channel、可分通道选 codec 的容器"。① 任意 channel:不止 RGBA,可以是 R / G / B / A / Z / object_id / motion.x / motion.y / normal.x / normal.y / normal.z / UV.x / UV.y 以及任意自定义命名。channel list 在 header 里,每个 channel 自带 pixelType(half / float / uint)和 sampling rate(支持次采样)。② 半精度 float16(half)是默认 pixelType,16 bit 表示 [−65504, +65504] + ±Inf + NaN——这是 ILM 跟 NVIDIA 在 1999 年一起定义的格式,后来被 IEEE 754-2008 收编(binary16),并成为 GPU 显存里 HDR 纹理的事实标准。③ 多压缩 codec:NONE(纯字节流)/ RLE(整数离散最佳)/ ZIP(zlib 通用)/ ZIPS(逐 scanline ZIP)/ PIZ(wavelet 无损,行业默认)/ PXR24(把 float32 截到 24-bit,Pixar 贡献,几乎无损)/ B44 / B44A(老 lossy)/ DWAA / DWAB(基于 DCT 的现代 lossy,DreamWorks 贡献,体积砍 5-10×,常用于 dailies)。可以 per-part 选不同 codec——RGBA 用 DWAA、Z 用 ZIP、Object ID 用 RLE,各取所长。④ Tile-based 部分加载:文件可选 scanline 或 tile 模式,tile 模式下 header 里有 offset table,Nuke / Houdini / Mari 加载 8K EXR 时只读视口看得到的几个 tile(可能 64 KB 而不是 2 GB)——这个能力是非线性合成软件 / 数字绘景的命脉。⑤ 多分辨率 mip:tiled EXR 可存 mipmaps(rip-maps 也行),IBL 环境贴图按 LOD 直接采样,不必外部生成 mip chain。⑥ 多帧 / multi-part:OpenEXR 1.x 用 .0001.exr / .0002.exr 帧序列(每帧一文件,管线友好);2.0(2013)引入单文件多 part,可在一个 .exr 里塞多个 layer / 多个 view(立体渲染左右眼)/ 多个分辨率,每个 part 独立 codec。这种"容器化"路线让 EXR 跟 USD / OCIO / ACES 这些现代 VFX 中间件无缝衔接。

OpenEXR's design takes a different road from PNG / JPEG / TIFF — it is not "store one RGBA image well" but "provide a streamable, partially-loadable container with arbitrary channels and per-channel codec choice for the compositing pipeline." ① Arbitrary channels: not just RGBA but R / G / B / A / Z / object_id / motion.x / motion.y / normal.x / normal.y / normal.z / UV.x / UV.y plus any custom name. The channel list lives in the header, each channel carrying its own pixelType (half / float / uint) and sampling rate (subsampling supported). ② Half-precision float16 is the default pixelType — 16 bits representing [−65504, +65504] plus ±Inf and NaN — a format ILM and NVIDIA jointly defined in 1999, later folded into IEEE 754-2008 (binary16) and now the de-facto standard for HDR textures in GPU VRAM. ③ Multiple compression codecs: NONE (raw bytes) / RLE (best for integer discrete data) / ZIP (general-purpose zlib) / ZIPS (per-scanline ZIP) / PIZ (wavelet lossless, industry default) / PXR24 (truncates float32 to 24 bits — Pixar's contribution, near-lossless) / B44 / B44A (legacy lossy) / DWAA / DWAB (modern DCT-based lossy, DreamWorks' contribution, 5-10× smaller — the dailies workhorse). Codecs can be picked per part — DWAA on RGBA, ZIP on Z, RLE on Object ID; each plays to strength. ④ Tile-based partial loading: files can be scanline or tile mode; in tile mode the header carries an offset table, so Nuke / Houdini / Mari loading an 8K EXR read only the visible tiles (possibly 64 KB out of 2 GB) — that capability is the lifeblood of node-based compositors and digital matte painters. ⑤ Multi-resolution mips: tiled EXR can carry mipmaps (or rip-maps), so IBL environment maps sample at the right LOD without an external mip chain. ⑥ Multi-frame / multi-part: OpenEXR 1.x used per-frame .0001.exr / .0002.exr sequences (one file per frame, pipeline-friendly); 2.0 (2013) added single-file multi-part, packing multiple layers, multiple views (stereo left/right), or multiple resolutions into one .exr with per-part codec choice. That "container-like" direction lets EXR plug seamlessly into modern VFX middleware — USD, OCIO, ACES.

$ exrheader scene.exr                          # 看 channel / codec / displayWindow / attrs
$ exrinfo scene.exr                            # 简洁版 header 摘要 (OpenEXR 3.x)
$ oiiotool scene.exr -ch R,G,B,A -o rgb.exr    # OpenImageIO 抽 channel
$ oiiotool scene.exr --channels=Z -o depth.exr # 单独抽 depth pass
$ exrenvmap input.exr cubemap.exr              # latlong → cube · IBL 预处理
$ exrmaketiled in.exr tiled.exr                # scanline → tiled (启用部分加载)
$ exrmultipart -combine -i a.exr b.exr -o m.exr# 多 part 合并到一个文件
$ exr2aces in.exr out.exr                      # 转 ACES2065-1 色彩空间

Radiance HDR — 光照贴图老兵

Radiance HDR — the lightmap veteran

"用 8-bit 共享指数装 32-bit float —— 1989 的 hack。"

"32-bit float packed via shared 8-bit exponent — a 1989 hack."

1989 年 Greg Ward 在 Lawrence Berkeley National Lab 写 Radiance —— 一套物理光照模拟渲染器,要算光在场景里的真实辐射度,输出值会从 1e-6(月光)横跨到 1e6(太阳直射)。当时的难题不是算法,而是把这些数装到磁盘里:浮点 IEEE 754 32-bit/通道的话,一张 1024×768 的图就要 12 MB,而 1989 的硬盘是几十 MB 起跳的奢侈品。Greg Ward 的 hack:RGB 三个通道共享一个 8-bit 指数—— 把 R/G/B 三个 float 归一化到同一个 2^E 之下,只存归一化后的 8-bit 尾数 + 一个 8-bit 指数,合计 32 bit/像素(跟 RGBA8 一样大)。范围理论上 1e-38 到 1e38,精度 ~1%(对光照足够,对色彩管理就显粗糙)。再配一个极简的 RLE 行内压缩,这就是 .hdr 格式。靠着这个 hack,IBL 环境贴图、PSPI(panoramic stereo painted images)、HDR 全景照片在 90 年代到 2000 年代撑了 20 年,直到 OpenEXR 把它替换掉。

In 1989 Greg Ward at Lawrence Berkeley National Lab was writing Radiance — a physically-based lighting-simulation renderer — and needed to store radiance values that spanned 1e-6 (moonlight) to 1e6 (direct sun). The challenge wasn't the math; it was fitting that range on disk: IEEE 754 32-bit per channel meant a 1024×768 image cost 12 MB, and 1989 hard drives were luxuries measured in tens of megabytes. Ward's hack: have R/G/B share a single 8-bit exponent — normalise the three floats to the same 2^E, store the normalised 8-bit mantissas plus one 8-bit exponent, totalling 32 bit/pixel (the same as RGBA8). Range nominally 1e-38 to 1e38, precision ~1 % (good enough for lighting, coarse for colour management). Add a minimal scanline RLE on top, and you have the .hdr format. The hack carried IBL environment maps, PSPI panoramas and HDR photography through the 1990s and 2000s for two decades, until OpenEXR finally retired it.

技术内核

Technical core

Radiance HDR 的内核小到只有三件事。① RGBE 编码——三个通道共用一个指数。编码时找 max(R, G, B),归一化到 [0, 1],尾数乘 256 取整,指数加 128 偏移存为一字节;解码时反向。共享指数让"亮度差异极大的颜色"(蓝色通道极弱、红色极强)精度退化——这是它跟 float16 / float32 在色彩管理意义上的本质差距。② 极简 RLE——文件里每一行像素分开压缩:旧格式整行 RGBE 一起 RLE,1991 之后改成"先把 R / G / B / E 四个字节流分别拆开,再各自 RLE",压缩率显著提升(因为 E 经常大段重复,RLE 在它上面收益最大)。压缩开销小到 90 年代的 SGI 能软件实时解码。③ 文本头——文件开头是 ASCII 头,几行 #?RADIANCE / 标识 / 曝光值 / EXPOSURE= / FORMAT= 32-bit_rle_rgbe,然后一个空行,然后是分辨率字符串(-Y 480 +X 640),再之后才是 RLE 二进制流。这种"文本头 + 二进制 payload"的设计后来被 PFM / NetPBM 继承。三件事加起来就是整个 .hdr 格式——简单、自包含、跨平台。代价:① 不支持 alpha;② 没有 metadata(没有 ICC profile、白点、色彩空间);③ 只有 RGB,不能存 Z / Normal / Motion;④ 共享指数精度天生粗糙。这些缺点直接催生了 OpenEXR 在 1999 年的设计目标——"做 Radiance HDR 做不到的所有事"。

The Radiance HDR core is just three things. ① RGBE encoding — three channels share one exponent. Encode by finding max(R, G, B), normalising to [0, 1], scaling mantissas by 256 and rounding, and storing the exponent biased by 128 in one byte; decode is the inverse. The shared exponent loses precision for "channels of wildly different magnitudes" (a strong red beside a weak blue) — that's the format's fundamental colour-management weakness compared to float16 / float32. ② Minimal RLE — pixels are compressed per scanline: the old format ran RLE over the interleaved RGBE bytes; the post-1991 format de-interleaves into four byte streams (R / G / B / E) and RLEs each separately, dramatically improving compression (E often has long runs, where RLE wins biggest). Compression overhead is light enough that 1990s SGI workstations decoded in software in real time. ③ Text header — the file begins with an ASCII header: a few lines of #?RADIANCE / identifier / EXPOSURE= / FORMAT=32-bit_rle_rgbe, then a blank line, then a resolution string (-Y 480 +X 640), and only then the RLE binary stream. The "text header + binary payload" pattern was later inherited by PFM and the NetPBM family. Those three things are the entire .hdr format — simple, self-contained, portable. The costs: ① no alpha; ② no metadata (no ICC profile, no white-point, no colour-space tag); ③ RGB only — no Z, normal or motion channels; ④ inherent precision floor from the shared exponent. Those very gaps drove OpenEXR's 1999 design brief: "do everything Radiance HDR can't."

PFM — Portable FloatMap

PFM — Portable FloatMap

"NetPBM 的 HDR 表亲 —— ASCII 头加裸 float。"

"NetPBM's HDR cousin — ASCII header plus raw float."

学术研究和渲染器中间格式有一种长期需求,主流格式都满足不了:"最简单的 HDR 容器"——不要任何压缩(读写都是 mmap,瞬间)、不要任何 metadata(纯净,bit 级 reproducibility)、能直接当 float* 数组操作(C 代码 fopen + fseek 过头部就能用,不需要任何库)。OpenEXR 太复杂(几百种 attribute、wavelet codec、tile / scanline 切换),Radiance HDR 精度太粗(RGBE shared exponent),float TIFF 的 IFD 解析又是一坨。Paul Debevec 等学术圈的人在 NetPBM(PPM / PGM / PBM)风格基础上,做了 PFM:三行 ASCII 头(magic / 宽高 / scale 字段)紧跟 raw float32 像素流。论文 supplementary、渲染器中间盘、调试图像 dump,这些场景里 PFM 是最舒服的——别的格式都嫌"太聪明"。

Academic research and renderer-internal storage share a recurring need that no mainstream format satisfies: the simplest possible HDR container — no compression (read / write is just mmap), no metadata (pure, bit-exact reproducibility), and direct use as a float* array (C code can fopen, fseek past the header, and operate on the bytes without any library). OpenEXR is too complex (hundreds of attributes, wavelet codecs, tile / scanline modes), Radiance HDR is too coarse (RGBE shared exponent), float TIFF's IFD parsing is its own mess. Paul Debevec and colleagues in academia took the NetPBM lineage (PPM / PGM / PBM) and produced PFM: three ASCII header lines (magic / width-height / scale) followed by a raw float32 pixel stream. For paper supplementaries, renderer internal dumps, and debugging-image scratch storage, PFM is the most comfortable choice — every other format feels "too clever."

技术内核

Technical core

PFM 内核三件事,合起来不到 30 行 C 代码就能写完读写器。① NetPBM 风格的文本头:像 PPM 一样,前几行是 ASCII。第一行是 magic 标识——PF 表示 float32 RGB,Pf 表示 float32 单通道灰度。第二行是宽高(空格分隔的整数)。第三行是 scale 字段——一个浮点数,绝对值是曝光 / 缩放因子(读取时通常忽略,渲染器自己处理),符号编码字节序:负数小端,正数大端。三行,十几个字符。② raw float32 RGB 像素流:头部紧跟二进制 float32 数据,RGB 交错(R₀ G₀ B₀ R₁ G₁ B₁ …),12 字节/像素;灰度模式 4 字节/像素。自下而上(像 BMP,但跟 OpenGL 纹理坐标天然吻合)——这是最常见的踩坑点,新人写 reader 很容易上下颠倒。③ 无任何压缩 / 无任何 metadata:这是故意的。没有 ICC profile,没有色彩空间,没有曝光记录,没有作者注释——纯粹"一张数字"。这恰好是论文实验、渲染器调试、参考实现里最重要的属性:你要 reproduce 别人的结果,任何额外 metadata 都是干扰。代价是它没法用于生产:文件大(4K RGB float32 ≈ 95 MB,无压缩),没法做色彩管理,工具支持 niche。但在它的位置——学术调试、bit-exact 中间盘——没人能替代它。

PFM has three core elements; a complete reader/writer fits in under 30 lines of C. ① NetPBM-style text header: like PPM, the first few lines are ASCII. Line 1 is the magic — PF for float32 RGB, Pf for float32 grayscale. Line 2 is the width and height (space-separated integers). Line 3 is the scale field — a float whose absolute value is an exposure / scale factor (typically ignored at read time; the renderer handles tone mapping itself), and whose sign encodes endianness: negative is little-endian, positive is big-endian. Three lines, a dozen characters. ② Raw float32 RGB pixel stream: binary float32 data follows the header, RGB-interleaved (R₀ G₀ B₀ R₁ G₁ B₁ …) at 12 bytes per pixel; grayscale is 4 bytes per pixel. Bottom-up (like BMP, but conveniently aligned with OpenGL's texture coordinate origin) — the most common pitfall when writing a reader is flipping the rows. ③ No compression, no metadata: intentional. No ICC profile, no colour-space tag, no exposure record, no author comment — just "the numbers." That happens to be the single most important property for paper experiments, renderer debugging, and reference implementations: when you reproduce someone else's result, any extra metadata is noise. The cost is that PFM is unsuitable for production: files are huge (a 4K RGB float32 image is ~95 MB uncompressed), there's no colour management, and tooling support is niche. But in its niche — academic debugging, bit-exact intermediate scratch — nothing else replaces it.

16/32-bit TIFF — 被忽视的扛把子

16/32-bit TIFF — the unsung workhorse

"40 年了,印刷厂还在用它,因为没有更好的替代。"

"40 years on, print shops still use it because nothing better replaced it."

1986 年 Aldus(后来被 Adobe 在 1994 收购)推 PageMaker —— 桌面排版革命的开端,问题是当时图像格式碎成一地:GIF 只有 256 色,EPS 是 PostScript 矢量,Mac PICT 跨不了平台,扫描仪厂商各自用私有格式。Aldus 跟扫描仪厂商一起设计了 TIFF —— Tag Image File Format —— 目标是"任何位深、任何 codec、任何 metadata、跨平台无损"。它的解法是 tag 系统:不像 BMP 那样固定字段,而是用 IFD(Image File Directory)装一个"几百种 tag 都可选"的描述表,payload 可换 codec,可多页,可跨设备元信息。从此所有需要"高保真 + 灵活元数据"的领域都默认 TIFF:印刷出版、扫描仪、显微镜、医学影像、卫星遥感、文物档案。40 年了,没人替代得了——不是因为它优秀,是因为它什么都能装:DICOM 内嵌它,GeoTIFF 是它的子集,DNG 是它的子集,Photoshop 16-bit 工作流默认它。被忽视的扛把子。

In 1986 Aldus (acquired by Adobe in 1994) launched PageMaker, the start of the desktop publishing revolution. The problem: image formats were a Tower of Babel — GIF only did 256 colours, EPS was vector PostScript, Mac PICT didn't cross platforms, scanner vendors each shipped a proprietary format. Aldus partnered with scanner vendors to design TIFF — Tag Image File Format — aiming for "any bit depth, any codec, any metadata, cross-platform, lossless." The solution was a tag system: rather than fixed fields like BMP, an IFD (Image File Directory) carries a descriptive table of "hundreds of optional tags," the payload swaps codecs, files can be multi-page, and device metadata travels with the image. From then on every domain needing "high fidelity + flexible metadata" defaulted to TIFF: print publishing, scanners, microscopes, medical imaging, satellite remote-sensing, museum archives. Forty years later nothing has replaced it — not because it's elegant, but because it can hold anything: DICOM embeds it, GeoTIFF is its subset, DNG is its subset, Photoshop's 16-bit workflow defaults to it. The unsung workhorse.

技术内核

Technical core

TIFF 的设计核心可以总结成四条规则,40 年没变。① 基于 IFD 的 tag 系统:文件不是"按字段顺序"装数据,而是"我有什么属性,就在 tag 表里加一行"。tag id 是 16-bit 无符号整数(0~65535),Adobe 保留 32768 以下,32768~65535 是private tags(GeoTIFF / DNG / OME-TIFF 等子集格式都在这个区域)。每个 tag 自带数据类型(BYTE / SHORT / LONG / RATIONAL / ASCII / FLOAT / DOUBLE 等 12 种),解析器只要"我认识的 tag 处理,不认识的跳过"。这种设计直接借鉴了 IBM 的 EBCDIC 数据描述传统,后来又被 ISOBMFF / Matroska 等容器借鉴。② 多页(IFD chain):每个 IFD 末尾有一个指向"下一个 IFD"的 offset,多页 TIFF 就是把 IFD 串成链表。最经典用例是传真组 4(Group 4 Fax)——黑白文档扫描多页存一个 .tif;现在扩展到扫描仪批量扫描、显微镜 z-stack、卫星多光谱波段,每页一个 IFD。③ 多种 codec 可选:NONE(原始)/ PackBits(早期 Mac RLE)/ LZW(默认无损,90 年代有专利争议)/ DEFLATE(zlib,无损,现在最常用)/ JPEG-in-TIFF(把 JPEG bitstream 当 strip 数据装,1992 加,但 spec 模糊导致实现不一致)/ Group 3 / Group 4 Fax(双值黑白图像专用)/ LERC(地理空间近无损)。每个 strip 或 tile 独立 codec。④ 任意位深:1 bit(黑白扫描)/ 4 / 8(普通照片)/ 16(高保真扫描、医学影像)/ 32-bit float(IEEE 754,科研、HDR)。BitsPerSample tag 是个数组——可以是 (16, 16, 16) 表示 RGB 各 16 bit,可以是 (8, 8, 8, 8) 表示 RGBA8,甚至 (16, 16, 16, 16, 16, 16) 表示 6 通道高光谱。SampleFormat tag 进一步指定每个通道是 unsigned int / signed int / IEEE float / void(自定义)——这就是 TIFF 能存 16-bit 摄影、32-bit float HDR、整数 ID buffer 的根源。

TIFF's design boils down to four rules, unchanged in 40 years. ① IFD-based tag system: the file isn't laid out as "fields in fixed order," it's "whatever properties exist, add a row to the tag table." Tag IDs are 16-bit unsigned integers (0–65535); Adobe reserves 0–32767 and the 32768–65535 range is private tags (where GeoTIFF, DNG, OME-TIFF and other subset formats live). Each tag carries its own data type (BYTE / SHORT / LONG / RATIONAL / ASCII / FLOAT / DOUBLE — 12 in total), and a parser simply handles tags it knows and skips the rest. The design borrows directly from IBM's EBCDIC data-description tradition and was later borrowed by ISOBMFF, Matroska and other modern containers. ② Multi-page (IFD chain): each IFD ends with an offset to the next IFD, so multi-page TIFFs are linked lists of IFDs. The classic use case is Group 4 fax — multi-page black-and-white document scans in a single .tif; today this extends to flatbed batch scans, microscope z-stacks, and satellite multi-spectral bands, one IFD per page. ③ Multiple codec options: NONE (raw) / PackBits (early Mac RLE) / LZW (default lossless, embroiled in 1990s patent disputes) / DEFLATE (zlib, lossless, today's most common choice) / JPEG-in-TIFF (a JPEG bitstream stuffed into strip data, added in 1992 but with vague enough spec language that implementations still disagree) / Group 3 and Group 4 fax (bilevel black-and-white only) / LERC (near-lossless geospatial). Each strip or tile picks its codec independently. ④ Arbitrary bit depth: 1-bit (B&W scans) / 4 / 8 (regular photos) / 16 (high-fidelity scans, medical imaging) / 32-bit float (IEEE 754, science, HDR). The BitsPerSample tag is an array — it can be (16, 16, 16) for RGB at 16 bpp, (8, 8, 8, 8) for RGBA8, or even (16, 16, 16, 16, 16, 16) for six-channel hyperspectral. The SampleFormat tag further specifies whether each channel is unsigned int / signed int / IEEE float / void (custom) — that combination is exactly why TIFF can hold 16-bit photography, 32-bit float HDR, and integer ID buffers in the same container.

RAW — 厂商林立的原始数据

RAW — the manufacturer-fragmented origin data

"所谓 RAW,不是一个格式,是几十个互不兼容的格式族。"

"'RAW' is not one format — it's a zoo of several dozen incompatible formats."

数码相机 sensor 的原始输出是 12-14 bit Bayer pattern raw 数据——每个像素位置上只有一个颜色样本(R 或 G 或 B),需要 demosaic 算法才能算出完整 RGB。如果在相机里直接转 JPEG,会立即丢掉四样东西:(a) 高位深(14 bit → 8 bit,动态范围砍 64 倍);(b) demosaic 之前的灵活性(JPEG 已经是固定算法插值过的结果,不能换);(c) 白平衡可调性(JPEG 已经把 WB 烘进像素,后期改容易出色偏);(d) 曝光宽容度(过曝 / 欠曝在 14 bit RAW 里能拉回来,JPEG clip 后无法恢复)。摄影师需要"把决定留到后期再做"的格式 = RAW。但每家相机厂商都自己定义,互不兼容,这是后期工作流 30 年的最大头疼——也是 LibRaw / Lightroom 这些工具存在的全部理由。

A digital camera sensor's raw output is 12-14 bit Bayer-pattern data — each pixel position carries only one colour sample (R or G or B), and a demosaic algorithm has to interpolate the full RGB. Convert to JPEG inside the camera and you immediately lose four things: (a) high bit depth (14 bit → 8 bit, dynamic range cut by 64×); (b) flexibility before demosaic (JPEG is already a fixed-algorithm interpolation, you can't swap it); (c) white-balance malleability (JPEG bakes WB into pixels; later changes risk colour casts); (d) exposure latitude (over- and under-exposure can be recovered in 14 bit RAW; JPEG clips and the data is gone). The format that lets photographers "defer decisions to post" is RAW. But every camera maker defined its own, none compatible with the others — that has been the post-production headache of the past 30 years, and the entire reason LibRaw / Lightroom / Capture One exist.

技术内核

Technical core

RAW 不是一个格式,是一种思路的几十种实现。技术上有五条共同线索。① Bayer mosaic CFA:sensor 上每个物理像素只盖一种颜色滤镜(R/G/B 中的一种),按 2×2 重复排列。每个 2×2 块里有 2 绿 + 1 红 + 1 蓝(模拟人眼对绿色亮度更敏感)。读 RAW 必须先知道是 RGGB / BGGR / GRBG / GBRG 哪种,再用demosaic 算法(AHD / VNG / PPG / AMaZE / DCB / Igv …十多种)插出每个像素完整的 RGB。Fuji X-Trans 是个异类——6×6 X 形排列,普通 demosaic 算法对它效果差,得用专门的 X-Trans demosaic。② 12-14 bit/channel:不是 8 bit。这意味着比 JPEG 多 4-6 stop 动态范围(高光 / 暗部都能拉)。CMOS sensor 物理 ADC 通常 14 bit,Phase One 等中画幅可达 16 bit。RAW 把这些位深原样保留,后期"曝光 +2 / -2"才不会出 banding。③ 白平衡 / 色彩矩阵 / tone curve 全部未应用:相机只在 EXIF / MakerNotes 里"记录"拍摄时的 WB 是 5500K 还是 Auto,但不烘进像素。色彩矩阵(把 sensor 厂商特定的 R/G/B 响应曲线映射到标准 XYZ 色彩空间的 3×3 矩阵)也是同样:存为 metadata,由后期解码器应用。这是 RAW 跟 JPEG 的根本不同——后者是"决定都做完了的最终结果",前者是"原料 + 配方,但还没开火"。④ 容器基本都基于 TIFF/IFD:Canon CR2 / Nikon NEF / Sony ARW / Fuji RAF / Olympus ORF / Pentax PEF / Panasonic RW2 几乎全是 TIFF base 加私有 tag 区(0x8769 EXIF + 0x927C MakerNote + 厂商私有 tag id)。例外是 Canon CR3(2018 起,改用 ISOBMFF / HEIF 同源容器)和 Sigma X3F(自家完全独立)。这种"TIFF + 私有 tag"的设计意味着标准 TIFF reader 能看到大致结构,但解不出像素——必须靠厂商 SDK 或 LibRaw 的逆向工程。⑤ 解码必须靠厂商 SDK 或 LibRaw:Adobe Camera Raw / Lightroom 的 RAW 解码引擎是闭源商业;开源世界里 LibRaw(Dave Coffin 单文件 C 程序 dcraw 的继承者)通过逆向工程支持几乎所有相机 RAW,是 darktable / RawTherapee / digiKam / Fiji 的共同底层。dcraw 本身是工程史奇迹——Coffin 一个人 20 年维护一份单文件 C,支持上千款相机。LibRaw 接手后变成了正式 lib + 持续更新。

RAW is not one format but one idea realised dozens of times. Five common technical threads. ① Bayer mosaic CFA: every physical sensor pixel sits behind a single colour filter (one of R/G/B), arranged in a repeating 2×2. Each 2×2 has 2 green + 1 red + 1 blue (mirroring the eye's stronger luminance response to green). Reading a RAW requires first knowing the arrangement (RGGB / BGGR / GRBG / GBRG) and then running a demosaic algorithm (AHD / VNG / PPG / AMaZE / DCB / Igv … more than ten exist) to interpolate the full RGB at every pixel. Fuji X-Trans is the oddball — a 6×6 X-shaped pattern, on which generic Bayer demosaicers do poorly; it needs a dedicated X-Trans demosaic. ② 12-14 bit/channel, not 8. That means 4-6 stops more dynamic range than JPEG (highlights and shadows both recoverable). CMOS sensor ADCs are usually physically 14 bit; Phase One and similar medium-format gear reach 16 bit. RAW keeps every bit, so post-exposure "+2 / −2" doesn't band. ③ White balance, colour matrix, and tone curve are not applied. The camera only records in EXIF / MakerNotes that WB was set to 5500K or Auto — it does not bake it into the pixels. The colour matrix (a 3×3 mapping from the sensor's vendor-specific R/G/B response into standard XYZ) is likewise stored as metadata for the decoder to apply later. That is the deep difference from JPEG: JPEG is "all decisions, finalised"; RAW is "ingredients plus recipe, but the burner is off." ④ The container is almost always TIFF/IFD: Canon CR2 / Nikon NEF / Sony ARW / Fuji RAF / Olympus ORF / Pentax PEF / Panasonic RW2 are all TIFF-based with private tag regions (0x8769 EXIF + 0x927C MakerNote + vendor-private tag ids). Exceptions: Canon CR3 (since 2018, ISOBMFF — the HEIF / MP4 family) and Sigma X3F (entirely independent). The "TIFF + private tags" design means a generic TIFF reader can see the gross structure but can't decode the pixels — that requires the vendor SDK or LibRaw's reverse-engineering. ⑤ Decoding leans on the vendor SDK or LibRaw: Adobe Camera Raw / Lightroom's RAW decoder is a closed-source commercial engine; in open source, LibRaw (the successor to Dave Coffin's single-file dcraw) supports nearly every camera RAW through reverse engineering and is the shared backend of darktable / RawTherapee / digiKam / Fiji. dcraw itself is an engineering miracle — Coffin maintained a single-file C program for 20 years that supported thousands of cameras solo. LibRaw took over and turned it into a proper library with continuous updates.

$ dcraw -v -w in.NEF                          # dcraw: 用相机 WB 解码 NEF 输出 PPM
$ dcraw -i -v in.CR2                          # 只读 metadata 不解码
$ rawtherapee-cli -o out.tif -t -c in.CR2     # RawTherapee 命令行 RAW → 16-bit TIFF
$ darktable-cli in.ARW out.jpg                # darktable 命令行 RAW → JPEG
$ exiv2 -p a in.RAF                           # 查 EXIF + MakerNotes
$ exiftool -a -G1 -s in.NEF                   # 万能元数据查看 · 厂商私有 tag 都列出来
$ libraw_unpack in.ARW                        # LibRaw 命令行: 输出未处理 Bayer raw
$ Adobe\ DNG\ Converter --convert in.CR3 out.dng # 转 DNG 归档

DNG — Adobe 想统一 RAW

DNG — Adobe's attempt to unify RAW

"想做 RAW 的 PNG,部分成功。"

"Tried to be the PNG of RAW. Partial success."

2004 年 Adobe 看到 RAW 生态彻底碎掉:Canon CR2、Nikon NEF、Sony ARW、Fuji RAF、Olympus ORF、Pentax PEF、Panasonic RW2…几十种格式互不兼容,每出一款新相机 Adobe Camera Raw / Lightroom 就得加一个 decoder profile,工作量惊人;摄影师归档时也心慌——5 年后还能不能开一张今天的 .ARW?Adobe 推出 DNG(Digital Negative),基于开放的 TIFF/EP(TIFF Electronic Photography)扩展,目标只有一个:"一个公开 spec 的 RAW 格式装所有厂商的数据"。结果一半成功:Pentax / Leica / Hasselblad 选择原生输出 DNG,Apple 2020 年的 iPhone ProRAW 也用 DNG 包装;但 Canon / Nikon / Sony 三巨头坚持自家专有,从未给 DNG 让路。Adobe DNG Converter 工具可以把任意厂商 RAW 离线转 DNG 做归档,但转换过程可能有损 metadata——某些 MakerNotes 字段在 DNG 里没有标准对应,只能丢弃。

By 2004 Adobe saw the RAW ecosystem fully fragmented: Canon CR2, Nikon NEF, Sony ARW, Fuji RAF, Olympus ORF, Pentax PEF, Panasonic RW2 — dozens of mutually incompatible formats. Every new camera body forced Adobe Camera Raw / Lightroom to add another decoder profile, the workload was extraordinary, and photographers were nervous about archiving — would today's .ARW still open in five years? Adobe introduced DNG (Digital Negative), built on the open TIFF/EP (TIFF Electronic Photography) extension, with one goal: "one publicly specified RAW format that holds every vendor's data". The result was half a success: Pentax / Leica / Hasselblad chose to output DNG natively, and Apple's 2020 iPhone ProRAW wraps DNG too — but Canon / Nikon / Sony stuck with their proprietary formats and have never made room for DNG. The Adobe DNG Converter can offline-convert any vendor's RAW to DNG for archive, but conversion may lose some metadata — certain MakerNotes fields have no standard DNG equivalent and are simply dropped.

技术内核

Technical core

DNG 三件事撑起整个设计。① 基于 TIFF/EP 扩展:DNG 不是从零设计的容器,而是在 TIFF 6.0 + TIFF/EP(TIFF Electronic Photography,1998 ISO 12234-2)上加了一组规范化的私有 tag。这意味着已有 TIFF reader 能看到大致结构(虽然不能正确出图),也意味着 DNG spec 公开后,任何人能写 DNG 解码器——Adobe 故意降低门槛。② 厂商私有 metadata 透传:DNG 在容器里专门留一块 MakerNotes 区,把原厂的私有元数据(比如 Sony ARW 里的某个加密曝光块)原样塞进去,DNG 解码器看不懂也不会丢。这是 Adobe 跟厂商的"和解":你转 DNG 不会丢你的相机特定信息,某天厂商 SDK 想读还能读回去。③ 包含 demosaic 后的可选预览 + 完整原始 sensor 数据:DNG 文件里通常嵌一张 JPEG preview(给 Lightroom 缩略图秒开)+ 完整的 Bayer raw payload(给后期重新解码)。比起原厂 RAW 多 5-10% 体积,但换来"打开就有缩略图"的体验。某些 DNG 还可选 lossy compressed 模式(Adobe Lossy DNG,基于 JPEG 在 raw 域上做有损,体积砍 50% 但有损 RAW 的灵活度——主要给 iPhone ProRAW 用)。

DNG rests on three pillars. ① Built on TIFF/EP: DNG is not a from-scratch container; it sits on TIFF 6.0 + TIFF/EP (TIFF Electronic Photography, ISO 12234-2 from 1998) with a standardised set of private tags. Existing TIFF readers can see the gross structure (without rendering correctly), and once the DNG spec was public anyone could write a DNG decoder — Adobe deliberately lowered the barrier. ② Vendor metadata passthrough: DNG reserves a MakerNotes region in the container and stores the original vendor's private metadata (e.g. some encrypted exposure block from Sony ARW) verbatim; the DNG decoder needn't understand it, but it isn't dropped. This is Adobe's reconciliation gesture to vendors: converting to DNG doesn't lose your camera-specific information, and a vendor SDK could in principle read it back later. ③ Optional demosaiced preview + full original sensor data: a DNG file usually carries an embedded JPEG preview (so Lightroom thumbnails appear instantly) plus the complete Bayer raw payload (for re-decoding in post). The cost is 5-10 % more bytes than the original vendor RAW, in exchange for "opens with a thumbnail" UX. Some DNGs also enable a lossy mode (Adobe Lossy DNG — JPEG-style lossy in the raw domain, 50 % smaller, at the cost of some RAW flexibility — primarily targeted at iPhone ProRAW).

CR3 / NEF / ARW — 主流厂商的 RAW

CR3 / NEF / ARW — the big-three vendor RAWs

"三家相机巨头各做一套,都不兼容,都活得很好。"

"Three camera giants, three formats, none compatible — and all thriving."

Canon / Nikon / Sony 三家占数码相机市场 80% 以上,各家拥有完整的 DSLR / 无反 + 镜头生态(EF / RF / F / Z / E / FE 卡口等),RAW 格式是其专有生态的最后一环——锁定到自家 RAW 意味着用户后期工作流也跟着锁定(用 Canon DPP / Nikon NX Studio / Sony Imaging Edge 时体验最完整,跨家就得依赖 LibRaw 或商业第三方)。Canon 2018 把 CR2 升级 CR3,容器从 TIFF 换成 ISOBMFF(同 HEIF / MP4 spec 族)——为的是跟 HEIF 工具链共享 box 解析器,顺便能在 RAW 文件里塞 HEIF 缩略图、HEVC 视频片段、AAC 音频(给"双重曝光"和短视频功能用)。Nikon NEF 一直是 TIFF base,从 1999 年 D1 到现在 Z 系列没换。Sony ARW 也是 TIFF base,但有臭名昭著的"有损 RAW"模式——早期 α 系列默认输出"压缩 RAW"实际上是有损,被摄影社区批评后才允许选"未压缩"。三家都不公开 RAW spec,LibRaw / dcraw 全靠逆向工程支持。

Canon / Nikon / Sony together hold over 80 % of the digital-camera market, each with a complete DSLR / mirrorless + lens ecosystem (EF / RF / F / Z / E / FE mounts and so on), and the RAW format is the final piece of that proprietary stack — being locked into a vendor's RAW means your post workflow follows (the experience is most complete in Canon DPP / Nikon NX Studio / Sony Imaging Edge; cross-vendor work depends on LibRaw or commercial third parties). Canon upgraded CR2 to CR3 in 2018, swapping the container from TIFF to ISOBMFF (same family as HEIF / MP4) — to share box parsers with the HEIF toolchain and incidentally to embed HEIF thumbnails, HEVC video clips, and AAC audio in the RAW file (for "double-exposure" and short-video features). Nikon NEF has been TIFF-based since the 1999 D1 and the Z series has not changed it. Sony ARW is also TIFF-based, but with the notorious "lossy RAW" mode — early α bodies defaulted to "compressed RAW" that was actually lossy, and only after sustained criticism from the photography community was an "uncompressed" option allowed. None of the three publish RAW specs; LibRaw / dcraw support them entirely through reverse engineering.

技术内核

Technical core

三巨头 RAW 共三条线索。① 容器:CR3 是 ISOBMFF,CR2 / NEF / ARW 是 TIFF 系。Canon 2018 把 CR2 升级 CR3 时换了容器,目的就是跟现代 ISOBMFF 生态(HEIF / MP4 / AVIF / JPEG XL)对齐,顺便能在一个 .CR3 里塞 RAW + JPEG preview + HEVC 视频片段 + AAC 音频(给"双重曝光"和短视频功能用)。Nikon NEF 和 Sony ARW 还是传统 TIFF base——文件开头 TIFF header,接 IFD chain,每个 IFD 装一张图(thumbnail / preview JPEG / 真正 RAW),Sony 还在 IFD 里加私有 SR2 sub-IFD 装额外 metadata。② 各家私有有损 RAW 压缩。Canon 有 CRaw(visually lossless,体积砍 30-40%);Nikon 有 NEF Compressed(实际是把 14-bit raw 用一个查找表压成 12-bit 等价精度,有损但视觉无损);Sony 早期默认就是有损"压缩 RAW"(被批评后允许选"未压缩")。这些有损模式都是闭源算法,LibRaw 逆向支持但有时跟厂商官方解码结果略有偏差。③ "有损 RAW"概念的兴起。原本 RAW 的精神就是"无损保留 sensor 数据",但 14-bit 有损压缩(类似 Lossy DNG)能砍体积 50-70%、视觉几乎无损,对存储敏感的场景(连拍 / 4K 视频拍摄间隙拍照)很有吸引力。Canon CRaw / Sony Compressed RAW / Nikon NEF Compressed 都属于这类——长远看 RAW 文件正在向 "有损但视觉无损" 滑动,这跟 JPEG XL / HEIC 的设计哲学不谋而合。

Three threads connect the big-three RAWs. ① Container: CR3 is ISOBMFF, CR2 / NEF / ARW are TIFF-family. When Canon upgraded CR2 to CR3 in 2018 it swapped the container — the goal was to align with the modern ISOBMFF ecosystem (HEIF / MP4 / AVIF / JPEG XL) and incidentally to pack RAW + JPEG preview + HEVC video clips + AAC audio into one .CR3 (used by "double-exposure" and short-video features). Nikon NEF and Sony ARW remain traditional TIFF-based — the file opens with a TIFF header, then an IFD chain, each IFD holding one image (thumbnail / preview JPEG / actual RAW); Sony additionally puts a private SR2 sub-IFD inside the IFD to carry extra metadata. ② Each vendor's private lossy RAW compression. Canon offers CRaw (visually lossless, 30-40 % size reduction); Nikon offers NEF Compressed (effectively a lookup-table that compresses 14-bit raw to a 12-bit-equivalent precision, lossy but visually lossless); Sony's early default was a lossy "compressed RAW" (after criticism, an "uncompressed" option was added). These lossy modes are closed-source algorithms — LibRaw supports them via reverse engineering but its decoder occasionally diverges slightly from the vendor's. ③ The rise of "lossy RAW". RAW's original spirit is "preserve sensor data losslessly," but 14-bit lossy compression (similar to Lossy DNG) cuts size by 50-70 % with virtually no visible loss — attractive in storage-sensitive scenarios (burst shooting, stills between 4K video clips). Canon CRaw / Sony Compressed RAW / Nikon NEF Compressed all belong here. Long-term, RAW files are sliding toward "lossy but visually lossless" — coincidentally the same design philosophy as JPEG XL / HEIC.

DICOM — 医学影像的封闭城堡 · 扛把子

DICOM — the walled city of medical imaging · heavy hitter

"它不是图片格式,是带 4000 个字段的医疗记录。"

"Not just an image format — a medical record with 4,000 attributes."

1980 年代,医院里 CT、MRI、X-ray、超声各家厂商各做一套协议:GE 的 CT 出不来 Siemens MRI 能读的文件,科室之间没法交换数据,医生想做一次跨设备影像会诊基本不可能。ACR(美国放射学院)与 NEMA(美国电气制造商协会)1985 年合作发布 ACR-NEMA 1.0,1993 年改名 DICOM 3.0 并加入网络协议。DICOM 同时定义了三件事:(a) 文件格式——一个 .dcm 既是图像也是患者完整病历;(b) 网络协议 DIMSE——医院里 CT 跟 PACS 之间的传输怎么走;(c) 元数据字典——4000+ 标准 tag 涵盖患者姓名、研究日期、modality、像素数据、窗宽窗位等任何医疗影像可能需要的字段。这套体系后来成了医院 IT 的事实标准——全球任何 CT / MRI / 超声 / 病理切片设备出厂时都说 DICOM,任何 PACS / EHR / 工作站默认输入也是 DICOM。30 年没人能挑战,因为它解决的不是"压像素",而是整个医疗影像的协议栈。

In the 1980s, hospital CT / MRI / X-ray / ultrasound vendors each defined their own protocols: a GE CT image couldn't be opened by a Siemens MRI station, departments couldn't exchange data, and a multi-modality consult was effectively impossible. ACR (American College of Radiology) and NEMA (National Electrical Manufacturers Association) jointly released ACR-NEMA 1.0 in 1985, then renamed it DICOM 3.0 in 1993 and added a network protocol. DICOM defines three things at once: (a) a file format — one .dcm is simultaneously an image and a complete patient record; (b) a network protocol, DIMSE — how images move between a CT scanner and a PACS server inside a hospital; (c) a metadata dictionary — 4,000+ standard tags covering patient name, study date, modality, pixel data, window width / level, and every medical-imaging attribute imaginable. The whole stack became the de-facto standard of hospital IT: every CT / MRI / ultrasound / pathology-slide device on Earth speaks DICOM out of the box, every PACS / EHR / workstation reads DICOM by default. 30 years later it remains unchallenged — because what it solved isn't "compressing pixels" but the entire medical-imaging protocol stack.