Raster images and pixels

Why you need this

Every sprite, tile, and background in this cluster is a raster image: a grid of numbers. The rules on the hub page about integer display scale, lossless files, and indexed palettes all come from how those numbers are stored and resampled. Once you can picture the grid, each rule turns from a superstition into a consequence.

The idea

A bitmap is a grid

A raster image (or bitmap) is a grid $W$ pixels wide and $H$ pixels tall. A pixel is one cell of that grid holding one color. Coordinates are $(x, y)$ with $x$ increasing to the right, $y$ increasing downward, and $(0, 0)$ at the top-left corner. A 16-pixel character on a 16 by 24 canvas has $W = 16$, $H = 24$, and $384$ pixels, and the pixel at $(15, 23)$ is the bottom-right one.

Channels

In true color each pixel stores three numbers, the red, green, and blue channels, each an integer from 0 to 255. #000000 is all three at 0 (black), #ffffff is all three at 255 (white), and #c96a5a is red 201, green 106, blue 90, a terracotta. Three channels at 8 bits each is 24 bits per pixel, which is where "24-bit color" comes from.

Alpha

The alpha channel is a fourth number, 0 to 255, that says how opaque the pixel is. 0 is fully transparent: the pixel contributes nothing and whatever is behind it shows through. 255 is fully opaque. 128 is about half (128/255 is 0.502), so the pixel and the background mix roughly evenly. Drawing a pixel with alpha $a$ (scaled to 0..1) over a background computes, per channel,

$$\text{out} = a \cdot \text{sprite} + (1 - a) \cdot \text{background}$$

With a sprite red of 201, a background red of 40, and alpha 128: $0.502 \cdot 201 + 0.498 \cdot 40 = 100.9 + 19.9 = 120.8$, stored as 121.

Pixel artists mostly use alpha as a hard mask, 0 or 255 and nothing in between. Partial alpha is reserved for things like shadows and glows that are meant to blend.

Premultiplied alpha, and the dark fringe

There are two ways to store a pixel with alpha. Straight alpha stores the color as drawn plus the alpha separately. Premultiplied alpha stores each color channel already multiplied by the alpha, so a half-transparent red 201 is stored as 101; engines prefer it because blending becomes a simple add. The trouble comes when a texture is filtered or resized under the wrong assumption. A transparent pixel is usually stored as black with alpha 0. If a filter averages that black with a neighboring opaque orange as if both were straight colors, the result is a dark, muddy orange at half alpha, and every edge of the sprite grows a dark fringe. Dark halos in the engine that are absent in your editor point at the alpha mode first.

Indexed color

An indexed image does not store a color per pixel. It stores a palette, a table of at most 256 colors, and one small number per pixel: the index into that table. A pixel with index 3 is whatever color sits in row 3. Pixel artists work indexed for three reasons. The palette is enforced: you cannot accidentally place a color one step off. Recoloring is instant: change row 3 and every pixel using it changes. And the file is small: one byte per pixel instead of three or four. Palettes and ramps covers how the rows are chosen.

PNG versus JPG

PNG is lossless: the decoded pixels are exactly the ones you saved, and it supports indexed color and alpha. JPG is lossy: it splits the image into 8 by 8 blocks, converts each block into frequencies, and throws away the high frequencies to save space. A hard edge between two flat colors is nothing but high frequency, so JPG blurs it and adds faint ripples on either side, called ringing: a one-pixel black outline against green comes back as a dark gray line with a greenish-gray pixel on one side and a slightly too-bright pixel on the other. Flat areas gain speckles, the block boundaries show faintly, and a sprite that used 8 colors now uses several hundred, so it can never be indexed again. JPG has no alpha at all. Save pixel art as PNG, always, and never let a JPG into the pipeline at any stage.

Worked example

Nearest-neighbor versus bilinear

Scaling up asks for output pixels that sit between source pixels. Nearest-neighbor picks the closest source pixel and copies it. Bilinear takes a weighted average of the four nearest source pixels. Here is a 2 by 2 grayscale source, one value per pixel, scaled to 4 by 4.

 10 200
 60 120

Nearest-neighbor at 2x: each source pixel becomes a 2 by 2 block of itself.

 10  10 200 200
 10  10 200 200
 60  60 120 120
 60  60 120 120

Bilinear, with the convention that the four corner output pixels land exactly on the four source pixels and the rest are spaced evenly between them, so each output row samples the source at fractions 0, 1/3, 2/3, 1 (values rounded):

 10  73 137 200
 27  76 124 173
 43  78 112 147
 60  80 100 120

The top-right of the middle row, for instance, is $\tfrac{2}{3} \cdot 200 + \tfrac{1}{3} \cdot 120 = 173.3$. Only the four corners kept an original value; twelve of the sixteen output pixels are colors that never existed in the source. For a photograph that is smoothing. For pixel art it is the whole drawing dissolving into mud, and the palette is gone.

Integer versus non-integer scale

At 2x every source pixel is exactly 2 by 2 on screen. At 3x, 3 by 3. Every pixel stays the same size and the drawing keeps its proportions.

At 1.5x, a 4-pixel-wide sprite has to fill 6 screen pixels. Nearest-neighbor gives the source columns widths of 2, 1, 2, 1: some pixels twice as wide as their neighbors. Move the sprite one screen pixel to the right and the rounding changes, so the widths become 1, 2, 1, 2 and a different set of pixels is fat. As the sprite walks, the fat columns crawl through it, which is the shimmer you see in games that scale by a non-integer. Bilinear hides the shimmer by blurring everything instead. Neither is acceptable, which is why the display scale in this cluster is always a whole number. The "scaling" demo on Sprite sheets and export shows one 16-pixel sprite at 2x, 3x, and 2.5x side by side.

Nearest-neighbor in code

This scales a 2D array by an integer factor k. Output pixel $(x, y)$ reads source pixel $(\lfloor x / k \rfloor, \lfloor y / k \rfloor)$.

function scaleNearest(grid, k) {
  const out = [];
  for (let y = 0; y < grid.length * k; y++) {
    const row = [];
    for (let x = 0; x < grid[0].length * k; x++) {
      row.push(grid[Math.floor(y / k)][Math.floor(x / k)]);
    }
    out.push(row);
  }
  return out;
}
scaleNearest([[10, 200], [60, 120]], 2);

Pasted into a browser console, the last line returns the 4 by 4 nearest-neighbor grid above.

In a game's art pipeline

Stage 1, deciding the rules, fixes the canvas size in pixels and the display scale. Stage 2 draws indexed, on a palette, so the pixel count and color count stay honest. Stage 5, shipping, is where this page pays off: the sprite sheet is exported as PNG with straight or premultiplied alpha matching what the engine expects, the engine's texture filter is set to nearest-neighbor, and the camera scales by 2x, 3x, or 4x and never by a fraction. Pixel art fundamentals turns these facts into the drawing rules.

Common mistakes

• Saving a work-in-progress as JPG. Symptom: soft outlines, speckled flat areas, and the color picker reporting dozens of near-identical colors where you placed one.
• Leaving the engine's texture filter on linear. Symptom: every sprite looks slightly out of focus, worst on 1-pixel outlines, even at an exact 2x.
• Letting the camera zoom to 2.5x. Symptom: sprites shimmer as they move, and a row of identical tiles has some tiles one pixel wider than others.
• Exporting straight alpha into an engine expecting premultiplied, or the reverse. Symptom: dark or light halos on every transparent edge.
• Painting in true color and "reducing to a palette later". Symptom: anti-aliased edges with a dozen in-between colors, and hours of cleanup that indexed mode would have prevented.

Cost

Storing an image costs $W \times H$ pixels times the bytes per pixel: 4 for true color with alpha, 1 for indexed. A 256 by 256 indexed sheet is 64 KB raw, and PNG compresses flat-colored pixel art far below that. Nearest-neighbor scaling touches each output pixel once, so scaling by a factor $k$ costs $O(k^2 W H)$ time, done every frame on the graphics card where it is effectively free. Bilinear costs four reads per output pixel instead of one, still free in practice; its real cost is that it destroys the art. Indexed mode costs the artist nothing extra and saves time on every recolor.

Going further

• Pixel art fundamentals, for the drawing rules that follow from the grid.
• Palettes and ramps, for how the rows of an indexed palette are chosen.
• Sprite sheets and export, for the scaling demo and the export checklist.
• Color basics, for what the three channel numbers mean to the eye.