Optimizing 16Bit Pixel Art for Classic Consoles
In order to create console-accurate pixel art, there are many restrictions that an artist has to follow to make their work functional. We've talked about 8bit systems in the past (like this NES Graphics Guide and this post about NES level and background art), but today we'd like to enter the 16bit era, and teach you how to create gorgeous art on these beloved consoles.
Many times, the artist will find themselves with a great piece of artwork that falls just short of meeting restrictions. The most common transgressions are against palette limitations or tile counts. This post is going to cover how to identify where your art stands in regard to both of those restrictions and how to successfully reduce them.
This post uses the following tools, and it might be helpful to download them and have them at the ready:
GIMP 2
The free version of Pyxel Edit
There are other options for tools to use for both of these processes, but these are the two that we generally employ due to their accuracy and ease of use.
Now that you're equipped, let's get started.
Palette Limitations
Oftentimes when creating retro pixel art, designers are simply unaware of the restrictions of golden era consoles. In such cases, the first rule they will break is using too many colors. Typically, 16-bit consoles work with 16-color palettes, where the first color of each palette is a shared color that represents transparency. Reducing the color count of an image can be a daunting task, and the automated process usually produces major losses in image quality.
GIMP 2 is a great open-source alternative to Photoshop, and it allows for user-friendly palette manipulation. It offers a range of functions and features to make the life of a retro pixel artist easier.
The first thing you will want to do is open your image in GIMP.
As you can see, this is a nice image, and in certain modes on the SNES, it will work fine. For the sake of this article though, let’s assume it needs to be 16 colors. It definitely has far too many for that to work.
After the image is open, you want to make sure that you have the colormap tab available to you. If it isn’t already available on the left pane, navigate to Windows>Dockable Dialogs>Colormap. If this opens in a new window, simply drag and drop it into the tabs on the left pane.
Now that your colormap tab is open, it is time to index the palette of your image. Go to the menu bar and click on Image>Mode>Indexed
When you click on Indexed, it will open a new window with options on how you want to index your image. You will want to click on “Generate optimum palette” and then set the maximum amount of colors to 255. (Note: You can set this to 16 to automatically reduce the colors down, but more often than not this ends up making the image look pretty bad since the automated process typically doesn’t make good judgements)
Doing this will extract all of the colors from the image and place them inside of your image map. As you can see in our example, it uses far too many colors, and it will have to be greatly reduced.
From here, the artist will be able to edit the colors in their palette freely by using the colormap. You can rearrange them to your liking and add colors. In order to reduce colors, you will want to look at two factors: What colors are extremely similar in value, and what colors aren’t being used very much.
Once you have identified a color you would like to remove, and what color you would like to replace it with, double click on the color that you are going to use to replace it. This will open up your color dialog.
When this window opens, select and copy the HTML notation for the color and then close it. Double click on the color that you are trying to replace, and paste the HTML notation in from the previous color. This will make the two colors have the exact same value.
Continue to do this until you have reduced your color count to your liking, or when you want to see your progress. In order to fully remove the color, go back to the menu and select Image>Mode>RGB to remove the index and then re-index the image.
Now that both colors have the same value they will only take up a single slot in your palette. If you are simply trying to see where color is being used, double click on the color in your colormap, and then change it to a high contrast value. Doing this will change that color in the image as well so that you can easily identify where it is being used.
For example, if I wanted to identify where this color is being used in the image, I could double click on it, and then change it to bright red.
When I do that it will change to bright red in the image so that I can see where it is being used. After that, I would simply undo the change to set it back to normal.
The important thing to keep in mind when doing color reductions is to always reduce the colors that will have the least amount of impact on the overall image. That is why using an automated process doesn’t work out very well - it will take color values into account, but it won’t be able to have their context in mind.
This process is most difficult when you are down to removing the last few colors. Once you are down that far, you will need to make some difficult judgement calls when it comes to what colors to remove, so make sure to try multiple options and make choices that will retain the most detail.
Tile Count
The other restriction that is commonly broken is the unique tile count. 16bit consoles load their tiles into VRAM just like 8-bit consoles do, but VRAM is also used to store other data. That makes it difficult to have a concrete maximum number of tiles to work off of which will lead to needing to reduce tile counts.
The first thing you will want to do is open your image in GIMP, and then go to View>Show Grid
This will overlay a grid on top of your image, but the default grid settings will not work for what we are doing. You will want to go to Image>Configure Grid to open up the grid settings.
In the window that pops up, you want to change the spacing settings to be 8x8 pixels.
The reason you are doing these steps is that you will be working from GIMP to reduce the tile count. In order to do that you need that 8x8 grid overlaid to represent the individual tiles that the image is going to be using. It’s also helpful to go to View>Snap to Grid to turn the grid snapping on.
Now that your canvas is set up, you will want to get your unique tile count. Classic consoles only store unique tiles, so having an automated way to break an image into tiles and then remove any duplicates is a must. This is where Pyxel Edit comes into play. Import your image into Pyxel Edit, and a window will pop up.
Set your tile size to 8x8 and then click on the checkbox next to Identify tiles and Transformations (Note: If you are working on an 8-bit console like the NES, chances are it won’t support transformations, so do not click that checkbox in that case). Then click Import.
This will open the image in two windows. You have your canvas on the left side and your tileset on the right side. We are simply interested in getting a tile count, so click on the very last tile in your tileset, and you should see a blue number appear on your canvas.
That blue number is the Tile ID for the final unique tile in your tileset, so in this case, it reads 1383. That means that the final tile in the set is the 1383rd unique tile in the image, which ultimately means there are 1383 unique tiles in the entire image. So if we ended up having a tile budget of around 900 tiles for a background, this image will need to be greatly reduced.
In order to reduce tiles, go back into GIMP. The only tool you should need for tile reduction is the Rectangle Select tool. Because you had turned on grid snapping earlier it is very easy to select one tile or a selection of tiles and then copy and paste them to different parts of the image.
The main way you are going to be reducing the tile count in your image is finding the areas with the most variance and then copying and pasting tiles so that there are more duplicate tiles in the image, thus reducing the number of unique tiles that appear. If you want to check your progress, you would simply save the image, and then import it into Pyxel Edit again.
In our example, a lot of the tiles that are being created are from where the stone ground is peeking out from the sand. This is a great area to reduce the tile count. While the shapes being created here are very similar, due to their placement it is creating a lot of unique tiles. If a single pixel is different between two similar objects, it will create unique tiles for all of them.
This is easily optimized by choosing one of the shapes as your base and creating the rest of them using your base shape’s tiles. That way all of the shapes are being created with the same art, and no new tiles are being made.
Go Forth and Optimize
Optimizations are not about reducing the quality of art - far from it. They are used to make graphics not only properly formatted for console restrictions but also to be functional with the other elements of the game's design. Optimization is not so much about efficiency as it is about maximization - rendering the art in such a way that the overall quality of the game, and the player's experience, is improved.