GRIB Tables 101

Work with GRIB data long enough and you inevitably will hit the issue where your GRIB reader doesn't know how to translate a parameter. Usually when this happens, instead of seeing a recognizable field name like "reflectivity" or "REFD", you'll see something like "discipline 0 category 15 parameter 1" and you're left to figure out for yourself what that means, or it may simply be listed as the even less helpful "unknown". So what gives?

GRIB is self-describing. Kinda.

A self-describing data file is one which contains within it all the metadata necessary to understand its data content. An example of this would be pretty much any common image format - PNG, JPEG, GIF, etc. Each of these contains within it some metadata describing its contents, such as the height and width of the image, color palette, and how the data itself is arranged, and you don't need to rely on any additional data to read one of these files; software that understands the PNG format can read any PNG file. That is also true of GRIB, but there's a caveat. Most of the metadata that's stored in GRIB files is stored as integer values which correspond to a row in a table, and you need those tables to be able to convert the integer values to their human-readable equivalents. These tables are not stored in the GRIB files themselves and are instead shipped along with whatever GRIB reading software you're using, the intention being to keep the size of the GRIB files as small as possible since most of these integer values can be stored in a single byte whereas the equivalent text representations would require significantly more space.

This is why I say GRIB is "kinda" self-describing. If you're blessed with the ability to memorize hundreds of lookup tables, then seeing "discipline 0 category 15 parameter 1" might be enough for you to know you're looking at a base reflectivity field, but the rest of us mere mortals need to rely on the lookup tables. When a GRIB file contains a reference to a row or table that doesn't exist on our local machine we are treated to anything from the aforementioned cryptic output to full-blown software crashes. In addition, there's a subset of tables known as "templates" which identify how the metadata is arranged in the file, and each template is identified by, you guessed it, an integer value stored in the GRIB. So in the case where you're missing these template tables, GRIB files become practically unreadable.

Diving deep into a GRIB table

There are currently 4 historical versions of GRIB tables, and each contains roughly 100 tables each. Some tables define locations of metadata, others indicate the sort of data from a high level (meteorological, hydrological, satellite, etc), and others provide more details on the individual data types. For this article we're going to focus on tables that provide the human-readable explanations of the parameters in the file. Brace yourself as we dive into one of these tables:

Ok so that doesn't look so bad.

The above shows the top 6 rows from table 4.2.0.2. This table naming scheme consists of the message section number (4), minor table version (2), product discipline (0) and product category (2). Fully understanding the first 2 parts requires a broader discussion of the overall GRIB file structure and will be covered in another post, but for now just know that product discipline 0 equates to meteorological products, and table 4.2 allows us to look up parameters by product discipline and category.

Understanding how the 4.2.0.X family of tables are named by category is easy: if the GRIB message specified category 2, we'd use the above table 4.2.0.2; if it specified category 5, the table we'd use to look up would be 4.2.0.5. The parameter number identifies which row in the table to use, and as you can see from the above these numbers begin at zero, not one. So a message (in discipline 0) with category 2, parameter 1 is a wind speed field whose values are in units of meters per second.

As you can see this isn't rocket science; the GRIB file tells you which tables to use and rows to select, all you need is the tables. Simple right?

Gears, meet Mr. Wrench

The more curious among you may have followed the above links, dug around a little in the various tables, and come across the phrase "Reserved for local use". GRIB tables are separated into "master tables" and "local tables". Master tables are the ones everyone should have and typically come shipped with any GRIB reading software or development package. Local tables are used to supplement additional fields and parameters that don't exist in the master table. This could be things like a new type of satellite measurement, or a new derived field. Whatever it is, unless you have the local tables that define these values, you're going to run into the same "category X parameter Y" type output, and unfortunately it's all too frequent to find locally defined parameters with no accompanying GRIB tables to help define what they are.

Another issue can arise if you are using old software which shipped with old master tables. The master tables are revised and edited periodically, and if you don't have the latest changes you may run into the same problem.

Finally, there doesn't exist any sort of standard for how the GRIB tables are represented. They could space-delimited or tab-delimited, separate from the code or hard-coded into the software, and two files with the same name from different packages may not even contain the same columns! This makes updating and adding to GRIB tables cumbersome, sometimes even impossible for the end user.

So...how do I fix it?

Unfortunately there's no one-size-fits-all answer here, simply because there's no standard format for GRIB tables and every software package uses its own variants. If at all possible, your first step should be to contact the creator of your GRIB package for help in adding missing table data. Each vendor has their own set of tables in their own format based on how their code works, so you'll save yourself a lot of trouble by just asking them directly for assistance. The meteorological community is generally full of amazing people who are all too happy to help, especially if you're using their software.

Some software such as NASA's Panoply has at least some built-in support for adding local GRIB tables. Others such as the eccodes library from ECMWF keep all their tables easily available in plain text, and in a format that's fairly intuitive to work through to figure out how to add local fields, as well as providing extensive documentation. If you have a specific package you need help with, post a comment down below or send us an email and let's figure it out together.