ULID: Universally Unique Lexicographically Sortable Identifier

Using ULID identifiers in Go programs running on Postgres database.

The UUID format is a highly popular and amazing standard for unique identifiers. However, despite its ubiquity, it can be suboptimal for many common use-cases because of several inherent limitations:

• It isn’t the most character efficient or human-readable.

• UUID v1/v2 is impractical in many environments, as it requires access to a unique, stable MAC address.

• UUID v3/v5 requires a unique seed.

• UUID v4 provides no other information than true randomness, which can lead to database fragmentation in data structures like B-trees, ultimately hurting write performance.

Introducing the ULID Identifier

Few projects I worked on used the ULID (Universally Unique Lexicographically Sortable Identifier), and I really enjoyed working with it, and would love to share this experience with you. Specifically for Go programs using Postgres database. But the same applies to other languages or databases too.

You can find the full spec here - github.com/ulid/spec

ulid() // 01ARZ3NDEKTSV4RRFFQ69G5FAV

What makes ULID great?

ULID addresses the drawbacks of traditional UUID versions by focusing on four key characteristics:

• Lexicographically sortable. Yes, you can sort the IDs. This is the single biggest advantage for database indexing.

• Case insensitive.

• No special characters (URL safe).

• It’s compatible with UUID, so you can still use native UUID columns in your database, for example.

ULID’s structure is key to its sortability. It is composed of 128 bits, just like a UUID, but those bits are structured for function: 48 bits of timestamp followed by 80 bits of cryptographically secure randomness.

 01AN4Z07BY      79KA1307SR9X4MV3

|----------|    |----------------|
 Timestamp          Randomness
   48bits             80bits

Go+Postgres example

The power of ULID is its seamless integration into existing systems, even those relying on the UUID data type. Here is a demonstration using Go with the popular pgx driver for PostgreSQL and the oklog/ulid package.

The code below first connects to a running PostgreSQL instance and creates a table where the primary key is of type UUID. We then insert records using both standard UUID v4 and ULID.

package main

import (
  "context"
  "fmt"
  "os"

  "github.com/google/uuid"
  "github.com/jackc/pgx/v5"
  "github.com/oklog/ulid/v2"
)

func main() {
  ctx := context.Background()

  conn, err := pgx.Connect(ctx, "postgres://...")
  if err != nil {
    panic(err)
  }
  defer conn.Close(ctx)

  _, err = conn.Exec(ctx, `
CREATE TABLE IF NOT EXISTS ulid_test (
  id UUID PRIMARY KEY,
  kind TEXT NOT NULL,
  value TEXT NOT NULL
);`)
  if err != nil {
    panic(err)
  }

  insertUUID(ctx, conn, “1”)
  insertUUID(ctx, conn, “2”)
  insertUUID(ctx, conn, “3”)
  insertUUID(ctx, conn, “4”)
  insertUUID(ctx, conn, “5”)

  insertULID(ctx, conn, “1”)
  insertULID(ctx, conn, “2”)
  insertULID(ctx, conn, “3”)
  insertULID(ctx, conn, “4”)
  insertULID(ctx, conn, “5”)
}

func insertUUID(ctx context.Context, conn *pgx.Conn, value string) {
  id := uuid.New()
  conn.Exec(ctx, "INSERT INTO ulid_test (id, value, kind) VALUES ($1, $2, 'uuid')", id, value)

  fmt.Printf("Inserted UUID: %s\n", id.String())
}

func insertULID(ctx context.Context, conn *pgx.Conn, value string) {
  id := ulid.Make()

  // as you can see, we don’t need to format the ULID as a string, it can be used directly
  conn.Exec(ctx, "INSERT INTO ulid_test (id, value, kind) VALUES ($1, $2, 'ulid')", id, value)

  fmt.Printf("Inserted ULID: %s\n", id.String())
}

The oklog/ulid package implements the necessary interfaces (specifically, database/sql/driver.Valuer and encoding.TextMarshaler) that allow it to be automatically converted into a compatible format (like a string or []byte representation of the UUID) that the pgx driver can successfully map to the PostgreSQL UUID column type. This allows developers to leverage the sortable advantages of ULID without having to change the underlying database schema type in many popular environments.

This allows developers to leverage the sortable advantages of ULID without having to change the underlying database schema type in many popular environments.

Sortability and Performance

The time-based prefix means that new ULIDs will always be greater than older ULIDs, ensuring that records inserted later will be physically placed at the end of the index. This contrasts sharply with UUID v4, where the sheer randomness means records are scattered throughout the index structure.

With traditional UUID v4, sorting records by their insert time is not possible without an extra column. When using ULID, the sort order is inherent in the ID itself, as demonstrated by the following database query output:

select * from ulid_test where kind = 'ulid' order by id;

019aaae4-be9c-d307-238f-be1692b3e8d7 | ulid | 1
019aaae4-be9d-011f-b82e-b870ca2abe9d | ulid | 2
019aaae4-be9f-e9d7-6efc-5b298ecc572b | ulid | 3
019aaae4-bea0-deae-6408-d89e7e3ce030 | ulid | 4
019aaae4-bea1-8ed2-c2f5-144bb1ffedde | ulid | 5

As we can see, the records are returned in the same order they were inserted. Furthermore, ULID is much shorter and cleaner when used in contexts like a URL:

/users/01KANDQMV608PBSMF7TM9T1WR4

ULID can generate 1.21e+24 unique ULIDs per millisecond, which should be more than enough for most applications.

Limitations and the future

There’s really no major drawback to using ULID, but you should understand its limitations. For very (and I mean very) high-volume write systems, ULIDs can become problematic. Since all writes are clustered around the current timestamp, you will have hot spots around the current index key, which can potentially lead to slower writes and increased latency on that specific index block.

While other alternative identifiers exist, such as CUID or NanoID, the benefits of ULID have become a major factor in the evolution of unique identifier standards.

It is worth noting that the newest proposed standard for unique identifiers, UUID v7, aims to address the sortability and database performance issues of older UUID versions by adopting a similar time-ordered structure to ULID.

Resources

• https://github.com/ulid/spec

• https://github.com/oklog/ulid

• https://github.com/jackc/pgx

• Watch on our Youtube channel