---
title: "Java 11 Upgrade Tip: Don't Rely on Generated serialVersionUID"
summary: "When upgrading to Java 11, be careful of changes to generated serialVersionUID."
date: 2020-06-24
authors: [jonathan-haber]
topics: [Backend, DevEx]
canonical: https://product.hubspot.com/blog/java-11-dont-rely-on-generated-serialversionuid
cover:
  scene: java-11-dont-rely-on-generated-serialversionuid
  alt: "A key opens a chest of saved data; then a little ghost floats off it, taking one of its teeth, and the key no longer turns the lock, which glows."
---

At HubSpot, pretty much all of our backend services are written in Java. As part of keeping our Java stack up-to-date, we've been working on the upgrade from Java 8 to Java 11 for a long time. Doing this sort of upgrade smoothly and safely across a codebase as large as ours, while not disrupting the workflow of our hundreds of backend engineers, is no small feat. This post will cover a snag we hit when we thought we were finally done.

We'd been building all of our apps with Java 11 for over a year (but still targeting Java 8 bytecode). We'd been running all of our apps on Java 11 in production for five months. We'd been targeting Java 11 in all of our leaf modules (ie, not libraries) for three weeks. We upgraded ASM, CGLIB, and ByteBuddy everywhere. We fixed all of the dependency issues with jaxb. We forked Spark 2.4 to make it work on Java 11. We thought we had done our due diligence and were ready to target Java 11 everywhere.

## Targeting Java 11

So we went ahead and made the change to target Java 11 globally (which we announced far in advance). Services picked up this change throughout the day as they rebuilt and redeployed. A few hours later, a report came in of an issue in our QA environment, with this stacktrace included (class names replaced for brevity):

```
Caused by: java.io.InvalidClassException: com.hubspot.SomeClass; local class incompatible: stream classdesc serialVersionUID = 4184345964864034491, local class serialVersionUID = 4704675227730405865
	at java.base/java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:689)
	at java.base/java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1960)
	at java.base/java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1829)
	at java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2117)
	at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1646)
	at java.base/java.io.ObjectInputStream.readObject(ObjectInputStream.java:464)
	at java.base/java.io.ObjectInputStream.readObject(ObjectInputStream.java:422)
	at com.hubspot.HBaseManager.readFromHBase(HBaseManager.java:75)
```

The `InvalidClassException` suggests that the error is related to Java serialization. Before this, if you asked me whether HubSpot uses Java serialization to persist data, I would've said "Of course not, it's not 2009 anymore!" But it turns out that a relatively important service was accidentally using Java serialization to store data in HBase, and the class it was serializing did not declare a `serialVersionUID`. And for some reason, the Java 11 upgrade changed the computed `serialVersionUID`, meaning that this service could no longer read data out of HBase. Luckily, our automated monitoring and alerting detected the issue in our QA environment, so there was no impact to our production environment. In response, we rolled back the upgrade and started to investigate.

## Investigation

As a sanity check, we first confirmed that the Java 8 and Java 11 versions of the class did in fact have different `serialVersionUID`s. We checked this using the `serialver` tool included with the JDK.

```sh
➜  ~ serialver -classpath HBaseLibrary-java8.jar com.hubspot.SomeClass
com.hubspot.SomeClass:    private static final long serialVersionUID = 4184345964864034491L;
➜  ~ serialver -classpath HBaseLibrary-java11.jar com.hubspot.SomeClass
com.hubspot.SomeClass:    private static final long serialVersionUID = 4704675227730405865L;
```

So, targeting Java 11 is changing the `serialVersionUID`, but why? To be fair, the [Javadoc](https://docs.oracle.com/javase/8/docs/api/java/io/Serializable.html) for `Serializable` makes it clear that `serialVersionUID` can break across compiler implementations:

> "\[I]t is _strongly recommended_ that all serializable classes explicitly declare serialVersionUID values, since the default serialVersionUID computation is highly sensitive to class details that may vary depending on compiler implementations, and can thus result in unexpected InvalidClassExceptions during deserialization. Therefore, to guarantee a consistent serialVersionUID value across different Java compiler implementations, a serializable class must declare an explicit serialVersionUID value."

But looking at the [specification for computing a `serialVersionUID`](https://docs.oracle.com/javase/8/docs/platform/serialization/spec/class.html#a4100), it wasn't clear why it would be different.

The source code hadn't changed, so the class name, modifiers, interfaces, fields, constructors, and methods should have all been the same. However, we were able to see the problem using `javap`, the class file disassembler included with the JDK.

At HubSpot, we generate immutable data classes and builders at compile time using a library called [Immutables](https://github.com/immutables/immutables). However, the generated code isn't very pretty to look at, so I crafted a heavily simplified version:

```java
package com.hubspot;

import java.io.Serializable;
import java.util.Objects;

public class SomeClass implements Serializable {
  private final long id;
  private final String name;

  private SomeClass(long id, String name) {
    this.id = id;
    this.name = name;
  }

  public static Builder builder() {
    return new Builder();
  }

  public long getId() {
    return id;
  }

  public String getName() {
    return name;
  }

  public static final class Builder {
    private Long id = null;
    private String name = null;

    private Builder() {}

    public Builder setId(Long id) {
      this.id = id;
      return this;
    }

    public Builder setName(String name) {
      this.name = name;
      return this;
    }

    public SomeClass build() {
      Objects.requireNonNull(id, "id");
      Objects.requireNonNull(name, "name");
      return new SomeClass(id, name);
    }
  }
}
```

First, we compile this class targeting Java 11 and then run it through `javap`:

```sh
➜  ~ javac --release 11 com/hubspot/SomeClass.java
➜  ~ javap com/hubspot/SomeClass
Compiled from "SomeClass.java"
public class com.hubspot.SomeClass implements java.io.Serializable {
  public static com.hubspot.SomeClass$Builder builder();
  public long getId();
  public java.lang.String getName();
}
```

Next, we do the same thing targeting Java 8:

```sh
➜  ~ javac --release 8 com/hubspot/SomeClass.java 
➜  ~ javap com/hubspot/SomeClass
Compiled from "SomeClass.java"
public class com.hubspot.SomeClass implements java.io.Serializable {
  public static com.hubspot.SomeClass$Builder builder();
  public long getId();
  public java.lang.String getName();
  com.hubspot.SomeClass(long, java.lang.String, com.hubspot.SomeClass$1);
}
```

And here we see the problem; when targeting Java 8, an extra package-private constructor is included. And the specification for Java serialization says that all non-private constructors are included in the `serialVersionUID` calculation. So now we're starting to see what's going on; when targeting Java 11, a package-private constructor is removed, which changes the computed `serialVersionUID`. But what's the deal with this constructor? Where is it coming from and why doesn't it appear in the Java 11 class file?

## Synthetic Constructors

The first thing to note is that `SomeClass` has a builder, which is an inner class. And this inner class calls the private constructor of `SomeClass`. But the builder is compiled to a separate class file, which won't be allowed to access this private constructor at runtime. In order to support this, the Java compiler (before Java 11) generated a synthetic, package-private constructor. The builder class can then use this package-private constructor, rather than invoking the private constructor directly. In Java 11, however, [JEP-181](https://openjdk.java.net/jeps/181) landed. This JEP enhanced the class file format and JVM to directly support these inner class accesses, which eliminates the need to generate a synthetic constructor. And now we fully understand why our `serialVersionUID` changed when targeting Java 11.

## Fixing the Issue

In the short term, we plan to use our tooling for automated refactoring to find serializable classes that don't declare a `serialVersionUID` and set an explicit value (equal to the current computed value). Next, we will start failing the build if any serializable classes are detected that don't declare a `serialVersionUID`. These changes should allow us to finish rolling out Java 11 safely. In the longer term, we plan to find and remove any lingering usages of Java serialization, as there are almost always better options available.
