Go claims to be a programming language that makes it easy to build simple, reliable, and efficient software.

Let’s explore those claims with a simple backend system that consumes a JSON object and stores it into a Postgres database in such a way that it highlights the issues a newcomer to go would have when coming from other programming languages.

JSON support

The JSON we will be consuming from an API looks like this:

[
  {
    "name": "Batman",
    "tags": [
      "martial artist",
      "strategist"
    ],
    "nextVaccinationDate": "2022-03-26T13:00:00-0700"
  }
]

We should be able to write some glue code to transform the JSON into a struct:

type Superhero struct {
  Name                string    `json:"name"`
  Tags                []string  `json:"tags"`
  Awards              int       `json:"awards"`
  NextVaccinationDate time.Time `json"next_vaccination_date"`
}

data := "[{\"name\":\"Batman\",\"tags\":[\"martial artist\",\"strategist\"],\"awards\":15,\"next_vaccination_date\":\"2022-03-26T13:00:00-0700\"}]"
var superhero []Superhero
err := json.Unmarshal([]byte(data), &superhero)
if err != nil {
    log.Panic("could not parse JSON: ", data)
}
fmt.Println("Hello ", superhero)

The output of the program is:

Panic: could not parse JSON: parsing time ""2022-03-26T13:00:00-0700"" as ""2006-01-02T15:04:05Z07:00"": cannot parse "-0700"" as "Z07:00"

Not good. The date is not parsing correctly. Judging by the JSON data, it would seem the date is in ISO 8601 format. Maybe go is using a different format by default?

Let’s pick a different format for the JSON parser. I would expect this to be a simple one-liner as it is a very common task in backend systems.

It turns out you can’t. Changing the format for time.Time json parser is not possible. As a workaround, we can write a custom type and a custom parser.

Why do we need to write a custom type for this? In my opinion, Go should have more powerful annotations to allow for this use case.

type SuperheroTime struct {
	time.Time
}

Let’s write the UnmarshalJSON function. We just need to find the format to parse ISO dates. We are looking for a constant similar to this Java ISO_OFFSET_DATE_TIME one.

Yet, there is no such constant in time.Time. Why?

Let’s write our own. This should typically mean writing a formatter in the form of yyyy-MM-dd.

Not in go. You need to write the date/time format as it would look like at a very specific date. Which date? I still haven’t figured this out.

func (st *SuperheroTime) UnmarshalJSON(b []byte) (err error) {
  // We need to remove the quotes from the input string. Because?
  s := strings.Trim(string(b), "\"")
  if s == "null" {
    st.Time = time.Time{}
    return
  }
  
  // Why 2006-01-02? Is this the day before the month or viceversa?
  st.Time, err = time.Parse("2006-01-02T15:04:05-0700", s)
  return
}

Running the program gives the correct output (I think? Why is `-07:00 twice in there?):

Hello  [{Batman [martial artist strategist] 15 2022-03-26 13:00:00 -0700 -0700}]

Database support

Let’s store the data in Postgres. First, let’s create the table:

CREATE TABLE superhero
(
  id        SERIAL NOT NULL PRIMARY KEY,
  name      TEXT NOT NULL,
  tags      TEXT[] NULL,
  awards    INTEGER NULL,
  next_vaccination_date TIMESTAMP NULL
);

We then edit the struct definition so the database driver serializes our superhero:

type Superhero struct {
	Name                string        `json:"name" db:"name"`
	Tags                []string      `json:"tags" db:"tags"`
	Awards              int           `json:"awards" db:"awards"`
	NextVaccinationDate SuperheroTime `json:"next_vaccination_date" db:"next_vaccination_date"`
}

func main() {
	data := "[{\"name\":\"Batman\",\"tags\":[\"martial artist\",\"strategist\"],\"awards\":15,\"next_vaccination_date\":\"2022-03-26T13:00:00-0700\"}]"
	var superheros []Superhero
	err := json.Unmarshal([]byte(data), &superheros)
	if err != nil {
		log.Panic("could not parse JSON: ", err)
	}
	fmt.Println("Hello ", superheros)

	db := sqlx.MustConnect("postgres", "user=user password=abc host=localhost port=3306 dbname=db sslmode=disable")
	defer db.Close()

	stmt, err := db.PrepareNamed("INSERT INTO superhero (" +
		"name, tags, awards, next_vaccination_date) " +
		"VALUES (:name, :tags, :awards, :next_vaccination_date)")
	if err != nil {
		log.Panic("Unable to store superhero (1)", err)
	}

	stmt.MustExec(&superheros[0])
}

Running the program we hit a new error:

panic: sql: converting argument $2 type: unsupported type []string, a slice of string

For some reason, you can’t directly save String slices to the database even though Postgres supports this feature. The solution is to change the type of Tags to pq.StringArray:

	Tags                pq.StringArray      `json:"tags" db:"tags"`

This code smells. This database implementation detail is not something that should live in the definition of a Superhero struct.

Next run:

panic: sql: converting argument $3 type: unsupported type main.SuperheroTime, a struct

That makes sense. We created a custom time.Time type so the database driver has no clue what to do with it. Let’s fix it with a custom serializer:

func (st *SuperheroTime) Value() (driver.Value, error) {
	return st.Time, nil
}

Let’s run it again:

panic: sql: converting argument $3 type: unsupported type main.SuperheroTime, a struct

What now? As a temporary measure, removing the next_vaccination_date from the INSERT command works as expected.

  stmt, err := db.PrepareNamed("INSERT INTO superhero (" +
		"name, tags, awards) " +
		"VALUES (:name, :tags, :awards)")

Process finished with exit code 0

Nullable types

After some time, Super Woman comes into the picture. We don’t know anything about her except her name:

[
  {
    "name": "Batman",
    "tags": [
      "martial artist",
      "strategist"
    ],
    "awards": 15,
    "nextVaccinationDate": "2022-03-26T13:00:00-0700"
  },
  {
    "name": "Super Woman",
    "tags": null,
    "awards": null,
    "nextVaccinationDate": null
  }
]

Running the program with that input shows that Super Woman received 0 awards. Except it shouldn’t be 0 but null (missing). We don’t know how many awards she has received, but it is not zero. Our data model is thus incorrect.

To fix the above, we need to add nullable types to our Superhero structure.

Except go has no concept of such thing. This can be implemented with sum types/discriminated unions or specifically union types like in Dart or option types as in Scala or even nil pointers. But an int cannot be nil so…

Let’s create an Option type. Given the lack of generics in go. We will need to reinvent the wheel and create an Option type for each nullable type we will ever need. In the case of int64:

// NullInt64 represents an int64 that may be null.
// NullInt64 implements the Scanner interface so
// it can be used as a scan destination, similar to NullString.
type NullInt64 struct {
	Int64 int64
	Valid bool // Valid is true if Int64 is not NULL
}

// Value implements the driver Valuer interface.
func (n NullInt64) Value() (driver.Value, error) {
	if !n.Valid {
		return nil, nil
	}
	return n.Int64, nil
}

The code above is implemented in sql.go so let’s use that instead.

type Superhero struct {
	Name                string         `json:"name" db:"name"`
	Tags                pq.StringArray `json:"tags" db:"tags"`
	Awards              sql.NullInt64  `json:"awards" db:"awards"`
	NextVaccinationDate SuperheroTime  `json:"next_vaccination_date" db:"next_vaccination_date"`
}

Running our code again:

2020/12/29 14:27:02 could not parse JSON: json: cannot unmarshal number into Go struct field Superhero.awards of type sql.NullInt64
panic: could not parse JSON: json: cannot unmarshal number into Go struct field Superhero.awards of type `sql.NullInt64`

In fairness, I wasn’t expecting this to work as sql.NullInt64 is a database specific type. We would need to write a custom JSON parser for it.

Note: after discussing nullable types on Reddit, user cre_ker pointed out that a simple integer pointer would suffice

type Superhero struct {
	Name                string         `json:"name" db:"name"`
	Tags                pq.StringArray `json:"tags" db:"tags"`
	Awards              *int           `json:"awards" db:"awards"`
	NextVaccinationDate SuperheroTime  `json:"next_vaccination_date" db:"next_vaccination_date"`
}

Data structures

Let’s do a simpler task:

• Given a text, find the superheroes for which their tags appear in the text. For example, if the text contains “strategist”. We should return the Batman object.

This should be easy right?

1. Create an empty set
2. Loop through the superheroes, then their tags
3. If the text contains the tag, add the superhero to the set
4. Return the set

Go doesn’t implement sets, but we can implement it with a map:

type Superhero struct {
	Name string
	Tags []string
}

var superheros map[Superhero]bool

Unless we can’t. The code above yields the following compiler error:

Invalid map key type: the comparison operators == and != must be fully defined for key type

In languages such as Java, I would not expect this to work without implementing hashcode() and equals(). In Scala this should work as long as Superhero is a case class. In golang it seems to be impossible to define the “==” operator for such a struct?

Error handling

At first sight, it would seem the language is properly designed to handle errors by forcing developers to acknowledge all error cases of a function call:

result, err := doWork()
if err != nil {
    log.Println("doWork failed", err)
}

However, what happens if doWork panics? The entire app crashes and the message “doWork failed” is not printed. So much for handling errors.

You could in theory use defer and recover to recover from such errors (exceptions?), but why would a language have both panics and error return codes? In reality, you will always have to use recover or hope that all of your code and 3rd party code does not panic.

Final words

Whilst my experience with go lang is most certainly limited, in my opinion, go lang does not live up to its claims of being a language that allows for building simple and reliable software:

• The typing system is not powerful enough to express data models concisely
• It lacks basic features like nullable types that would prevent a broad class of bugs. Moreover, the absence of objects can be expressed in too many variants: nil pointers, empty structs, custom Optional types. This is asking for bugs
• The lack of standard data structures encourages duplication of not only code but their bugs too
• There is too much magic behind the scenes (reflection?), yet there is very little flexibility:
  • The JSON and Database annotations are simple but can barely be adjusted if at all
  • Comparison operators cannot be user-defined
• Error handling is half baked and promotes unreliable software

Epilogue

I wrote this post to highlight all the issues I faced when building a backend service in go.

Whilst I have highlighted what I consider to be several language deficiencies, I would still use go for specific use cases.

For instance, I maintain a Java web service that uses more than 100GB of RAM. A significant chunk of it is lost due to JVM memory inefficiencies. Replacing this system with go would probably be a good fit. I would not use go, however, as my daily backend/web service tool.

All in all, this is my personal opinion. Feel free to use the tool that best suits you.