These are some of the things that I learnt while writing a project that's reached roughly 15k loc in my current job.
As with any technical advice, take it with a grain of salt.
One of the main advantages of golang is explicit error handling. if err != nil allows you to handle errors in a very explicit way. If you're writing an application that should never crash, you should never use panic. Instead, you should return an error and handle it in the caller function. Panics stop the application. Error handling doesn't.
Let's say we have a struct that represents a user.
type User struct {
CreatedAt time.Time
Email string
Name string
}
One way to create a new user is to write a variable like this and specify the values of each field manually
user := User{
CreatedAt: time.Now(),
Email: "john@example.com",
Name: "John Doe",
}
But there's a problem with this approach. If we forget to specify a field, it will be set to the default value of the type. For example, if we forget to specify the Name field, it will be set to "".
In most cases, this is not what we want.
This is why you can often see tutorials / codebases using a function called NewX that returns a value of type X. For example, we can write a function called NewUser that returns a new User struct.
func NewUser(createdAt time.Time, name, email string) User {
return User{
CreatedAt: createdAt,
Email: email,
Name: name,
}
}
This way if we create a new User by using the NewUser function, we have to specify all the fields.
user := NewUser(time.Now(), "John Doe", "john@example.com")
If working on big projects, using function to create the structs reduces the chance of forgetting to specify a field. Thus, reducing the chance of bugs.
Using NewX wrapper functions is also helpful when you're dealing with customly created structs that have time.Time fields.
Let's assume that we use the struct to insert a new user in the database. In order to standartize the time format, we want to ensure that the CreatedAt field is always in UTC. We can do this by creating a NewUser function that converts the CreatedAt field to UTC.
func NewUser(name, email string) User {
return User{
CreatedAt: time.Now().UTC(),
Email: email,
Name: name,
}
}
If we create new users only using the defined function, then we can be certain that the CreatedAt field will always be in UTC.
It's quite common to see global state in go projects. While it's not a bad thing per se, there are some disadvantages to it in the long term. More on them later.
One of the most frequent use cases for global state is database connections. A lot of tutorials write examples like this:
package db
// Conn is the database connection
var Conn *sql.DB
// Connect connects to the database
func Connect() error {
db, err := sql.Open("postgres", "...")
if err != nil {
return err
}
Conn = db // Store the connection in the global variable
}
package main
import "my-project/db"
func main() {
// Initialize the db connection and store it in the db.Conn global variable
if err := db.Connect(); err != nil {
log.Fatal(err)
}
// Do something with the variable which holds the db connection
data, err := db.Conn.Query("SELECT * FROM users")
if err != nil {
log.Fatal(err)
}
}
So how do we get rid of global state? The answer is dependency injection. Most of the tutorials that I have read on this subject are a bit too complex, and don't really explain why it's useful. So the easiest way to explain it is:
pass down state to places which need access to it
For example, refactoring the code above to use dependency injection would look like this:
package db
// Db struct holds the database connection and has access to functions ( methods ) associated with it
type Db struct {
conn *sql.DB
}
// Conn returns the database connection
func (db *Db) Conn() *sql.DB {
return db.conn
}
// Connect connects to the database
func NewDb(host, port, user, password, string) (*Db, error) {
conn, err := sql.Open("postgres", "...")
if err != nil {
return nil, err
}
return &Db{conn: conn}, nil
}
Just as we used the NewUser function to create a new user, we can use the NewDb function to create a struct which holds a database connection.
Now, lets assume that we have an application which needs 2 dependencies to work - environment variables which are stored in the config and a database connection. We can create a struct which initializes and holds these dependencies.
Just to be clear, dependency is a fancy way of saying struct in this case
package app
type App struct {
env *settings.Env // evironment variables read on initialization
db *db.Db // struct which holds database connection and custom util methods associated with it
}
// Db returns the database connection
func (a *App) Db() *db.Db { return a.db }
// Env returns the environment variables
func (a *App) Env() *settings.Env { return a.env }
func NewApp(ctx contxt.Context, configPath string) (*App, error) {
env, err := settings.NewEnv(configPath)
if err != nil {
return nil, err
}
// setup the db conn using the read env config
db, err := db.NewDb(env.DbHost, env.DbPort, env.DbUsername, env.DbPassword)
if err != nil {
return nil, err
}
return &App{db, env}, nil
}
// Exit is called when the application is closed.
func Exit(ctx context.Context, app *App) error {
// close the database connection
if err := app.Db().Close(); err != nil {
return err
}
return nil
}
If using global state, then this would be calling 2 global variables from 2 different packages that are initialized from the main function.
Lets assume that the number of dependencies grows from 2 to 10, which all need initialization. At some point, keeping a track of all of them becomes a nightmare. So to avoid this, instead of initializing the dependencies seperately, we define a setup function which handles all of this logic and returns all of the state that we need.
With dependency injection, you can group these values into a single struct and use it as the state of your application that is passed down to the places which need it.
package myservice
type MyService struct {
// set the app struct as a field so that we can access the db + env vars
app *app.App
}
// Once the app is initialized, we pass it down to the MyService
// struct so that it has access to the db + env vars
func NewService(app *app.App) *MyService {
return &MyService{app: app}
}
func (s *MyService) DoSomethingWithDb() error {
db := s.app.Db()
// ... do something with the db
return nil
}
package main
import (
"context"
"log"
"my-project/app"
"my-project/myservice"
)
func main() {
configPath := "path/to/config"
// initialize the app
app, err := app.NewApp(context.Background(), configPath)
if err != nil {
log.Fatal(err)
}
// close the db connection when the program is exited
defer app.Exit(context.Background())
// pass down the app to the service
service := service.NewService(app)
// use the passed down state to do something with the db
if err := service.DoSomethingWithDb(); err != nil {
log.Fatal(err)
}
}
One advantage of using dependency injection is that it makes testing easier. As we're relying on passing down the state to the places which need it, code becomes more deterministic.
package mypackage
func TestSomething(t *testing.T) {
// Create a new app
app, err := app.NewApp(context.Background(), "path/to/config")
if err != nil {
t.Fatal(err)
}
service := service.NewService(app)
if err := service.DoSomethingWithDb(); err != nil {
log.Fatal(err)
}
}
But there's a problem when running tests on all of the packages, assuming that relative paths are used in the NewApp function. As multiple packages can be defined on multiple levels, the relative path to the config file will be different for each package. Thus the only real way of setting it correctly is using environment variables with absolute paths + variadic input params for the NewApp function.
First off, we can create a new function which returns the path to the config file based on the input params / env variables.
// GetConfigPath returns the path to the config file.
func GetConfigPath(path ...string) string {
if len(path) == 1 {
return path[0]
}
envPath, ok := os.LookupEnv("GO_CONF_PATH")
if ok && envPath != "" {
return envPath
}
return "./conf/config.toml"
}
Then we can refactor the NewApp function to use the GetConfigPath function.
func NewApp(ctx context.Context, configPath ...string) (*App, error) {
env, err := settings.NewEnv(GetConfigPath(configPath...))
if err != nil {
return nil, err
}
// setup the db conn using the read env config
db, err := db.NewDb(env.DbHost, env.DbPort, env.DbUsername, env.DbPassword)
if err != nil {
return nil, err
}
return &App{db, env}, nil
}
After this refactor, we can run tests on all of the packages by setting the GO_CONF_PATH env variable.
GO_CONF_PATH="$(pwd)/conf/config.test.toml" go test ./... -count=1
-count=1 is used to avoid caching
Whenever you need to deal with reflections to validate a value, you should always check if the value is of the type that you expect it to be. When using the value.(float64) syntax, if the value is not of the type float64, the application will panic. To avoid this, you should always check if the value is of the type that you expect it to be, using the ok boolean,
func GetUser(value interface{}) error {
user, ok := value.(User)
if !ok { // if the input interface does not match the User struct, return an error.
return fmt.Errorf("value is not of type User")
}
// do something with the user
}
The way you structure the project will be soley based on what you're doing. If you're writing a simple package or a basic backend server, then the following guidelines are an overkill. But if you're working on a codebase which would share packages and would be able to build multiple projects, then this is a guideline that has worked well for me.
To avoid clutter and reduce the shared code surface, you should make functions and structs unexportable by default. This means that everything starts with a lowercase letter.
If project will be built and run on a very stripped down linux distro, there might be a problem if the operating system does not have the timezone data. To avoid this, you can embed the timezone data in the binary, by adding the following line to the build command
go build -tags timetzdata cmd/my-project/main.go
Carmack has explained these things better than i will ever do. So just read this: John Carmack on Functional Programming in C++
Let's say that you're writing flow for making http requests with authentication headers.
func requestNewAccessToken() (*Jwt, error) {
// ... make a request and validate if the returned jwt is valid
return jwt, nil
}
func requestDataWithAuthToken(jwt *Jwt) error {
accessToken := jwt.AccesToken
// .. do stuff with the access token
}
To be clear, there is nothing wrong with the upper example. But it can be slightly better.
If in this hypothetical scenario, requestDataWithAuthToken function only needs access to one of the values from the passed down jwt, you can refactor such functions to take the input as a string.
func requestNewAccessToken() (string, err) {
// ... make a request and validate if the returned jwt is valid
return jwt.AccesToken, nil
}
func requestDataWithAuthToken(accessToken string) err {
// .. do stuff with the access token
}
This approach has a few advantages:
requestNewAccessToken function or the Jwt struct without editing the requestDataWithAuthToken function. As long as it recieves a correct string, everything will work.