Efficient Workflow in Go with Temporal.io: Layered Architecture

TLDR: In this article, we explore how layer patterns and Temporal.io can be combined to efficiently manage workflows and activities in Golang applications. This approach allows for separation of business logic and data access functions, resulting in a more streamlined and modular code structure. We provide code examples and explain the benefits of this architectural style for stronger and more efficient workflow management, reducing errors and streamlining the development process.

In this article, we will explore how layer patterns and Temporal.io can be integrated to create an efficient workflow orchestration for Golang applications. This approach allows for the separation of business logic and data access functions, making the code more streamlined, modular, and maintainable.

to implement event based systems check out this tutorial: Using Temporal to Build Scalable and Fault-Tolerant Applications in Golang

Layered Architecture

This pattern involves separating a system into distinct layers that handle different responsibilities such as presentation, business logic, and data access.

Structure:

main.go
config/
    config.go
repository/
    database.go
service/
    service.go
workflow/
    example_workflow.go
activity/
    example_activity.go
messages/
    example_messages.go

The main.go file is the entry point of the application. It imports the necessary packages and starts the workflow.

The config package contains a config.go file that reads configuration values from a file or environment variables to set up the application.

The repository package contains a database.go file that interacts with the database to retrieve or store data.

The service package contains a service.go file that encapsulates complex business logic and interacts with the database through the repository layer.

The workflow package contains a example_workflow.go file that defines the main workflow logic for Temporal.io engine.

The activity package contains a example_activity.go file that defines the activity logic executed by Temporal.io engine.

The messages package contains a example_messages.go file that defines the message and types.

Here’s an example of the ExampleWorkflow and ExampleActivity functions in example_workflow.go and example_activity.go respectively:

// example_workflow.go
func ExampleWorkflow(ctx workflow.Context) error {
    var data []string
    err := workflow.ExecuteActivity(ctx, ExampleActivity, "example").Get(ctx, &data)
    if err != nil {
        return err
    }

    // pass the data through the service layer
    service := NewExampleService()
    result, err := service.DoSomeComplexBusinessLogic(data)
    if err != nil {
        return err
    }

    // return the result to the caller
    return workflow.SetResult(ctx, result)
}

// example_activity.go
func ExampleActivity(ctx context.Context, input string) ([]string, error) {
    // Try to retrieve data 3 times with 2 seconds between each retry in case of any error
    retryOptions := temporal.RetryOptions{
        InitialInterval:        time.Second * 2,
        BackoffCoefficient:     2.0,
        MaximumAttempts:        3,
        DoNotRetry:             []error{ErrNonRetryable},
        NonRetryableErrorTypes: []reflect.Type{reflect.TypeOf(ErrNonRetryable)},
    }

    data, err := RetrieveDataFromDatabase(ctx, input)
    if err != nil {
        return nil, temporal.NewContinueAsNewError(ctx, ExampleActivity, input)
    }
    return data, nil
}

// database.go
func RetrieveDataFromDatabase(ctx context.Context, key string) ([]string, error) {
    // Use the repository to retrieve data from the database
    repo := NewExampleRepository()

    data, err := repo.GetDataFromDatabaseByKey(ctx, key)
    if err != nil {
        return nil, err
    }

    return data, nil
}

These functions demonstrate how the layered architecture works with Temporal.io engine to manage workflows and activities. The ExampleActivity function retries the RetrieveDataFromDatabase function in case of any error with backoff exponential, and pass the retrieved data through service layer to do business logic. When business logic is executed without any error, the ExampleWorkflow function uses the Temporal.io engine to set the result of the workflow.

By structuring the application in this way, we can keep the data access and business logic layers decoupled, making it easier to maintain, manage and test the application.