Fullstack Go: Going Asynchronous
In a previous post, I described how I used Go on Frontend (FE) and Backend (BE) to create a simple Todo application: the FE was implemented using Vugu, the BE was implemented as Lambda functions on AWS — a secure Swagger defined API was used for FE/BE communication. Here, I wanted to consider asynchronous communications between FE and BE and hence I describe how I implemented the asynchronous variant of the Todo application.
AsyncAPI
AsyncAPI is trying to do for asynchronous communications what Swagger/OpenAPI has done for synchronous communications: define a standardized and machine readable description of an asynchronous API and provide language independent tooling around that which makes it easier to work with such APIs.
AsyncAPI is still in quite early stages of evolution and is taking on a very challenging problem — trying to support many different asynchronous communications technologies (eg websockets, Kafka, NATS, Rabbitmq, AWS SQS, etc), as well as navigating a space littered with many base message formats.
While AsyncAPI does have some support for node.js and Java platforms — support for code generation for Go is not yet provided; also, as both my FE and BE were implemented in Go, there was not a strong need for a language independent definition of the message format. For these reasons, I decided not to pursue the use of AyncAPI here, although I do find the initiative interesting.
Home grown Websockets solution
With code generation tools not meeting my requirements, it was clear I needed to roll my own solution. Websockets seemed the most natural choice for this, having support from browsers and the AWS API Gateway. Hence, I decided to focus on a Websockets based asynchronous communications solution.
Websockets and the API Gateway
As Websockets was designed to support an asynchronous communications model, the operation of Websockets in the API Gateway differs from classical HTTP(S) REST APIs. With REST APIs employing a request/response model, triggering a Lambda function on the request and returning the response is natural; websockets, on the other hand, do not necessitate a response — indeed the default behaviour within the API Gateway is to assume that clients do not expect a response.
My rather primitive approach here was to use Websockets in a synchronous manner — although this clearly does not hit the sweet spot for Websockets (there are questions regarding whether this approach is flawed in principle), the point here to was understand and demonstrate use of asynchronous communications in this context, rather than specifically showcasing it as a sensible solution for this sample problem.
For Websockets, the API Gateway performs key based routing, on the assumption that each message contains a specified key. The associated value can then be used to route the message to different Lambda functions as necessary; it is also possible to route disparate key values to the same Lambda function. A default handler can be specified to deal with exceptional cases, eg the message does not contain the key specified or the value associated with the key does not match any defined in the routing table.
As well as the default handler, there are specific hooks for connection establishment and termination: typically, these are used to store/remove connection IDs which are needed when sending messages to specific clients.
To use the API Gateway with a Lambda function in a request/response manner for Websockets, it is necessary to set it up in a specific way:
- the Lambda function should not be called using a Proxy integration (as this cannot be used for two way communications)— the integration mode should be
AWS - the Route must have a defined
RouteResponse - these must be an IntegrationResponse which takes the Lambda function output and provides a response to the client
As the Lambda function is not called using a Proxy, the input it receives is simply the message sent — rather than a fully defined HTTP request — and the Lambda function should be designed accordingly.
Logic to set up the API Gateway using the Serverless Application Model (SAM) is in the github repo linked to below.
Websockets in the Frontend
A couple of issues arose while implementing websockets logic in the FE — none of them very challenging but, nonetheless, worth noting here.
The first gotcha that I encountered was that the standard gorilla/websockets module does not work within a WASM context: this has been flagged and there is an alternative websockets implementation — nhooyr.io/websocket— which does work within a WASM context. Working with this library, it was possible to Dial() a remote service and send and receive messages via a secure websockets connection.
Unlike the request/response modus, it was necessary to have a single websockets connection available to all parts of the application as required. Hence a websockets connection was created at application initialization and made available globally.
Changing the FE write functionality — creating todos, modifying todos — was then straightforward, employing a WriteMessage() with appropriately marshalled data on the socket.
Reading the set of todos at application start time proved a little more tricky: naturally, this information was requested from the backend but two more specific issues arose — how to capture the response from the backend and how to perform a rerendering on the FE with the new information. The former was solved by simply creating a go-routine which constantly listens on the websocket — this function then had to determine the message type and figure out what to do with it. Performing a rerendering on the FE required a deeper understanding of the event handling in vugu — some primitives are provided for exactly this case in which locks can be acquired, data updated and the locks released and the component rerendered.
It is probably worth noting that this remains a primitive websockets implementation which does not specifically track responses to messages sent from the FE and has minimal error handling.
Final remarks
The experience showed that building a Fullstack Go application using Websockets is possible today: however, it did highlight the fact that there are some gaps in multiple aspects (tooling, documentation, support in widely used libraries) which demonstrate that these technologies are not yet mature, at least in the (admittedly niche) Fullstack Go context.
Github repo containing the application
