I recently worked on a project where we wanted to port an existing API to Azure. Problem was all the code was written in Go and Azure is not known for its Go support. Enter Azure Functions custom handlers which allows you to bring your own HTTP server to receive and respond to Azure Functions requests. I was impressed by this preview feature and how easily it enabled us to bring our existing code to Azure. In this post, I’m going to walk through creating some simple Go based Azure Functions to demonstrate how it works.
At a high level, custom handlers enables the Azure Function host to proxy requests to a HTTP server written in any language. The server application must run on the Azure Function host (meaning you may need to cross-compile the application to work on the Function Host’s operating system). The only requirements are the server must be able to receive and return HTTP requests in a specific, Azure Function centric format and the server needs to startup and respond in 60 seconds or less.
What We’re Building
The problem we’re going to solve is the automated creation of developer environments on Azure, called “playgrounds”. The goal is to add some automation around the creation of resource groups our developers need to create and test their applications. The requirements our functions needs to meet are:
Creation of “Playgrounds” (a.k.a. resource groups) by an administrator and assignment of access to a developer
Getting list and details of existing “Playgrounds” to help with management
Deletion of “Playgrounds” once the developer’s work has completed
We also have some requirements for developer experience:
Short inner loop for local development (more info)
Automated deployment of infrastructure and code checked into the master branch
The Azure Functions team has provided a set of basic samples on Github that walk through some basic input and output binding scenarios. The official documentation is also quite good in describing the request and response payload required for writing a custom handler. Let’s start with a basic, modified version of the samples repository to see if we can get a simple HTTP function working.
Let’s start with an empty directory and copy over the following files from the samples repository linked above:
We’ll need to make some changes in these files to get our basic sample working. Below is the new host.json code. The executable name needs to change for the Go HTTP server we’ll compile soon. Also, set enableForwardingHttpRequest to false so we’re able to handle other output bindings in the future (like blobs and queues).
We’ll need to pair down and modify our Go code quite a bit to isolate our sample and properly instruct our output HTTP binding to return the correct headers, status code, and body.
And finally, we’ll need to setup our function bindings. Note the modification of $return to a http output - this is required to properly set the res output (but may be a bug that will be fixed).
Now we just need to compile our Go HTTP server and run our function.
1# Compile the server2go build -o azure-playground-generator
3chmod +x azure-playground-generator
45# Start Azure Function locally6func host start
This will expose http://localhost:7071/api/HttpTriggerWithOutputs which we can test to ensure we are receiving the body, content type, and response code we specified.
Before adding any more functionality to our code, let’s spend some time maturing the project structure. I’ll be leaning heavily on the project layout pattern as defined in this repository, especially since I’m a Golang newcomer. If you are creating your own Go on Azure project, I recommend reviewing this repository for pointers. Now, we’re going to start with the below project structure. The goal is to split out separate responsibilities into separate packages of our project (config, API helpers, standard errors, and a shell for the playground functionality) and isolate the Azure Function specific configuration. We’re also adding some automation around building and running the code to reduce our inner loop.
First, create a functions folder at the root of the repository and copy the HttpTriggerWithOutputs folder and the host.json file into this folder. While this doesn’t follow the guide mentioned above, these files are special enough to warrant their own directory. No changes are required for these files.
Next, create the config.go and env.go files inside of internal/config. We only have one value for configuration at the moment (the Go server port), but this isolation is useful when adding more configuration. This setup will allow setting our configuration by passing environment variables.
internal/config/config.go
1
2
3
4
5
6
7
8
9
10
11
12
13
14
// Package config manages loading configuration from environment
packageconfig// Source:
// https://github.com/Azure-Samples/azure-sdk-for-go-samples/blob/3d51ac8a1a5097b8881a8cf29888d4a44f7205f5/internal/config/config.go
var (
functionHTTPWorkerPortint = 8082)
// FunctionHTTPWorkerPort is the port the go server will run on and Azure Function will send requests to
funcFunctionHTTPWorkerPort() int {
returnfunctionHTTPWorkerPort}
internal/config/env.go
1
2
3
4
5
6
7
8
9
10
11
12
13
14
// Package config manages loading configuration from environment
packageconfig// Source:
// https://github.com/Azure-Samples/azure-sdk-for-go-samples/blob/3d51ac8a1a5097b8881a8cf29888d4a44f7205f5/internal/config/config.go
var (
functionHTTPWorkerPortint = 8082)
// FunctionHTTPWorkerPort is the port the go server will run on and Azure Function will send requests to
funcFunctionHTTPWorkerPort() int {
returnfunctionHTTPWorkerPort}
Next, let’s isolate API handling specific code into pkg/api. Create two files here: request.go for helping handle requests, and response.go for help with building response objects. As we add more Azure Functions, isolating this code will be useful to standardize how we handle requests and responses.
packageapiimport (
"encoding/json""net/http""azure-playground-generator/pkg/errors""azure-playground-generator/pkg/playground")
// InvokeRequest is a struct that represents an Azure Function input
typeInvokeRequeststruct {
Datamap[string]interface{}
Metadatamap[string]interface{}
}
// InvokeHTTPRequest is a struct that represents an Azure Function input with HTTP input
typeInvokeHTTPRequeststruct {
Datamap[string]HTTPInputMetadatamap[string]interface{}
}
// HTTPInput is a struct that represents the data of a HTTP binding for Azure Functions
typeHTTPInputstruct {
BodystringHeadersmap[string][]string//Identities string
MethodstringParamsmap[string]stringQueryinterface{}
URLstring}
funcrequestDecoder(r*http.Request) (*HTTPInput, error) {
// Decode Azure Function Request
varinvokeReqInvokeHTTPRequestd:=json.NewDecoder(r.Body)
decodeErr:=d.Decode(&invokeReq)
ifdecodeErr!=nil {
returnnil, errors.NewBadRequest("Invalid format from function")
}
// Pull out request
req, ok:=invokeReq.Data["req"]
if !ok {
returnnil, errors.NewBadRequest("Function input req not found or improperly constructed")
}
return&req, nil}
// HTTPTriggerHandler returns test data from server request
funcHTTPTriggerHandler(whttp.ResponseWriter, r*http.Request) {
// Decode request object
req, err:=requestDecoder(r)
iferr!=nil {
WriteHTTPErrorResponse(w, err)
return }
// Get data from playground package
test, err:=playground.Test(r.Context(), req.Body)
iferr!=nil {
WriteHTTPErrorResponse(w, err)
return }
WriteHTTPResponse(w, http.StatusOK, test)
}
packageapiimport (
"encoding/json""net/http""azure-playground-generator/pkg/errors")
// Source:
// https://github.com/Azure-Samples/functions-custom-handlers/blob/3bd2a534130992af6f8af6608a3dc6007fd31161/go/GoCustomHandlers.go
// https://github.com/Optum/dce/blob/790404bd0b336994d627add7d3a176d90d4d6156/pkg/api/error.go
// InvokeResponse is an Azure Function construct showing how the function expects a return from the Go server
typeInvokeResponsestruct {
Outputsmap[string]interface{}
Logs []stringReturnValueinterface{}
}
// HTTPBindingResponse is a output of InvokeResponse used to pass data back to a HTTP Output for an Azure Function
typeHTTPBindingResponsestruct {
statusCodeintbodyinterface{}
headersmap[string]string}
// WriteHTTPResponse takes data and writes it in a standard way to a http.ResponseWriter
funcWriteHTTPResponse(whttp.ResponseWriter, statusint, bodyinterface{}) {
outputs:= make(map[string]interface{})
headers:= make(map[string]string)
headers["System"] = "Azure Playground Generator"headers["Content-Type"] = "application/json"// Serialize body
jsonData, err:=json.Marshal(body)
iferr!=nil {
WriteHTTPErrorResponse(w, errors.NewInternalServer("error serializing body", err))
return }
// Create response object
res:= make(map[string]interface{})
res["statusCode"] = statusres["body"] = string(jsonData)
res["headers"] = headersoutputs["res"] = resinvokeResponse:=InvokeResponse{Outputs: outputs}
// Serialize response object
j, err:=json.Marshal(invokeResponse)
iferr!=nil {
w.WriteHeader(http.StatusInternalServerError)
return }
w.Header().Set("Content-Type", "application/json")
w.Write(j)
}
// WriteHTTPErrorResponse takes an error and writes it in a standard way to a http.ResponseWriter
funcWriteHTTPErrorResponse(whttp.ResponseWriter, errerror) {
// If custom error object
switcht:=err.(type) {
caseerrors.HTTPCode:
WriteHTTPResponse(w, t.HTTPCode(), err)
return }
// If standard error
WriteHTTPResponse(
w,
http.StatusInternalServerError,
errors.NewInternalServer("unknown error", err),
)
}
Now, lets add a couple standard errors to make creating errors in our application simpler. Create pkg/errors/error.go and add the below code.
packageerrorsimport"net/http"// Source:
//https://github.com/Optum/dce/blob/82521f9b906194df4b69ea8e852d9b3f763e4c89/pkg/errors/error.go
// StatusError is the custom error type we are using.
// Should satisfy errors interface
typeStatusErrorstruct {
httpCodeintcauseerrormessagestring}
// Error allows conversion to standard error object
func (se*StatusError) Error() string {
returnse.message}
// HTTPCode returns the http code
func (seStatusError) HTTPCode() int { returnse.httpCode }
// HTTPCode returns the API Code
typeHTTPCodeinterface {
HTTPCode() int}
// NewBadRequest returns a new error representing a bad request
funcNewBadRequest(mstring) *StatusError {
return&StatusError{
httpCode: http.StatusBadRequest,
cause: nil,
message: m,
}
}
// NewInternalServer returns an error for Internal Server Errors
funcNewInternalServer(mstring, errerror) *StatusError {
return&StatusError{
httpCode: http.StatusInternalServerError,
cause: err,
message: m,
}
}
For the final package in our project, let’s stub out the playground package and add a simple test method which will return some test data. We’re finally making use of the defined POST method of our function - here we will simply return the data sent in the request body to verify we are correctly parsing the object from Azure Functions.
pkg/playground/test.go
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
packageplaygroundimport (
"context")
// Test returns a Test string
funcTest(ctxcontext.Context, reqBodyinterface{}) (interface{}, error) {
// Build Response
resp:= make(map[string]interface{})
resp["requestBody"] = reqBodyresp["value"] = "test return value"returnresp, nil}
To wire all of this together, we need to create two files at the root of the project: main.go (which replaces GoCustomHandlers.go) and go.mod which will allow our project to reference its internal packages.
Let’s also improve our development inner loop (the time it takes to build and run our application) to make it easier to test our changes. Create build.sh and run-function.sh under the scripts directory. Once you have done that, create a file called Makefile in the root of the repository that we’ll use for simple build and run commands.
scripts/build.sh
1
2
3
4
5
6
7
8
9
10
11
12
#!/bin/bash
set -eou pipefail
SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}")" >/dev/null 2>&1&& pwd )"# Build go codecd ${SCRIPT_DIR}/../
go build -o functions/azure-playground-generator
chmod +x functions/azure-playground-generator
echo "Build complete!"
scripts/run-function.sh
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
#!/bin/bash
set -eou pipefail
SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}")" >/dev/null 2>&1&& pwd )"cd $SCRIPT_DIR/../
# First ensure you have a proper .env file (in the same format as .env.sample) with the deployment values filled out# Load settings from .env fileif[[ -f ".env"]]; then export $(grep -v '^#' .env | xargs)ficd $SCRIPT_DIR/../functions/
func host start
Makefile
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
definePROJECT_HELP_MSGUsage: make help show this message
make build build the golang components of the function make run run the solution locally
endefexport PROJECT_HELP_MSG
help: @echo "$$PROJECT_HELP_MSG" | less
build: bash ./scripts/build.sh
run: bash ./scripts/build.sh
bash ./scripts/run-function.sh
Now let’s test! Fire up the function host with the make run command from the root of the repository. This will use the Makefile we just created to build our Go HTTP server and then launch the Azure Functions host for local execution. The function now will handle both GET requests and POST requests with a payload.
The goal of this sample is some light automation around the management of Azure resource groups. In this section, we’ll add code to interact with the Azure Management APIs to list, create, get, and delete resource groups, which maps to Playgrounds in our application. We’ll be adding actual functionality to our API to accomplish our basic use case to create “Playgrounds” or resource groups for developers to test out their applications.
Authenticating to Azure
Since the code will be accessing Azure and creating resource groups on the user’s behalf, we need to create an Azure service principal that can create resource groups on our behalf. Follow the instructions here to create a service principal with the Azure CLI and then make sure to give it “Contributor” rights to it can create the resource groups and “User Access Manager” rights to it can assign users to resource groups. Create a file named “.env” at the root of the repository in the following format which our Go application will load.
.env
1
2
3
4
5
6
# Needed for running Go sampleAZURE_SUBSCRIPTION_ID=00000000-0000-0000-0000-000000000000
AZURE_TENANT_ID=00000000-0000-0000-0000-000000000000
AZURE_CLIENT_ID=00000000-0000-0000-0000-000000000000
AZURE_CLIENT_SECRET=00000000-0000-0000-0000-000000000000
FUNCTIONS_HTTPWORKER_PORT=8082
Now let’s change our configuration loaded to pull these values from our application. I’ll be borrowing heavily from the Azure SDK for Go (link here and here). We need to add the values required by the SDK to access Azure to config.go and the appropriate loader to env.go.
packageconfig// Source:
// https://github.com/Azure-Samples/azure-sdk-for-go-samples/blob/3d51ac8a1a5097b8881a8cf29888d4a44f7205f5/internal/config/config.go
var (
functionHTTPWorkerPortint = 8082subscriptionIDstringuserAgentstring)
// FunctionHTTPWorkerPort is the port the go server will run on and Azure Function will send requests to
funcFunctionHTTPWorkerPort() int {
returnfunctionHTTPWorkerPort}
// SubscriptionID is a target subscription for Azure resources.
funcSubscriptionID() string {
returnsubscriptionID}
// UserAgent specifies a string to append to the agent identifier.
funcUserAgent() string {
if len(userAgent) > 0 {
returnuserAgent }
return"playground-manager"}
varerrerror// these must be provided by environment
functionHTTPWorkerPortString, err:=envy.MustGet("FUNCTIONS_HTTPWORKER_PORT")
iferr!=nil {
returnfmt.Errorf("Expected env vars not provided: %s", err)
}
functionHTTPWorkerPort, err = strconv.Atoi(functionHTTPWorkerPortString)
iferr!=nil {
returnfmt.Errorf("Expected env vars must be an integer: %s", err)
}
subscriptionID, err = envy.MustGet("AZURE_SUBSCRIPTION_ID")
iferr!=nil {
returnfmt.Errorf("Expected env vars not provided: %s", err)
}
returnnil}
Accessing Azure with the Go SDK
Now that we can authenticate to Azure, let’s write the code to actually access Azure and translate from the Azure concept of resource groups to our application’s concept of Playgrounds. We’re going to focus on adding the list, get, create, and delete operations for Playgrounds in this example. But first, let’s add some common code to get started.
First, let’s add a model for our “Playground” object. This is almost a direct map from the Azure resource group object, except we have an owner field.
packageplaygroundimport (
"github.com/Azure/azure-sdk-for-go/profiles/latest/resources/mgmt/resources")
// Playground is a resource group for our system
typePlaygroundstruct {
ID*string`json:"id,omitempty"`Name*string`json:"name,omitempty"`Location*string`json:"location,omitempty"`OwnerID*string`json:"ownerId,omitempty"`Tagsmap[string]*string`json:"tags"`}
funcgroupToPlayground(gresources.Group) Playground {
varpPlaygroundp.ID = g.IDp.Name = g.Namep.Location = g.Locationp.Tags = g.Tagsifval, ok:=g.Tags["OwnerId"]; ok {
p.OwnerID = val }
returnp}
The common.go file is meant for code that may be used across the package. Here we have the code for the necessary Azure Go SDK clients.
message: m,
}
}
// NewAlreadyExists returns a new error representing an already exists error
funcNewAlreadyExists(namestring) *StatusError {
return&StatusError{
httpCode: http.StatusConflict,
cause: nil,
message: fmt.Sprintf("%s already exists", name),
}
}
// NewNotFound returns an a NotFound error with standard messaging
funcNewNotFound(namestring) *StatusError {
return&StatusError{
httpCode: http.StatusNotFound,
cause: nil,
message: fmt.Sprintf("%s not found", name),
}
}
Now let’s start with our CRUD like operations. This code isn’t complete - there are edge cases that were not considered in the spirit of this sample. First we’ll retrieve a list of Playgrounds, converted from Azure resource groups. We can identify which resource groups mag to Playgrounds via the System tag.
packageplaygroundimport"context"// ListPlaygrounds returns a list of all active playgrounds in subscription
funcListPlaygrounds(ctxcontext.Context) ([]Playground, error) {
groupsClient, err:=getResourceGroupsClient()
iferr!=nil {
returnnil, err }
// Get resource groups created by our Playground System
groups, err:=groupsClient.ListComplete(ctx, "tagName eq 'System' and tagValue eq 'Playground'", nil)
iferr!=nil {
returnnil, err }
playgrounds:= make([]Playground, 0)
for_, v:=range*groups.Response().Value {
playgrounds = append(playgrounds, groupToPlayground(v))
}
returnplaygrounds, nil}
Next, to get a specific Playground, we look for a resource group with the same name. We also will throw one of our custom errors if the resource group is not found.
packageplaygroundimport (
"azure-playground-generator/pkg/errors""context""net/http")
// GetPlayground returns a playground with a given name
funcGetPlayground(ctxcontext.Context, namestring) (*Playground, error) {
groupsClient, err:=getResourceGroupsClient()
iferr!=nil {
returnnil, err }
// See if playground exists (throw error if not)
existResp, err:=groupsClient.CheckExistence(ctx, name)
iferr!=nil {
returnnil, err }
ifexistResp.StatusCode==http.StatusNotFound {
returnnil, errors.NewNotFound(name)
}
group, err:=groupsClient.Get(ctx, name)
iferr!=nil {
returnnil, err }
playground:=groupToPlayground(group)
return&playground, nil}
Creating a Playground is our most complex function. Here we need to create a resource group and assign the correct permissions to the owner provided. We also check and see if the Playground already exists and return a custom error if so.
packageplaygroundimport (
"azure-playground-generator/pkg/errors""context""net/http")
// DeletePlayground deletes a playground with a given name
funcDeletePlayground(ctxcontext.Context, namestring) (interface{}, error) {
groupsClient, err:=getResourceGroupsClient()
iferr!=nil {
returnnil, err }
// See if playground exists (throw error if not)
existResp, err:=groupsClient.CheckExistence(ctx, name)
iferr!=nil {
returnnil, err }
ifexistResp.StatusCode==http.StatusNotFound {
returnnil, errors.NewNotFound(name)
}
response, err:=groupsClient.Delete(ctx, name)
iferr!=nil {
returnnil, err }
returnresponse, nil}
Wiring up the HTTP server
We are making progress! Now we need to enable our Playground package to be called from a HTTP request. I’ve decided to do all of this in the request.go file. As you
Disclaimer - While I work at Microsoft, I do not work on Azure Functions or with the Azure Functions team. This post is a analysis of my work with the technology as an end-user. Microsoft and Azure are registered trademarks ot Microsoft Corporation