Deploying AWS Lambda Functions On GKE using Knative and Triggermesh KLR

Back in Nov 2014, AWS has started the serverless revolution by introducing AWS Lambda and since then, the serverless story has grown from strength to strength with Azure and Google Cloud joining the party with their own alternatives — Azure Functions and Google Cloud Functions. Going one step further — Google gave serverless story a shot in the arm by introducing Knative, bringing serverless goodness to Kubernetes.

We recently had to help one of our customers migrate from an AWS — one of their applications was a collection of 100+ AWS lambda functions behind an API gateway with CloudSQL / DynamoDB backend. We explored ways to port these services to GCP and initially explored Cloud Functions, but most of these lambda functions were on Node 12.x runtime on AWS which is still not supported on Cloud Functions(As of this writing, Node 10 is GA on Cloud Functions) .

The next natural choice was to run your own Functions As a Service on GKE using Knative. We ran into Triggermesh Knative Lambda runtime (Triggermesh KLR) which allows you to deploy Lambda functions to Knative with minimal changes. This is still new, may not be fully production ready — but this post shows how to deploy a sample Lambda function on GKE + Knative with Triggermesh CLR.

Setup GKE cluster Infra with Istio

1. Create a GKE cluster and wait for nodes becomes healthy

gcloud beta container clusters create knative --zone us-central1-c --machine-type=n1-standard-4 --release-channel regular

2. Grant cluster-admin permissions to the current user

kubectl create clusterrolebinding cluster-admin-binding
\--clusterrole=cluster-admin
\--user=$(gcloud config get-value core/account)

3. Enter the following commands to download Istio and install the Istio CRDs

# Download and unpack Istio
export ISTIO_VERSION=1.4.6
curl -L https://git.io/getLatestIstio | sh -
cd istio-${ISTIO_VERSION}
for i in install/kubernetes/helm/istio-init/files/crd*yaml; do kubectl apply -f $i; done

4. Create istio-system namespace

cat <
apiVersion: v1
kind: Namespace
metadata:
name: istio-system
labels:
istio-injection: disabled
EOF

5. Installing Istio without sidecar injection(Recommended default installation)

# A lighter template, with just pilot/gateway.
# Based on install/kubernetes/helm/istio/values-istio-minimal.yaml
helm template --namespace=istio-system \
--set prometheus.enabled=false \
--set mixer.enabled=false \
--set mixer.policy.enabled=false \
--set mixer.telemetry.enabled=false \
`# Pilot doesn't need a sidecar.` \
--set pilot.sidecar=false \
--set pilot.resources.requests.memory=128Mi \
`# Disable galley (and things requiring galley).` \
--set galley.enabled=false \
--set global.useMCP=false \
`# Disable security / policy.` \
--set security.enabled=false \
--set global.disablePolicyChecks=true \
`# Disable sidecar injection.` \
--set sidecarInjectorWebhook.enabled=false \
--set global.proxy.autoInject=disabled \
--set global.omitSidecarInjectorConfigMap=true \
--set gateways.istio-ingressgateway.autoscaleMin=1 \
--set gateways.istio-ingressgateway.autoscaleMax=2 \
`# Set pilot trace sampling to 100%` \
--set pilot.traceSampling=100 \
--set global.mtls.auto=false \
install/kubernetes/helm/istio \
> ./istio-lean.yaml

kubectl apply -f istio-lean.yaml

Install knative Components:

The following commands install all available Knative components as well as the standard set of observability plugins

1. To install Knative, first install the CRDs by running the kubectl apply command once with the -l knative.dev/crd-install=true flag

kubectl apply --selector knative.dev/crd-install=true \
--filename https://github.com/knative/serving/releases/download/v0.12.0/serving.yaml \
--filename https://github.com/knative/eventing/releases/download/v0.12.0/eventing.yaml \
--filename https://github.com/knative/serving/releases/download/v0.12.0/monitoring.yaml

2. To complete the install of Knative and its dependencies, run the kubectl apply command again, this time without the –selector flag, to complete the install of Knative and its dependencies

kubectl apply --filename https://github.com/knative/serving/releases/download/v0.12.0/serving.yaml \
--filename https://github.com/knative/eventing/releases/download/v0.12.0/eventing.yaml \
--filename https://github.com/knative/serving/releases/download/v0.12.0/monitoring.yaml
3. Verify the Knative components until all of the components show a STATUS of Running
kubectl get pods --namespace knative-serving
kubectl get pods --namespace knative-eventing
kubectl get pods --namespace knative-monitoring

Install Tekton Pipelines:

1. Run the following command to install Tekton Pipelines and its dependencies

kubectl apply --filename https://storage.googleapis.com/tekton-releases/pipeline/latest/release.yaml
kubectl get pods --namespace tekton-pipelines --watch

Configuring Istio DNS:

Note down the Istio-ingress gateway Static IP and edit the Configmap config-domain to use the wildcard domain (*.xip.io)

kubectl edit cm config-domain --namespace knative-serving

Given the external IP above, change the content to:

apiVersion: v1
kind: ConfigMap
metadata:
name: config-domain
namespace: knative-serving
data:
# xip.io is a "magic" DNS provider, which resolves all DNS lookups for:
# *.{ip}.xip.io to {ip}.
34.72.127.100.xip.io: ""

Install and configure Triggermesh

Install trigger mesh binary in your local machine using the below the commands

wget https://github.com/triggermesh/tm/releases/download/v0.2.1/tm-linux-amd64
chmod +x tm-linux-amd64
mv tm-linux-amd64 tm
mv tm /usr/local/bin/

Configure google container registry in triggermesh to use the custom registry in the deployment. Please edit the bolded values as per your environment.

TOKEN=$(gcloud auth print-access-token)
tm set registry-auth gcr --registry eu.gcr.io --project Project_ID/knative-test --username oauth2accesstoken --password $TOKEN

Generate sample python function

To verify tm installation will deploy sample python function using tm generate command

tm generate python
tm deploy -f python --registry-secret gcr --wait

Deploying Sample Lambda on Node runtime

1. Run the following command to install node 4.3 run time.

tm deploy task -f https://raw.githubusercontent.com/triggermesh/knative-lambda-runtime/master/node-4.x/runtime.yaml

2. Deploy example nodejs function using the following command where will include the runtime and registry name

tm deploy service node4-test -f https://github.com/serverless/examples \
--build-template knative-node4-runtime \
--build-argument DIRECTORY=aws-node-serve-dynamic-html-via-http-endpoint \
--build-argument HANDLER=handler.landingPage \
--registry-secret gcr \
--wait

3. Once deployment completed will get the HTTP endpoint.

4. When verifying the container registry after the deployment. All the deployment images pushed to GCR.

Serverless containers:

When you hit the URL you will notice the pod is in running state after some time it will terminate automatically. Once again if you hit the URL it will again create pod and terminate.

Congratulations, you have deployed an AWS Lambda-compatible function on Knative within the GKE cluster!!!
Hope you found this useful! Happy serverless hosting! 🙂

Reference Links:

[1]: https://knative.dev/v0.12-docs/install/knative-with-gke/
[2]: https://knative.dev/v0.12-docs/install/installing-istio/
[3]:https://github.com/tektoncd/pipeline/blob/master/docs/install.md#before-you-begin
[4]: https://github.com/triggermesh/tm
[5]:https://github.com/triggermesh/knative-lambda-runtime

This article originally appeared in Medium and is written by Suganya – Cloud Architect.