Producing and Consuming Data with NKN

In oder to build event driven applications you need an eventing substrate to connect your event producers and consumers. Traditionally this can be done with Pub/Sub systems common to the enterprise stack like Kafka, Redis, RabbitMQ or other Cloud based solutions like AWS Kinesis. To connect event producers and consumers globally there are also solutions like Synadia NGS or Solace which provide a global secure and scalable network to link all your devices together.

In this blog post we will be looking at an emerging technology a little off the beaten path of the enterprise IT stack called NKN. Specifically, we will be looking at how to leverage the messaging interface that NKN provides to send and receive data using the NKN Golang SDK and therefore ultimately build event-driven applications with Triggermesh sources, targets and transformations.

What is NKN?

Before we begin, lets have a small primer on what NKN is.

The official NKN website describes it as: "... a new kind of peer to peer network connectivity protocol and ecosystem powered by a novel public blockchain. It uses economic incentives to motivate Internet users to share network connections and utilize unused bandwidth."

The platform boasts a ton of features, including:

• Millions of nodes
• Aggregated high speed
• Zero Server
• Unique and global ID
• Extra security and privacy
• Low latency

Leveraging NKN's Messaging Interface

Let's start using NKN step by step, starting from scratch with a consumer in Go.

Creating a Consumer

1. Create a new Go project.

mkdir nknconsumer
cd nknconsumer
touch main.go
go mod init
go mod tidy

‍

2. Import the NKN SDK.

This can be accomplished by adding the following to your main.go file:

package main

import (
    nkn "github.com/nknorg/nkn-sdk-go"
)

3. Create a new function called start . This function will be called when the program is started and will handle all of the logic in our example program.

func start() error {}

4. Begin populating the start function by creating a new account.

‍

account, err := nkn.NewAccount(nil)
if err != nil {
	return err
}

5. Create a new client.

‍
NKN does something interesting here. You can create a new client with the use of either nkn.NewClient() or nkn.NewMulticlient().

The nkn.NewClinet() method creates a single standard client, as one might expect. However, the nkn.NewMulticlient() method creates a client that can connect send/receive packets concurrently. This will greatly increase reliability and reduce latency at the cost of more bandwidth usage (proportional to the number of clients).

For this example, we will be using the nkn.NewMulticlient() method.

toClient, err := nkn.NewMultiClient(account, "", 4, false, nil)
if err != nil {
	return err
}
defer toClient.Close()

‍

note the "" parameter in the NewMultiClient() method. This is called an identifier and is used to distinguish different clients sharing the same key pair. Since we will be using two distinct clients, we do not need to include this in the call.

1. Log the client address we just created.

log.Println("Recieving messages at ", toClient.Address())

2. Create a loop to listen for incoming messages.

<-toClient.OnConnect.C
for {
	msg := <-toClient.OnMessage.C
	isEncryptedStr := "unencrypted"
	if msg.Encrypted {
		isEncryptedStr = "encrypted"
	}
	log.Println("Receive", isEncryptedStr, "message", "\""+string(msg.Data)+"\"", "from", msg.Src)
	// []byte("World") can be replaced with "World" for text payload type
	msg.Reply([]byte("World"))
}

‍

3. Call the start function from the main function.

if err := start(); err != nil {
    log.Fatal(err)
}

‍

Full code review

Your nknconsumer.go file should look something like this:

package main

import (
	"log"

	nkn "github.com/nknorg/nkn-sdk-go"
)

func main() {
	if err := start(); err != nil {
		log.Fatal(err)
	}
}

func start() error {
	account, err := nkn.NewAccount(nil)
	if err != nil {
		return err
	}

	toClient, err := nkn.NewMultiClient(account, "", 4, false, nil)
	if err != nil {
		return err
	}
	defer toClient.Close()

	log.Printf("listening on %s", toClient.Address())
	<-toClient.OnConnect.C
	for {
		msg := <-toClient.OnMessage.C
		isEncryptedStr := "unencrypted"
		if msg.Encrypted {
			isEncryptedStr = "encrypted"
		}
		log.Println("Receive", isEncryptedStr, "message", "\""+string(msg.Data)+"\"", "from", msg.Src)
		// []byte("World") can be replaced with "World" for text payload type
		msg.Reply([]byte("World"))
	}

	return nil
}

Run the program and retrieve your client address.

go mod tidy
go run .

‍

The following shows a sample client address returned at the terminal:

2022/07/26 10:00:05 listening on  8326bf7d27267211f59cf03e52758b1bd0d478a00d86d0cbebefd8a919668665

Now that we have a consumer set up and have its address, we can create a producer and start sending messages!

Creating a Producer

1. Create a new Go project.

mkdir nknproducer
cd nknproducer
touch main.go
go mod init
go mod tidy

‍

2. Import the NKN SDK.

This can be accomplished by adding the following to your main.go file:

package main

import (
    nkn "github.com/nknorg/nkn-sdk-go"
)

‍

3. Create a new function called start. This function will be called when the program is started and will handle all of the logic in our example program.

func start() error {}

4. Begin populating the start function by creating a new variable called ca that will pull from an environment variable "CONSUMER_ADDRESS". This will enable us to change the consumer address in the future, without modifying the code.

ca := os.Getenv("CONSUMER_ADDRESS")

5. Create a new account.

account, err := nkn.NewAccount(nil)
if err != nil {
	return err
}

‍

6. Create a new NKN client from the account and start listening for connections.

fromClient, err := nkn.NewMultiClient(account, "", 4, false, nil)
if err != nil {
	return err
}
defer fromClient.Close()
<-fromClient.OnConnect.C

‍

7. Send a message to the client.

log.Println("Send message from", fromClient.Address(), "to", ca)
// []byte("Hello") can be replaced with "Hello" for text payload type
onReply, err := fromClient.Send(nkn.NewStringArray(ca), []byte("Hello"), nil)
if err != nil {
	return err
}

‍

8. Include some logic to handle the reply

reply := <-onReply.C
isEncryptedStr := "unencrypted"
if reply.Encrypted {
	isEncryptedStr = "encrypted"
}
log.Println("Got", isEncryptedStr, "reply", "\""+string(reply.Data)+"\"", "from", reply.Src)

‍

Full code review

Your nknconsumer.go file should look something like this:

package main

import (
	"fmt"
	"log"
	"os"

	nkn "github.com/nknorg/nkn-sdk-go"
)

func main() {
	err := start()
	if err != nil {
		fmt.Println(err)
	}
}

func start() error {
	ca := os.Getenv("CONSUMER_ADDRESS")

	account, err := nkn.NewAccount(nil)
	if err != nil {
		return err
	}

	fromClient, err := nkn.NewMultiClient(account, "", 4, false, nil)
	if err != nil {
		return err
	}
	defer fromClient.Close()
	<-fromClient.OnConnect.C

	log.Println("Send message from", fromClient.Address(), "to", ca)
	// []byte("Hello") can be replaced with "Hello" for text payload type
	onReply, err := fromClient.Send(nkn.NewStringArray(ca), []byte("Hello"), nil)
	if err != nil {
		return err
	}

	reply := <-onReply.C
	isEncryptedStr := "unencrypted"
	if reply.Encrypted {
		isEncryptedStr = "encrypted"
	}

	log.Println("Got", isEncryptedStr, "reply", "\""+string(reply.Data)+"\"", "from", reply.Src, "after")

	return nil
}

Bring it all together

Now that we have a consumer and a producer set up, we can start to send some messages.

1. Launch the nknconsumer.go program and note down the consumer address.

go run .

go run .

You will see a new consumer address printed out:

2022/07/26 10:46:21 listening on ae1d985e47aaf881e06e86df6aba934f5f6674bc97624435a8df29f911c460e2

2. Open a new terminal and navigate to the nknproducer directory.

3. Export the consumer address to an environment variable.

export CONSUMER_ADDRESS=ae1d985e47aaf881e06e86df6aba934f5f6674bc97624435a8df29f911c460e2

4. Launch the nknproducer.go program.

go run .

Results will be printed out at both of the consumer and producer terminals and will resemble the following

nknproducer.go

2022/08/08 14:38:33 Send message from 882448a740c95f3acb9768b511f305ea1e46be77e06d3b58f06c32f879adbcda to ad00e867bc53701880d7733f39679c2879af2cf283c0ffa0838688b926384826
2022/08/08 14:38:33 Got encrypted reply "World" from ad00e867bc53701880d7733f39679c2879af2cf283c0ffa0838688b926384826 after

nknconsumer.go

2022/08/08 14:38:33 Receive encrypted message "Hello" from 882448a740c95f3acb9768b511f305ea1e46be77e06d3b58f06c32f879adbcda

Conclusions

NKN is a "New Kind of Network" built on a public Blockchain technology, it offers a scalable, secure, low latency peer to peer network with a simple pub/sub programming interface that can be used to build event driven applications.

While tied to a cryptocurrency based economy it can be used by TriggerMesh sources, targets, transformations to build highly distributed event-driven applications for the enterprise and provides an alternative to more main stream pub/sub global substrates.

‍