NGS Day 0

Congratulations on the start of your next-generation connective fabric journey! This guide offers a brief sampling of technologies and solutions now available to you with NGS account in-hand.

In this guide we use the flexible CLI tool for demonstrating service communication, but your real applications will use the rich client libraries available for Go, Rust, Java, .NET, Python, JavaScript, C, and many other languages. The CLI has no "special powers" -- it uses the Go client library!

This guide takes approximately 5 minutes.

After you complete this "Day 0" guide, you may be interested in The Battery Spa end-to-end scenario on "Day 1".

Guide Pre-requisites

Multi-Geo, Multi-Cloud, and Edge Pub-Sub

Send a message to any NGS-connected subscriber who has expressed interest in "foo" subject:

nats pub foo 'Hello, World!'
17:52:59 Published 13 bytes to "foo"

Why is this cool?

An app publishes messages to purely functional addresses (subject) without physical concern, pre-knowledge, or configuration headache of delivery URLs, DNS resolvers, geographic locations, or traffic routing.

Any app that has current interest in a subject (subscribed) will be sent the message asyncronously. NGS transparently takes care of routing, access control, and more; all at extremely high speed and low latency.

Only subscribers that are permitted within your account and any external accounts that have accepted your share offer (export defined by subject) is eligible to receive your published messages.

Multi-Geo, Multi-Cloud, and Edge Services

Have a "conversation" with a service
nats req ngs.echo 'You out there NGS Echo service?'
17:55:30 Sending request on "ngs.echo"
17:55:30 Received with rtt 42.685134ms
[Oregon, US]: "You out there NGS Echo service?"

Load-balanced service ngs.echo lives in every geo region of NGS and can be invoked by any NGS account. NGS will route your "echo" request to the instance nearest your NGS connection.

Why is this cool?

Asynchronous API! An app sends requests to purely functional endpoints (subject) and receives replies over a dynamic reply subject. Unlike HTTP-based APIs, the service conversation is non-blocking and sessionless which enables great speed and scale; replies can be singular or a stream of reply messages!

Have a scatter-gather service conversation
nats req ngs.echo.status --replies=5 'All NGS Echo Status services report!'

Service ngs.echo.status lives in every geo region of NGS. NGS will route your "echo status" request to every instance.

Make a service available

Providing a service on NGS is as easy as subscribing your service app to a subject -- an endpoint -- and responding to request messages that are sent.

nats reply my.echo --echo --queue echopool
17:57:32 Listening on "my.echo" in group "echopool"

Run as many instances of my.echo as you like from anywhere in the world. NGS will pick one instance of "echopool" for each client request.

Field a time service such that a caller will get a localized weather report from every instance on NGS.

nats reply --command 'curl -s'
17:59:19 Listening on "" in group "NATS-RPLY-22"

Here's an example request for (in another terminal):

nats req --replies=3 'Give me local weather please!'
18:02:51 Sending request on ""
18:02:52 Received with rtt 1.208008744s
University Place, Washington, United States: ⛅️  +58°F

Why is this cool?

We can easily field global services deployed as a group member of a load-balanced pool or working independently. The endpoint of our service is simply the subscribed subject. NGS will immediately route requests on that subject to your service -- no additional configuration required!

Reliable messaging: Instant Streams, Queues, and Topics

NGS provides real-time communication with At-Most-Once quality of service as in the Pub-Sub and Service examples.

For At-Least-Once and even Once-and-Only-Once messaging, simply employ JetStream between publishers and subscribers on a specific subject (or set of subjects). JetStream ingests published messages and stores them for later read by subscribers -- an extremely flexible, durable communication buffer!

JetStreams can be placed in any Geo location or Cloud Provider of NGS (or near you by default), and has sophisticated message, growth, lifecycle, and availability policies.

Let's create a JetStream in the western region of the US that captures any message published that matches the some.event.> subject hierarchy, keeping a rolling max of 10 MiB or 1,000 messages with lifetime of up to 7 days. Messages will be limited to 128 KiB in size and any duplicates within 2 minute window ignored.

nats stream add SOME-EVENTS \
   --description 'Capture some events in an NGS stream' \
   --tags 'geo:us-west' \
   --subjects 'some.event.>' \
   --retention limits --discard old --dupe-window 2m \
   --max-age 7d --max-bytes 10485760 --max-msgs 1000 --max-msgs-per-subject=-1 --max-msg-size 131072 \
   --replicas 1 --storage file \
   --no-deny-delete --no-deny-purge --no-allow-rollup

By specifying different retention policies, JetStream provides Stream, Queue, or Topic styles of consumption:

Retention PolicyPatternDescription
LimitsStreamN consumers can read (and re-read!) the stream independently
WorkQueueQueueQueue - Message consumption deletes messages
InterestTopic (Multi-Queue)When all N consumers have read, messages are deleted

Here's the output of nats stream add:

Stream SOME-EVENTS was created

Information for Stream SOME-EVENTS created 2022-04-17T18:07:06-07:00


          Description: Capture some events in an NGS stream
             Subjects: some.event.>
     Acknowledgements: true
            Retention: File - Limits
             Replicas: 1
       Discard Policy: Old
     Duplicate Window: 2m0s
    Allows Msg Delete: true
         Allows Purge: true
       Allows Rollups: false
     Maximum Messages: 1,000
        Maximum Bytes: 10 MiB
          Maximum Age: 7d0h0m0s
 Maximum Message Size: 128 KiB
    Maximum Consumers: unlimited
       Placement Tags: geo:us-west

Cluster Information:

                 Name: ngstest-az-uswest2
               Leader: az-uswest2-natsjs-1


             Messages: 0
                Bytes: 0 B
             FirstSeq: 0
              LastSeq: 0
     Active Consumers: 0

Why is this cool?

With JetStream we temporally decouple publishers from subscribers enabling durable subscription and rich consumption patterns. With JetStream, a published message's journey to one or many subscribing apps can now occur in milliseconds or months.

You choose how a JetStream operates -- as a continuous and replayable stream, as a traditional queue, or as a multi- consumer topic!

Materialized Views: JetStream-enabled Key-Value and Object Stores!

Streams, Queues, and Topics are all message-passing patterns, but JetStream also "materializes" other access patterns including Key-Value store and Object store solutions!

Key-Value Store

Each KV store in NGS is a JetStream with each unique subject materializing a key.

KV stores access is by familiar key-value idioms such as put, get, and history. Watches can be placed -- at store level or key level -- so apps can act on change events!

Create a KV store to hold the last 10 values of key1:

nats kv add KV-STORE \
   --description 'KV store for key values' \
   --replicas 1 --storage file --history 10 --max-bucket-size 10485760 --max-value-size 131072
Information for Key-Value Store Bucket KV-STORE created 2022-04-17T18:19:05-07:00


         Bucket Name: KV-STORE
        History Kept: 10
       Values Stored: 0
  Backing Store Kind: JetStream
         Description: KV store for key values
 Maximum Bucket Size: 10485760
  Maximum Value Size: 131072
    JetStream Stream: KV_KV-STORE
             Storage: File

Cluster Information:

                Name: ngstest-az-uswest2
              Leader: az-uswest2-natsjs-4

Put three values for key1 into KV-STORE:

nats kv put KV-STORE 'key1' \
   '{ "alpha": "23456", "beta": "1648583305" }'
nats kv put KV-STORE 'key1' \
   '{ "alpha": "13890", "beta": "1648647305" }'
nats kv put KV-STORE 'key1' \
   '{ "alpha": "23890", "beta": "1648594301" }'

The latest value of key1:

nats kv get KV-STORE 'key1'
KV-STORE > key1 created @ 18 Apr 22 01:20 UTC

{ "alpha": "23890", "beta": "1648594301" }

A history of key1:

nats kv history KV-STORE 'key1'
│                               History for KV-STORE > key1                                │
│ Key  │ Revision │ Op  │ Created             │ Length │ Value                             │
│ key1 │ 1        │ PUT │ 17 Apr 22 18:20 PDT │ 42     │ { "alpha": "234... "1648583305" } │
│ key1 │ 2        │ PUT │ 17 Apr 22 18:20 PDT │ 42     │ { "alpha": "138... "1648647305" } │
│ key1 │ 3        │ PUT │ 17 Apr 22 18:20 PDT │ 42     │ { "alpha": "238... "1648594301" } │

Watch the KV-STORE for any key updates:

nats kv watch KV-STORE

Watch the KV-STORE for any updates to key1:

nats kv watch KV-STORE 'key1'

Why is this cool?

Instant Key-Value stores that you can place anywhere in the world, accessible from any application you connect to NGS. JetStream provides easy to use Key-Value access methods as an abstraction over message stream.

Object Store

Each Object store is a JetStream with ordered messages of a subject (taken together) materializing an object (or blob). Object stores directly leverage blob-storage idioms like add, get, and ls. Watches can be placed on an Object store so clients can be evented any object adds or deletes!

Grab a file from the Synadia web site (or provide a file of your own):

curl -s > /tmp/ad1234.jpg

Create an Object store OBJ-STORE to hold your files:

nats object add OBJ-STORE \
   --description 'Object store for my files' \
   --max-bucket-size 10485760
Information for Object Store Bucket OBJ-STORE created 2022-04-17T18:28:47-07:00


         Bucket Name: OBJ-STORE
         Description: Object store for my files
            Replicas: 1
                 TTL: unlimited
              Sealed: false
                Size: 0 B
 Maximum Bucket Size: 10 MiB
  Backing Store Kind: JetStream
    JetStream Stream: OBJ_OBJ-STORE

Cluster Information:

                Name: ngstest-az-uswest2
              Leader: az-uswest2-natsjs-3

Upload your file to OBJ-STORE:

nats object put OBJ-STORE '/tmp/ad1234.jpg' --name 'ad1234.jpg'
104 KiB / 104 KiB [========================================================================================================]

Object information for OBJ-STORE > ad1234.jpg

               Size: 104 KiB
  Modification Time: 18 Apr 22 01:29 +0000
             Chunks: 1
             Digest: sha-256 a9e3e78fe3c796c3b9fd469230ccb33f124c8a9335d0025d9f941fc5349a

List files in OBJ-STORE:

nats object ls OBJ-STORE
│                 Bucket Contents                  │
│ Name       │ Size    │ Time                      │
│ ad1234.jpg │ 104 KiB │ 2022-04-17T18:29:59-07:00 │

Download your file from OBJ-STORE:

nats object get OBJ-STORE 'ad1234.jpg' --output '/tmp/edgecache-ad1234.jpg'
104 KiB / 104 KiB [========================================================================================================]

Wrote: 104 KiB to /tmp/edgecache-ad1234.jpg in 0.00s

Monitor OBJ-STORE for new uploads or deletes:

nats object watch OBJ-STORE

Why is this cool?

Instant blob storage that you can place anywhere in the world, accessible from any application you connect to NGS! JetStream provides easy-to-use access methods as an Object store abstraction over message stream.