GSL Fundamentals

Introduction

Greymatter Specification Language, GSL for short, is a declarative domain specific language designed for application networking in a modern mesh-like topology. GSL reduces boilerplate by providing natural object relationships and drop-in customization. GSL empowers users from across the experience spectrum to make complex configuration changes rapidly and with confidence.

The CUE language forms the foundation of GSL. CUE is a data configuration language with roots to both the field of linguistics and to internal Google projects. This gives CUE a unique set of advantages compared to other data configuration languages. Namely, it was birthed from environments steeped in extreme complexity and scale. If you want to read more about the origins or technical details, we recommend the official CUE documentation.

CUE grants GSL powerful features including:

• blazing fast schema validation and versioning

• boilerplate reduction

• improved but familiar JSON-like syntax

• unique configuration philosophies

To clarify, GSL is not a separate language or an offshoot of CUE. Rather, it is a CUE package that describes a complex set of data constraints and relationships which model the greymatter control plane. As a result, all CUE constructs extend into GSL.

This document contains concepts and high-level discussion on GSL, its use in Greymatter, and how it fits into the tenant project. If you are a more hands-on learner or want to get started immediately, we diving in with a staring tutorial.

Basics of GSL

Working with GSL requires a basic understanding of CUE’s syntax and concepts as it is a collection of CUE schemas. We will cover the basics here swiftly, but highly recommend following their tutorial for a deeper understand of CUE and its potential.

CUE and GSL

CUE is a JSON superset—it follows JSON’s basic structure with syntactic improvements including:

• no need to conclude lines with commas

• no need to wrap object keys in quotation marks

• situationally optional curly braces

For example, to define an object (a struct in CUE terminology):

my_object: {
  key1: "value1"
  key2: 2
  key3: true
  key4: [key1, key2, key3]
}

CUE supports all typical data types. It can also create dynamic structures by accessing other values declared in CUE like key4 does.

CUE also supports schemas called definitions that enforce type safety against data.

GSL exports a variety of definitions that you will use to build out application networking configurations. Definitions are denoted by the # prefix:

typed_value: #MyDefinition

This example means that the type_value must conform to whatever rules are defined in the #MyDefinition schema.

You can also enforce a schema against values using the & (unification) operator. This is used in GSL to pass values into certain schemas to override or set their data fields.

typed_value: #Person & { 
  name: "Luke Skywalker"
  age: 19
  address: "Lars Moisture Farm, Tatoonie"
}

The above will take some data (data describing Luke Skywalker) and enforce that it follows the schema #Person and then sets the resulting value to the value, typed_value.

If any key or value in the unified data conflicts with the constraints defined in the definition, CUE will error.

You can also build new definitions and values from other definitions. This is called embedding and forms the basis of the many GSL mix-ins you will use. To embed a definition into another:

typed_value: {
  #Person
  #StarWarsCharacter

  // old fields from #Person
  name: "Luke Skywalker"
  age: 19
  address: "Lars Moisture Farm, Tatoonie"

  //new fields gained from #StarWarsCharacter
  force_sensitive: true
  allegiance: "Rebel Alliance"
}
  

Embedding creates a new type by composing multiple definitions together. It allows for mixing-in new attributes and constraints to base templates. In GSL this feature creates an intuitive “plug and play” type of configuration system.

Project Organization

Before learning about GSL, we will start with understanding how GSL is organized. We also recommend learning about our GitOps process for some basic understanding of a tenant and tenant project.

GSL configurations are organized into projects. A project maps to a single tenant namespace. Each project contains the configurations for all the tenant's services. GSL projects are bootstrapped by the greymatter init command.

// a tree listing of a GSL project

cue.mod/
k8s/
├── manifests.yaml
└── sync.yaml
greymatter/
├── core/
│   └── edge.cue
├── globals.cue
├── application-stack/
└── policies/

The cue.mod directory contains the GSL schemas. You shouldn’t interact with that folder.

The k8s folder contains Kubernetes manifests for the tenant’s default edge proxy and Sync service. It can also store the manifests of the services deployed into the  namespace.

The greymatter folder contains all the GSL configurations for services deployed within the project namespace.

Inside the greymatter folder, there are a few items that get generated. core contains GSL configurations for any Greymatter applications, like the tenant edge proxy. globals.cue contains project-wide settings and values from which other services can import and reference. policies is a folder where custom CUE definitions can reside.

The organization of your application’s GSL files are more loosely based. Typically, each application stack receives a folder. An application stack includes the service and any direct dependencies.  A direct dependency would include databases or separate services that are used only by that service and would never exist independently.

Alternatively, you can bucket your services however makes sense for your deployments. Note, however, one service file should correspond to one service.

Bundled Applications

A freshly initialized project contains two manifests for Greymatter tenant services: one for the Sync service and one for the edge proxy. Each tenant project should contain at least one Sync service and edge proxy. With the exception of version upgrades, image tags, and ancillary Kuberenetes manifests like Service and ConfigMap, you shouldn’t have a need to change these files. You will need to manually apply them—the Sync service does not handle Kubernetes resources.

Sync Deployment

The Sync manifest found in the file sync.yaml launches the CLI in Sync mode. Sync enables the GitOps pipeline for tenants. It watches a repository filled with a GSL projects (or projects in the case of monorepo mode) and applies them to the mesh. The base install requires a nominal amount of setup which you can follow here.

Edge Proxy Deployment

Each tenant should have at least one edge proxy. This edge proxy is responsible for North/South traffic and can help create a wall around the tenant project. By default, we scaffold out a Kubernetes LoadBalancer Service that exposes a port on the Kubernetes and maps it to the edge’s main listener. Ports will naturally be subject to change, so when you make one, ensure that targetPort matches the correct GSL listener. Likewise, if you want to expose multiple ports from the proxy to outside the cluster, you will need to edit the Kubernetes Service manifest and then add a corresponding GSL listener to the edge service file.

Service discovery

Service discovery only occurs on a single port, whichever one is named ingress in the Kubernetes manifest—in this case 10809 but usually 10908. Service discovery should map to the service’s main listener.

When adding multiple edges to a project, you will need to copy the original manifest and update ports and names to prevent conflicts.

The GSL Service File

A GSL service file contains all the configuration for one data plane proxy. This configuration can one of two types:

1. #Edge

2. #Service

These are “top level definitions” because the form the configuration root for a single service.

Top Level Definitions

The #Edge definition declares the file as a Greymatter edge proxy. Only the greymatter application edge node created by the greymatter init command or extra edge nodes you create should implement this definition.

The #Service definition declares the file as a normal service. All the tenant's deployed services will use the service definition. It both asserts a configuration schema and injects configuration automatically to help reduce boilerplate or to simplify complex yet crucial features.

Other than slight internal differences, both the #Service and #Edge definitions are functionally equivalent. Their inner schemas can be considered interchangeable.

To initialize a service, unify the #Service definition with the service data:

my_service: gsl.#Service & {
...
}

gsl prefix

You’ll note that the definitions are prefixed with gsl.. This is because they must be imported. We typically use gsl as the import alias. If you use the init command, that import will be setup automatically.

Application Networking Object Hierarchy

The application networking object hierarchy refers to the collection of configuration objects stacked on top each other to form a complete end-to-end request handling pipeline. Listed from parent to child, they are:

• Listeners

• Routes

• Upstreams

The root of the hierarchy is the #Service or the #Edge definition.

Each parent object has a one-to-many relationship with its children, starting with the #Service definition. In other words, the service can have multiple listeners which can have multiple routes which can have multiple upstreams. By combining all these elements, we can express complex networking principles while enforcing human-readable models.

The next sections provide an overview of each type of object and a simplified example. For guidance on how to actually construct each object, either follow the getting started with GSL tutorial or refer to each object's reference page linked in the section.

Listeners

Listeners instruct the proxy to listen for connections on a specific network interface and port. There are two types of network interfaces: ingress and egress. Ingress listeners only accept requests originating from outside the local machine and egress listeners only accept requests originating from inside the local machine.

Regardless of whether they are ingress or egress listeners, listeners can listen for three types of connections: HTTP, TCP, and UDP. HTTP listeners use the #HTTPListener definition whereas TCP listeners use the #TCPListener definition and finally UDP listeners are defined with #UDPListener.

Defining a listener on a service looks like this, where listener1, listener2, and listener3 are arbitrary names:

gsl.#Service & {

    ingress: {
        "listener1": {
            gsl.#HTTPListener
        }
        "listener2": {
            gsl.#UDPListener
        }
    }

    egress: {
        "listener3": {
            gsl.#TCPListener
        }
    }
}

This is an example of “embedding”.

Routes

Routes define string matching criteria a listener will use to forward incoming requests. These are only allowed from within a #HTTPListener. When a connection matches one of the routes, the listener will first pass the request through its filter chain and then perform any route prefixing, route rewriting, redirects, or retries that are specified by the route configuration. The proxy finally sends it to the connected upstream.

Setting a route on a listener looks like this:

gsl.#Service & {
	ingress: "listener1": {
        gsl.#HTTPListener

        routes: {
            "/app": { ... }
            "/app2": { ... }
            "/" : { ... }

        }
    }
}

Upstreams

Upstreams refer to the destination of a request and are the last object in the hierarchy. In #HTTPListeners they attach to routes and in #TCPListeners or #UDPListeners they attach directly onto the listener. Upstreams can either refer to their destination through a static IP or through service discovery. In the latter case, you must know the name and namespace of the service.

Creating upstreams on a #HTTPListener and a #TCPListener looks like this:

gsl.#Service & {
	ingress: "listener1": {
        gsl.#HTTPListener

        routes: {
            "/": {
                upstreams: {
                    gsl.#Upstream

                    "serviceDiscoveryName": {
                        namespace: "namespaceOfServiceDiscoveryName"
                    }
                    "staticIP": {
                        instances: [{host: "100.100.100.100", port: 80}]
                    }
                }
             }

        }
    }

    egress: "listener2": {
        gsl.#TCPListener
         
        // TCP listeners only have a single upstream
        upstream: {
            gsl.#Upstream

            name: "serviceDiscoveryName"
            namespace: "namespaceOfServiceDiscoveryName"
        }
    }
}

Notice how listener2 can only have a single upstream and how listener1 can address multiple upstreams. Connecting multiple upstreams to one route is useful for more advanced traffic shaping patterns like blue-green deployments.

Upstreams are another area where GSL expects a definition embedding.