Introduction

One of the thorniest problems faced by engineers and operators is how to best expose and manage software configuration. Historically, centralized configuration files or environment variables offer strong solutions to this problem. However, with the advent of distributed networks and microservice architectures, the sprawl of configurations creates potentially dangerous problems. One small change in a sea of data points could now disrupt critical, day-to-day business operations.

Greymatter deals with this inevitability by introducing two technologies: GitOps and GSL (CUE).

Greymatter Specification Language & CUE

Greymatter's configuration management starts with how configuration is written. All configurations are written in a data validation language known as CUE. Depending on their user category, Greymatter users write CUE directly or through a bespoke domain-specific language built on top of CUE called Greymatter Specification Language GSL. Unlike other application networking platforms that utilize generic data formats like JSON or YAML,GSL and CUE provide a much richer experience centered around the developer and not the machine a developer interacts with.

We recommend reading about CUE on their official website.

GitOps

GitOps, in short, refers to an operations paradigm centered on the idea that git can be used both as an information store and a signal for change. All GitOps processes largely follow this pattern:

• Create a local change.

• Commit and push the change to a remote repo.

• On a detected change to the repo, have an automated process change a system based on the newly pushed information.

GitOps in Greymatter

The start of Greymatter's GitOps flow begins at how we segment the different user groups. A Greymatter git repository comes in two types:

• Mesh Administrator

• Tenant

Users of both types use similar GitOps flows to push their configurations into Greymatter. The generalized process looks like this:

1. An operator or tenant makes a configuration change on their local machine.

2. They use git to add, commit, and push their local change.

3. They create a request to merge the incoming changes with the main branch of their repo.

4. They solicit review from authorized members within their team.

5. The authorized member accepts the changes and they are merged into the main branch.

6. One of the Greymatter GitOps synchronization services detects a change and applies it to the mesh (assuming configuration correctness).

This process closely follows the standard GitOps process for any software system.

You might ask yourself why we chose to separate the two. In large enterprise systems, users depend on segmentation and self-service. Without those, their ability to easily locate, parse, and enact configuration changes dramatically decreases. Enterprises contain different stakeholders who follow unique cadences. We designed the systems from the start to avoid actors from stepping on each other's toes.

Mesh Admin GitOps

Mesh admins include engineers and other IT administrators who manage a core Greymatter deployment (and possibly the underlying infrastructure). They hold expertise in networking and cloud infrastructure technologies as well as the permissions to interact with their organization's infrastructure (e.g. creating namespaces in the Kubernetes cluster). Their primary responsibility lies in the installation of Greymatter and the continuous management of its core service configuration. As a result, their repo, referred to as the "operator repo" (since the Greymatter Operator ingests it), "Greymatter core repo", or simply, the "core repo", contains the configurations for core Greymatter software and the Operator's application settings.

Each core repo is connected to a single Greymatter Operator and a single Greymatter mesh. For this GitOps flow, the Greymatter Operator stands as the "GitOps synchronization service" which evaluates and applies configuration changes made to the core repo.

Mesh admins write core configurations in pure CUE.

Tenant GitOps

Tenants refer to a coalition of users who are concerned with the development of their application firstly and the integration of their application into the mesh secondly. They write code to satisfy business goals and may not know networking and cloud technologies as well as operators. Their git repositories, called "tenant repos", contain the service configuration specifically for the services within their project.

For instance, the team responsible for an enterprise user-facing dashboard application might have a tenant repo named "User Dashboard" which contains the configuration for all directly associated applications. By creating a project like this, users are guided into an organization system that naturally decreases the cognitive strain associated with understanding a project and the full scope of its configuration.

For each tenant project, the "GitOps synchronization service" is the Greymatter sync service. Every project has one and exactly one sync service.

Tenants write their service configurations in the Greymatter Specification Language.

Benefits of Greymatter's Configuration Management

Ultimately, Greymatter's configuration management works synergistically with the rest of the platform to increase automation and to give users higher fidelity in their systems. Automation and fidelity are two concepts that we believe best combat the complexity nightmare of modern application networks.

When we automate configuration tasks, we reduce boilerplate and shrink the points of failure caused by human error. This drives the speed, ease, and reliability of delivering application networking changes.

Fidelity, on the other hand, refers to the confidence users have in writing, maintaining, discovering, and evaluating configuration. Complexity can lead to a breakdown of trust. CUE, GSL, and GitOps re-introduces trust by increasing transparency, capturing change history, increasing configuration clarity, and providing strong pre-flight schema validation. With higher trust, or fidelity, in systems, users and decision-makers can both safely respond to shifting conditions and rest easy knowing that current systems perform to their expectations.