maurice/snix
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

chore(kontemplate): Prepare kontemplate for depot-merge

Browse source 
This merge will not yet include moving over to buildGo.nix, as support
for testing and such is not present in that library yet.
This commit is contained in:
Vincent Ambo 2019-12-20 22:13:07 +00:00
parent 064f65dec2
commit 795a974665
52 changed files with 0 additions and 195 deletions
Download patch file Download diff file Expand all files Collapse all files

106
ops/kontemplate/docs/cluster-config.md Normal file
View file

@ -0,0 +1,106 @@
Cluster configuration
==========================
Every cluster (or "environment") that requires individual configuration is specified in
a very simple YAML file in Kontemplate.
An example file for a hypothetical test environment could look like this:
```yaml
---
context: k8s.test.mydomain.com
global:
clusterName: test-cluster
defaultReplicas: 2
import:
- test-secrets.yaml
include:
- name: gateway
path: tools/nginx
values:
tlsDomains:
- test.oslo.pub
- test.tazj.in
- path: backend
values:
env: test
include:
- name: blog
values:
url: test.tazj.in
- name: pub-service
```
<!-- markdown-toc start - Don't edit this section. Run M-x markdown-toc-refresh-toc -->
**Table of Contents**
- [Cluster configuration](#cluster-configuration)
- [Fields](#fields)
- [`context`](#context)
- [`global`](#global)
- [`import`](#import)
- [`include`](#include)
- [External variables](#external-variables)
<!-- markdown-toc end -->
## Fields
This is documentation for the individual fields in a cluster context file.
### `context`
The `context` field contains the name of the kubectl-context. You can list context names with
'kubectl config get-contexts'.
This must be set here so that Kontemplate can use the correct context when calling kubectl.
This field is **required** for `kubectl`-wrapping commands. It can be left out if only the `template`-command is used.
### `global`
The `global` field contains a key/value map of variables that should be available to all resource
sets in the cluster.
This field is **optional**.
### `import`
The `import` field contains the file names of additional YAML or JSON files from which global
variables should be loaded. Using this field makes it possible to keep certain configuration that
is the same for some, but not all, clusters in a common place.
This field is **optional**.
### `include`
The `include` field contains the actual resource sets to be included in the cluster.
Information about the structure of resource sets can be found in the [resource set documentation][].
This field is **required**.
## External variables
As mentioned above, extra variables can be loaded from additional YAML or JSON files. Assuming you
have a file called `test-secrets.yaml` which contains variables that should be shared between a `test`
and `dev` cluster, you could include it in your context as such:
```yaml
# test-secrets.yaml:
mySecretVar: foo-bar-12345
# test-cluster.yaml:
context: k8s.test.mydomain.com
include:
- test-secrets.yaml
# dev-cluster.yaml:
context: k8s.dev.mydomain.com
include:
- test-secrets.yaml
```
The variable `mySecretVar` is then available as a global variable.
[resource set documentation]: resource-sets.md

170
ops/kontemplate/docs/resource-sets.md Normal file
View file

@ -0,0 +1,170 @@
Resource Sets
================
Resource sets are collections of Kubernetes resources that should be passed to `kubectl` together.
Technically a resource set is simply a folder with a few YAML and/or JSON templates in it.
<!-- markdown-toc start - Don't edit this section. Run M-x markdown-toc-refresh-toc -->
**Table of Contents**
- [Resource Sets](#resource-sets)
- [Creating resource sets](#creating-resource-sets)
- [Default variables](#default-variables)
- [Including resource sets](#including-resource-sets)
- [Fields](#fields)
- [`name`](#name)
- [`path`](#path)
- [`values`](#values)
- [`args`](#args)
- [`include`](#include)
- [Multiple includes](#multiple-includes)
- [Nesting resource sets](#nesting-resource-sets)
- [Caveats](#caveats)
<!-- markdown-toc end -->
# Creating resource sets
Simply create a folder in your Kontemplate repository and place a YAML or JSON file in it. These
files get interpreted as [templates][] during Kontemplate runs and variables (as well as template
logic or functions) will be interpolated.
Refer to the template documentation for information on how to write templates.
## Default variables
Sometimes it is useful to specify default values for variables that should be interpolated during
a run if the [cluster configuration][] does not specify a variable explicitly.
This can be done simply by placing a `default.yaml` or `default.json` file in the resource set
folder and filling it with key/value pairs of the intended default variables.
Kontemplate will error during interpolation if any variables are left unspecified.
# Including resource sets
Under the cluster configuration `include` key resource sets are included and required variables
are specified. For example:
```yaml
include:
- name: some-api
values:
version: 1.2-SNAPSHOT
```
This will include a resource set from a folder called `some-api` and set the specified `version` variable.
## Fields
The available fields when including a resource set are these:
### `name`
The `name` field contains the name of the resource set. This name can be used to refer to the resource set
when specifying explicit includes or excludes during a run.
By default it is assumed that the `name` is the path to the resource set folder, but this can be overridden.
This field is **required**.
### `path`
The `path` field specifies an explicit path to a resource set folder in the case that it should differ from
the resource set's `name`.
This field is **optional**.
### `values`
The `values` field specifies key/values pairs of variables that should be available during templating.
This field is **optional**.
### `args`
The `args` field specifies a list of arguments that should be passed to `kubectl`.
This field is **optional**.
### `include`
The `include` field specifies additional resource sets that should be included and that should inherit the
variables of this resource set.
The fully qualified names of "nested" resource sets are set to `${PARENT_NAME}/${CHILD_NAME}` and paths are
merged in the same way.
This makes it easy to organise different resource sets as "groups" to include / exclude them collectively
during runs.
This field is **optional**.
## Multiple includes
Resource sets can be included multiple times with different configurations. In this case it is recommended
to set the `path` and `name` fields explicitly. For example:
```yaml
include:
- name: forwarder-europe
path: tools/forwarder
values:
source: europe
- name: forwarder-asia
path: tools/forwarder
values:
source: asia
```
The two different configurations can be referred to by their set names, but will use the same resource
templates with different configurations.
## Nesting resource sets
As mentioned above for the `include` field, resource sets can be nested. This lets users group resource
sets in logical ways using simple folder structures.
Assuming a folder structure like:
```
├── backend
│   ├── auth-api
│   ├── message-api
│   └── order-api
└── frontend
├── app-page
└── login-page
```
With each of these folders being a resource set, they could be included in a cluster configuration like so:
```yaml
include:
- name: backend
include:
- name: auth-api
- name: message-api
- name: order-api
- name: frontend:
include:
- name: app-page
- name: login-page
```
Kontemplate could then be run with, for example, `--include backend` to only include the resource sets nested
in the backend group. Specific resource sets can also be targeted, for example as `--include backend/order-api`.
Variables specified in the parent resource set are inherited by the children.
### Caveats
Two caveats apply that users should be aware of:
1. The parent resource set can not contain any resource templates itself.
2. Only one level of nesting is supported. Specifying `include` again on a nested resource set will be ignored.
[templates]: templates.md
[cluster configuration]: cluster-config.md

153
ops/kontemplate/docs/templates.md Normal file
View file

@ -0,0 +1,153 @@
Kontemplate templates
=====================
The template file format is based on Go's [templating engine][] in combination
with a small extension library called [sprig][] that adds additional template
functions.
Go templates can either simply display variables or build more complicated
*pipelines* in which variables are passed to functions for further processing,
or in which conditionals are evaluated for more complex template logic.
It is recommended that you check out the Golang [documentation][] for the templating
engine in addition to the cherry-picked features listed here.
<!-- markdown-toc start - Don't edit this section. Run M-x markdown-toc-refresh-toc -->
**Table of Contents**
- [Kontemplate templates](#kontemplate-templates)
- [Basic variable interpolation](#basic-variable-interpolation)
- [Example:](#example)
- [Template functions](#template-functions)
- [Examples:](#examples)
- [Conditionals & ranges](#conditionals--ranges)
- [Caveats](#caveats)
<!-- markdown-toc end -->
## Basic variable interpolation
The basic template format uses `{{ .variableName }}` as the interpolation format.
### Example:
Assuming that you include a resource set as such:
```
- name: api-gateway
values:
internalHost: http://my-internal-host/
```
And the api-gateway resource set includes a ConfigMap (some fields left out for
the example):
```
# api-gateway/configmap.yaml:
---
kind: ConfigMap
metadata:
name: api-gateway-config
data:
internalHost: {{ .internalHost }}
```
The resulting output will be:
```
---
kind: ConfigMap
metadata:
name: api-gateway-config
data:
internalHost: http://my-internal-host/
```
## Template functions
Go templates support template functions which you can think of as a sort of
shell-like pipeline where text flows through transformations from left to
right.
Some template functions come from Go's standard library and are listed in the
[Go documentation][]. In addition the functions declared by [sprig][] are
available in kontemplate, as well as five custom functions:
* `json`: Encodes any supplied data structure as JSON.
* `gitHEAD`: Retrieves the commit hash at Git `HEAD`.
* `passLookup`: Looks up the supplied key in [pass][].
* `insertFile`: Insert the contents of the given file in the resource
set folder as a string.
* `insertTemplate`: Insert the contents of the given template in the resource
set folder as a string.
## Examples:
```
# With the following values:
name: Donald
certKeyPath: my-website/cert-key
# The following interpolations are possible:
{{ .name | upper }}
-> DONALD
{{ .name | upper | repeat 2 }}
-> DONALD DONALD
{{ .certKeyPath | passLookup }}
-> Returns content of 'my-website/cert-key' from pass
{{ gitHEAD }}
-> Returns the Git commit hash at HEAD.
```
## Conditionals & ranges
Some logic is supported in Golang templates and can be used in Kontemplate, too.
With the following values:
```
useKube2IAM: true
servicePorts:
- 8080
- 9090
```
The following interpolations are possible:
```
# Conditionally insert something in the template:
metadata:
annotations:
foo: bar
{{ if .useKube2IAM -}} iam.amazonaws.com/role: my-api {{- end }}
```
```
# Iterate over a list of values
ports:
{{ range .servicePorts }}
- port: {{ . }}
{{ end }}
```
Check out the Golang documentation (linked above) for more information about template logic.
## Caveats
Kontemplate does not by itself parse any of the content of the templates, which
means that it does not validate whether the resources you supply are valid YAML
or JSON.
You can perform some validation by using `kontemplate apply --dry-run` which
will make use of the Dry-Run functionality in `kubectl`.
[templating engine]: https://golang.org/pkg/text/template/
[documentation]: https://golang.org/pkg/text/template/
[sprig]: http://masterminds.github.io/sprig/
[Go documentation]: https://golang.org/pkg/text/template/#hdr-Functions
[pass]: https://www.passwordstore.org/

77
ops/kontemplate/docs/tips-and-tricks.md Normal file
View file

@ -0,0 +1,77 @@
Kontemplate tips & tricks
=========================
<!-- markdown-toc start - Don't edit this section. Run M-x markdown-toc-refresh-toc -->
**Table of Contents**
- [Kontemplate tips & tricks](#kontemplate-tips--tricks)
- [Update Deployments when ConfigMaps change](#update-deployments-when-configmaps-change)
- [direnv & pass](#direnv--pass)
<!-- markdown-toc end -->
## Update Deployments when ConfigMaps change
Kubernetes does [not currently][] have the ability to perform rolling updates
of Deployments and other resource types when `ConfigMap` or `Secret` objects
are updated.
It is possible to make use of annotations and templating functions in
Kontemplate to force updates to these resources anyways.
For example:
```yaml
# A ConfigMap that contains some configuration for your app
---
kind: ConfigMap
metadata:
name: app-config
data:
app.conf: |
name: {{ .appName }}
foo: bar
```
Now whenever the `appName` variable changes or we make an edit to the
`ConfigMap` we would like to update the `Deployment` making use of it, too. We
can do this by adding a hash of the parsed template to the annotations of the
created `Pod` objects:
```yaml
---
kind: Deployment
metadata:
name: app
spec:
template:
metadata:
annotations:
configHash: {{ insertTemplate "app-config.yaml" | sha256sum }}
spec:
containers:
- name: app
# Some details omitted ...
volumeMounts:
- name: config
mountPath: /etc/app/
volumes:
- name: config
configMap:
name: app-config
```
Now any change to the `ConfigMap` - either by directly editing the yaml file or
via a changed template variable - will cause the annotation to change,
triggering a rolling update of all relevant pods.
## direnv & pass
Users of `pass` may have multiple different password stores on their machines.
Assuming that `kontemplate` configuration exists somewhere on the filesystem
per project, it is easy to use [direnv][] to switch to the correct
`PASSWORD_STORE_DIR` variable when entering the folder.
[not currently]: https://github.com/kubernetes/kubernetes/issues/22368
[direnv]: https://direnv.net/