Update readme and general cleanup

This commit is contained in:
Daniel Flanagan 2022-07-05 14:57:18 -05:00
parent 6d79e716f7
commit eb550f422b
Signed by: lytedev
GPG key ID: 5B2020A0F9921EF4
2 changed files with 8 additions and 93 deletions

View file

@ -1,9 +1,7 @@
repos: repos:
- repo: https://github.com/k8s-at-home/sops-pre-commit - repo: https://github.com/k8s-at-home/sops-pre-commit
rev: v2.1.0 rev: v2.1.1
hooks: hooks: [{id: forbid-secrets}]
- id: forbid-secrets
- repo: https://github.com/zricethezav/gitleaks - repo: https://github.com/zricethezav/gitleaks
rev: v8.6.1 rev: v8.8.11
hooks: hooks: [{id: gitleaks}]
- id: gitleaks

View file

@ -3,9 +3,8 @@
[🖥️ Upstream][upstream] • [🐙 GitHub Mirror][github] [🖥️ Upstream][upstream] • [🐙 GitHub Mirror][github]
This repository contains the configuration, scripts, and other goodies for This repository contains the configuration, scripts, and other goodies for
building and managing my kubernetes clusters (right now, that's just my home building and managing my kubernetes clusters. I share the source with you so
cluster). I share the source with you so you can make exactly the same mistakes you can make exactly the same mistakes as I do.
as I do.
## Setup ## Setup
@ -17,87 +16,9 @@ pre-commit install --install-hooks
pre-commit autoupdate pre-commit autoupdate
``` ```
## Provision Machines ## Usage
Before we interact with the cluster, we have some manual work to do. See each cluster's individual readme for more information
### Manual Preparation
- Currently, my nodes are Arch Linux machines on bare metal
- Nodes must be ready to be controlled via Ansible
- Have `python3` installed
- Need to be `ssh`-able from a controller (my workstation)
- `curl -L files.lyte.dev/key.pub >> ~/.ssh/authorized_keys`
- Nodes must support Longhorn: https://longhorn.io/docs/1.2.3/deploy/install/#installation-requirements
- Nodes must be running on a host filesystem that supports `file extents`
- Provisioning takes care of the rest
### Automated Provisioning
- Setup Ansible on the controller (from `./ansible`)
- `pushd ansible && ansible-galaxy install -r requirements.yml --force`
- Verify Ansible can reach hosts (from `./ansible`)
- `pushd ansible && ansible all -i inventory/hosts.yml --list-hosts`
- `pushd ansible && ansible all -i inventory/hosts.yml -m ping`
- Use Ansible to build the cluster as configured on all nodes (from `./ansible`)
- `pushd ansible && ansible-playbook -i inventory/hosts.yml ./build-k3s-cluster.yml`
And the cluster is up! If you want to interact with it from your controller,
you can do this:
- Copy the cluster information from the `./k3s-cluster-config.kubeconfig.yaml` file into
your existing `~/.kube/config` (or just copy it there if it doesn't exist)
- You will need to edit the host from `localhost`/`127.0.0.1` to the correct host
```bash
ansible -i ansible/inventory/hosts.yml $REMOTE_HOST -m fetch \
-a "src=/etc/rancher/k3s/k3s.yaml dest=./k3s-cluster-config.kubeconfig.yaml flat=yes"
# TODO: this did not work for me
# env KUBECONFIG="~/.kube/config:./k3s-cluster-config.kubeconfig.yaml" \
# kubectl config view --flatten | sed "s/127.0.0.1/$REMOTE_HOST/" > ~/.kube/new-config
sed -i 's/127\.0\.0\.1/10.0.0.87' ~/.kube/config
```
### Automated Teardown
```bash
cd ansible
ansible-playbook -i inventory/hosts.yml ./nuke-k3s-cluster.yml
```
## Setting up Flux
- Install the `flux` CLI on a machine that can `kubectl` into the shiny, new cluster
- `paru -S flux-bin`
- or `curl -s https://fluxcd.io/install.sh | sudo bash`
- https://fluxcd.io/docs/installation/
- Run the pre-flight check (you must have `~/.kube/config` setup!)
- `flux check --pre`
- Create the `flux-system` namespace
- `kubectl create namespace flux-system --dry-run=client -o yaml | kubectl apply -f -`
- Add the `sops-age` encryption key to the namespace
```bash
pass k8s-clusters | grep age-secret-key | awk '{printf $2}' | \
kubectl --namespace flux-system create secret generic sops-age \
--from-file=age.agekey=/dev/stdin
```
- Install Flux (note the `fish`-isms here, so you may need to translate to `bash`-isms)
```
flux bootstrap git --url=(git remote get-url origin) --branch=master \
--path=./cluster/home --private-key-file=$HOME/.ssh/flux-k8s-clusters
```
### Troubleshooting
If you screw something up here, here are some things you can do:
- `flux uninstall` will nuke flux from the cluster so you can retry from the
beginning of this section
- If you get something like `sync path configuration ... would overwrite path ... of existing Kustomization`, you can edit the `path: ...` field in the
`flux-system/gotk-sync.yaml` file in whatever you're passing as `--path`,
commit, and try the bootstrap again
- You can pretty easily nuke the entire cluster and start from scratch as
a last resort?
# To Do & Status # To Do & Status
@ -118,10 +39,6 @@ If you screw something up here, here are some things you can do:
- Change dummy domain to real domain - Change dummy domain to real domain
- Change domain's nameserver to cloudflare - Change domain's nameserver to cloudflare
- Should be all set! - Should be all set!
- I want to look into Talos/Sidero + PXEBoot, since that could remove a lot of the ansible stuff?
- `k3s` has a decent amount of magic AFAICT, so I'd like to learn more about it
and all its components so I better understand what my system is actually
_doing_
[upstream]: https://git.lyte.dev/lytedev/k8s-clusters [upstream]: https://git.lyte.dev/lytedev/k8s-clusters
[github]: https://github.com/lytedev/k8s-clusters [github]: https://github.com/lytedev/k8s-clusters