diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index 9aeabff..8415508 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -1,9 +1,7 @@ repos: - repo: https://github.com/k8s-at-home/sops-pre-commit - rev: v2.1.0 - hooks: - - id: forbid-secrets + rev: v2.1.1 + hooks: [{id: forbid-secrets}] - repo: https://github.com/zricethezav/gitleaks - rev: v8.6.1 - hooks: - - id: gitleaks + rev: v8.8.11 + hooks: [{id: gitleaks}] diff --git a/readme.md b/readme.md index aa79483..8bd018f 100644 --- a/readme.md +++ b/readme.md @@ -3,9 +3,8 @@ [🖥️ Upstream][upstream] • [🐙 GitHub Mirror][github] This repository contains the configuration, scripts, and other goodies for -building and managing my kubernetes clusters (right now, that's just my home -cluster). I share the source with you so you can make exactly the same mistakes -as I do. +building and managing my kubernetes clusters. I share the source with you so +you can make exactly the same mistakes as I do. ## Setup @@ -17,87 +16,9 @@ pre-commit install --install-hooks pre-commit autoupdate ``` -## Provision Machines +## Usage -Before we interact with the cluster, we have some manual work to do. - -### Manual Preparation - -- Currently, my nodes are Arch Linux machines on bare metal - - Nodes must be ready to be controlled via Ansible - - Have `python3` installed - - Need to be `ssh`-able from a controller (my workstation) - - `curl -L files.lyte.dev/key.pub >> ~/.ssh/authorized_keys` - - Nodes must support Longhorn: https://longhorn.io/docs/1.2.3/deploy/install/#installation-requirements - - Nodes must be running on a host filesystem that supports `file extents` - - Provisioning takes care of the rest - -### Automated Provisioning - -- Setup Ansible on the controller (from `./ansible`) - - `pushd ansible && ansible-galaxy install -r requirements.yml --force` -- Verify Ansible can reach hosts (from `./ansible`) - - `pushd ansible && ansible all -i inventory/hosts.yml --list-hosts` - - `pushd ansible && ansible all -i inventory/hosts.yml -m ping` -- Use Ansible to build the cluster as configured on all nodes (from `./ansible`) - - `pushd ansible && ansible-playbook -i inventory/hosts.yml ./build-k3s-cluster.yml` - -And the cluster is up! If you want to interact with it from your controller, -you can do this: - -- Copy the cluster information from the `./k3s-cluster-config.kubeconfig.yaml` file into - your existing `~/.kube/config` (or just copy it there if it doesn't exist) - - You will need to edit the host from `localhost`/`127.0.0.1` to the correct host - -```bash -ansible -i ansible/inventory/hosts.yml $REMOTE_HOST -m fetch \ - -a "src=/etc/rancher/k3s/k3s.yaml dest=./k3s-cluster-config.kubeconfig.yaml flat=yes" -# TODO: this did not work for me -# env KUBECONFIG="~/.kube/config:./k3s-cluster-config.kubeconfig.yaml" \ -# kubectl config view --flatten | sed "s/127.0.0.1/$REMOTE_HOST/" > ~/.kube/new-config -sed -i 's/127\.0\.0\.1/10.0.0.87' ~/.kube/config -``` - -### Automated Teardown - -```bash -cd ansible -ansible-playbook -i inventory/hosts.yml ./nuke-k3s-cluster.yml -``` - -## Setting up Flux - -- Install the `flux` CLI on a machine that can `kubectl` into the shiny, new cluster - - `paru -S flux-bin` - - or `curl -s https://fluxcd.io/install.sh | sudo bash` - - https://fluxcd.io/docs/installation/ -- Run the pre-flight check (you must have `~/.kube/config` setup!) - - `flux check --pre` -- Create the `flux-system` namespace - - `kubectl create namespace flux-system --dry-run=client -o yaml | kubectl apply -f -` -- Add the `sops-age` encryption key to the namespace - ```bash - pass k8s-clusters | grep age-secret-key | awk '{printf $2}' | \ - kubectl --namespace flux-system create secret generic sops-age \ - --from-file=age.agekey=/dev/stdin - ``` -- Install Flux (note the `fish`-isms here, so you may need to translate to `bash`-isms) - ``` - flux bootstrap git --url=(git remote get-url origin) --branch=master \ - --path=./cluster/home --private-key-file=$HOME/.ssh/flux-k8s-clusters - ``` - -### Troubleshooting - -If you screw something up here, here are some things you can do: - -- `flux uninstall` will nuke flux from the cluster so you can retry from the - beginning of this section -- If you get something like `sync path configuration ... would overwrite path ... of existing Kustomization`, you can edit the `path: ...` field in the - `flux-system/gotk-sync.yaml` file in whatever you're passing as `--path`, - commit, and try the bootstrap again -- You can pretty easily nuke the entire cluster and start from scratch as - a last resort? +See each cluster's individual readme for more information # To Do & Status @@ -118,10 +39,6 @@ If you screw something up here, here are some things you can do: - Change dummy domain to real domain - Change domain's nameserver to cloudflare - Should be all set! -- I want to look into Talos/Sidero + PXEBoot, since that could remove a lot of the ansible stuff? -- `k3s` has a decent amount of magic AFAICT, so I'd like to learn more about it - and all its components so I better understand what my system is actually - _doing_ [upstream]: https://git.lyte.dev/lytedev/k8s-clusters [github]: https://github.com/lytedev/k8s-clusters