Dylancyclone

joined 2 months ago

No that's totally fair! I'm a huge fan of making things reproducible since I've ran into too many situations where things need to be rebuilt, and always open to ways to improve it. At home I use ansible to configure everything, and at work we use ansible and declare our entire Jenkins instance as (real) code. I don't have the time (and I'm low-key scared of the rabbit hole that is) Nix, and to me my homelab is something that is configured (idempotently) rather than something I wanted to handle with scripts.

I even wrote some pytest-like scripts to test the playbooks to give more productive errors than their example errors, since I too know that pain well :D

That said, I've never heard of PyInfra, and am definitely interested in learning more and checking out that talk. Do you know if the talk will be recorded? I'm not sure I can watch it live.

I love the "Warning: This talk may cause uncontrollable urges to refactor all your Ansible playbooks" lol I'm ready

While it is true that Ansible is a different tool that you need to learn the basics of (if you want to edit/add applications), all of the docker stuff is pretty comparable. For example, this is the equivalent of a docker compose file for SilverBullet (note taking app): https://github.com/Dylancyclone/ansible-homelab-orchestration/blob/main/roles/silverbullet/tasks/main.yml

You can see it's volumes, environment variables, ports, labels, etc just like a regular docker compose (just in a slightly different format, like environment variables are listed as env instead of environment), but the most important thing is that everything is filled in with variables. So for SilverBullet, any of these variables can be overwritten, and you'd never have to look at/tweak the "docker compose." Then, if any issue is found in the playbook, anyone can pull in the changes and get the fix without any work from themselves, and if manual intervention is needed (like an app updated and now requires a new key or something), the playbook can let you know to avoid breaking something: https://dylancyclone.github.io/ansible-homelab-orchestration/guides/updating/#handling-breaking-changes

[–] Dylancyclone@programming.dev 14 points 2 days ago (6 children)

If you'll let me self promote for a second, this was part of the inspiration for my Ansible Homelab Orchestration project. After dealing with a lot of those projects that practically force you to read through the code to get a working environment, I wanted a way to reproducably spin up my entire homelab should I need to move computers or if my computer dies (both of which have happened, and having a setup like this helped tremendously). So far the ansible playbook supports 117 applications, most of which can be enabled with a single configuration line:

immich_enabled: true
nextcloud_enabled: true

And it will orchestrate all the containers, networks, directories, etc for you with reasonable defaults. All of which can be overwritten, for example to enable extra features like hardware acceleration:

immich_hardware_acceleration: "-cuda"

Or to automatically get a letsencrypt cert and expose the application on a subdomain to the outside world:

immich_available_externally: true

It also comes with scripts and tests to help add your own applications and ensure they work properly

I also spent a lot of time writing the documentation so no one else had to suffer through some of the more complicated applications haha (link)

If you'll let me self promote for a second, this was part of the inspiration for my Ansible Homelab Orchestration project. After dealing with a lot of those projects that practically force you to read through the code to get a working environment, I wanted a way to reproducably spin up my entire homelab should I need to move computers or if my computer dies (both of which have happened. So far the ansible playbook supports 117 applications, most of which can be enabled with a single configuration line:

<application_name>_enabled: true

For example:

portainer_enabled: true
nextcloud_enabled: true
[–] Dylancyclone@programming.dev 1 points 2 months ago

That is very true, I suppose a more accurate way to say it would be the playbook does not need to run as the root user, and can instead use the permissions of a regular user. This lets all the volume mounts be owned by your user, instead of root. I think it's still an important distinction to make though, since by not running the playbook as root, the playbook can't directly change any of your server's settings, and only has the access the user you're SSHing as has.

Yes, this playbook is intended to orchestrate an already set up environment. I know Ansible can easily install and set up docker (using something like the awesome https://github.com/geerlingguy/ansible-role-docker), but I decided against it (at least for now) for two main reasons: Firstly to avoid becoming the root user, and secondly to avoid the Ansible role installing a second version of Docker, causing things to break. I ran into that myself while testing this playbook, where I had set up a Ubuntu VM, told the installer to install Docker during setup not knowing it would install it through snap, then the Ansible playbook would install docker again through conventional means causing a lot of strange problems. So instead I opted to let the user install docker however they'd like and not have any gotchas like "Remember to add --skip-tags="docker" if you installed Docker during OS installation on Ubuntu" or uninstalling their version of docker for them

[–] Dylancyclone@programming.dev 10 points 2 months ago (3 children)

This is definitely inspired by ansible-nas! I'd also used it for awhile, and made my own fork to add/fix things since the project has gone a little dormant. I started making so many changes though that I started fresh and it turned it into a whole project of it's own. You can see a list of differences here: https://dylancyclone.github.io/ansible-homelab-orchestration/guides/introduction/

Or copy-pasted:

  • Does not require root to run
  • Runs containers with minimum privileges (no root unless absolutely necessary)
  • Allow any application to have it's name, image name and version overridden (in case of version pinning or running forks)
  • Does not modify existing system settings/configuration
  • Separates DNS access from external access (for example, access portainer.example.com on local network without exposing it to the internet)
  • Notifies the user of breaking changes in an application's ansible role before updating them
  • Everything is optional, doesn't install anything except what's configured by the user
  • Cleans up networks and containers more gracefully after disabling applications, and ensure nothing is left dangling
  • Supports more OSs than just Ubuntu
  • Includes suite of tests to ensure clean code and functionality
    • All created containers and networks are removed when stopping applications
    • Ensures all applications have consistent variable names and settings
    • Ensures all applications properly implement DNS and external access settings
    • Ensures no port conflicts between applications
    • etc
[–] Dylancyclone@programming.dev 12 points 2 months ago

This project was basically how I learned Ansible haha. It is incredibly powerful and this really only scratches the surface

 

Hi! I wanted to share a project I've been working on that lets you set up a huge variety of applications really quickly and painlessly. For example, instead of setting up and configuring a large docker compose for a complicated application like Immich, this playbook lets you simply write:

immich_enabled: true

And it will orchestrate all the containers, networks, directories, etc for you with reasonable defaults. All of which can be overwritten, for example to enable extra features like hardware acceleration:

immich_hardware_acceleration: "-cuda"

Or to automatically get a letsencrypt cert and expose the application on a subdomain to the outside world:

immich_available_externally: true

Included in the playbook is a ton of complicated applications, like Paperless, Meelo, TTRSS, Dawarich, as well as common self-hosted apps like Jellyfin, Home Assistant, Syncthing, Navidrome, Grafana, and a whole lot more (totaling 116 right now!).

It also comes with scripts and tests to help add your own applications (ansible roles) and ensure they work properly

I've been using this project myself for awhile, including to assist with moving from one homelab computer to another (point the playbook to the new address, run it to install everything, and restore a backup from the previous computer for data), and I hope it can help someone else build up their homelab without sinking too much time into setting things up

Here's the repository: https://github.com/Dylancyclone/ansible-homelab-orchestration

And the documentation (that I'm really proud of :D): https://dylancyclone.github.io/ansible-homelab-orchestration/