🤖 GPT 4 IaC Usecases

I’ve made some practical experiments with GPT in the IaC context and want to share the gist from the article:

🔥 Documentations & presentations
✅ Create Ansible lookup plugin
✅ Generate documentation
✅ Explain Jinja2 expressions
❓ Fix errors in Ansible roles
⛔️ Debug Ansible OTC dynamic inventory plugin
⛔️ Explain dependencies across the project
💡 PR reviewer
💡 Increase IaC test coverage

#article

✈️ Terraform for VMware: Key Takeaways and Lessons from Managing Hundreds of VMs

Managing a large number of virtual machines can be challenging. As our infrastructure grew, we needed a better way to handle the complexity. That’s when we turned to Terraform to help us manage our VMware environment more efficiently. the gist from the article:

Lessons learned
1. Use remote state
2. It's tricky to catch more than 10 changes at one time
3. With great power comes great responsibility
4. Bulk changes can lead to performance degradation
5. Avoid configuration drift between VMware & terraform
6. Instead of recreating VM you can simply move it
7. Use prevent_destroy to avoid VMs deletion
8. Use simple & obvious structure to make entrance level lower
9. Use to ansible dynamic inventory based on tags
10. It's possible to add the missing functionality

Benefits

1. It's really easy to perform bulk updates
2. Re-create infra from nothing
3. Trackable changes
4. Single source of the truth

Want to try
1. Terraform pull request automation via Atlantis
2. Build green button in jenkins
3. Windows automation
4. Cloud-init for setting hostname like here (sry it's in RU)
5. Tests for terraform like for ansible

#article

👩‍🦯 yoke - make IaC more complicated

I've just found the https://yokecd.github.io/docs/ . Authors tries to solve portability problem & make reproducible solution for IaC. In general it's a good idea in the right direction. But it introduces one more abstraction level. They propose pack the configuration into webassembly and execute it later. I assume, that it will be quite tricky troubleshoot in the feature in complex environment.

links:
- https://xeiaso.net/blog/2025/yoke-k8s/
- https://yokecd.github.io/docs/

#iac

Sometimes simplicity is a road to the hell

#fun

🎯 10 Lessons from Werner von Siemens

I’ve just read Personal Recollections of Werner von Siemens and noted:

1️⃣ Solve Real Problems – Siemens didn’t invent for fun. He built things like better telegraphs and generators because people needed them.
2️⃣ Start Small, Think Big - the initial idea was simple: create remote detonator, but ended as cross Atlantic land line.
3️⃣ Make Things More Efficient - each iteration of landlines, dinamos were better than the previous one. It like ci/cd.
4️⃣ Turn Ideas into Products - He didn’t just study science; he turned discoveries into useful inventions.
5️⃣ Take Smart Risks - Siemens took steps in new technology and business, but always calculated the risks.
6️⃣ Keep It Simple - He believed good engineering should be practical and useful.
7️⃣ Be Ethical and Long-Term Oriented - He treated employees well and built trust with customers.

Summary: engineering isn’t about making over complicated things - it’s about making life easier for people. It was 150 years ago and it’s still relevant.

#book

⌨️ Elementary computer

I have quite a weird idea for the next project:
1. Build a computer from scratch
2. Run a modern OS like https://collapseos.org/

What for? There is too much complexity and specialization so sometimes people don’t have a clue what’s happening 2 levels above. I’ve been working in IT almost 20 years and I’ve saw my hardware only couple of times.
So, the idea behind is to learn missing hardware part in my knowledge and have the big picture from a frontend to the hardware.

Right now I’m collecting the ideas, so if you have something interesting in that field..

Links:
- https://www.coursera.org/learn/build-a-computer
- https://collapseos.org/
- https://www.instagram.com/albin.jd

#diy

🔨 Simple, but not too simple

There is nice speech https://www.youtube.com/watch?v=ZSRHeXYDLko Preventing the Collapse of Civilization / Jonathan Blow (Thekla, Inc). My key takeaways:
- We are losing knowledge, instead of gaining it
- Software development is also affected. It's becoming too complex
- Many developers rely on frameworks and libraries without understanding how they work
- Ancient civilizations collapsed partly because they lost the ability to maintain critical knowledge. We may be repeating the same mistakes.
- Open-source and documentation aren't enough. Without real understanding and hands-on experience, it becomes useless over time.
- Short-term thinking leads to long-term problems & creating tech debt.
- We need to relearn simplicity. Instead of making systems more complex, we should focus on simplifying them so they can be understood by future generations.

As a sollution for the state problem there the answer https://aws.amazon.com/builders-library/reliability-and-constant-work/ :
- Simple systems fail less: The more complex a system, the more things can break.
- Avoid over-engineering: Adding unnecessary features or dependencies increases risk. Build only what’s needed.
- Automation should simplify, not overcomplicate: Automate repetitive tasks, but don’t create overly complex automation that’s harder to maintain than the problem it solves.
- Fewer Moving Parts = Fewer Failures
- Chaos Testing: If random failures cause unpredictable issues, the system is likely too complicated and it's time to review it.
- Learning from failures: After an outage, analyzing what went wrong and improving processes helps avoid similar failures in the future.

#refl

😮 freebsd alive

I used to use freebsd ages ago before 2010. I thought thar project is almost dead. But today i found that somebody uses it because of simplicity https://aumont.fr/posts/FreeBSD-Home-Server/

Prons:
- simplicity
- minimal ram/cpu footprint
- back compatibility

Cons:
- not possible to run desktop

Fun fact that I used freebsd as a desktop at my laptop. I spent almost 2 months compiling wine, x, etc to play fallout 2. So I can’t count it ad a downside. However, I can’t imagine IRL scenarios for freebsd in 2025

#freebsd

🛠 VmWare & Hardware version debug

Jenkins runs a powershell script and it creates a VM. Normally it works, but this morning I got this:

 

Virtual Machine [xxx] does not exist
Importing new Virtual Machine
New-VM : 4/4/2025 9:55:27 AM New-VM  An invalid argument "configSpec.guestId" was specified. 
At C:\...\powershell\commonScripts\Initialize-VM.ps1:247 char:14
+ ...    $newVM = New-VM -Name $VMName -ContentLibraryItem $templateObj -Da ...
+                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [New-VM], ViError
    + FullyQualifiedErrorId : ViCore_VMServiceImpl_DeployFromLibraryItem_ViNetException,VMware.VimAutomation.ViCore.Cm    dlets.Commands.NewVM

The first idea was that something has been changed in the powershell wrapper. But with the terraform I got more a less the same error. It was complaning about unsupported guest ID. In packer we explicitly specify rhel8_64Guest, but just in case I also decided to specify it in terraform https://registry.terraform.io/providers/hashicorp/vsphere/latest/docs/resources/virtual_machine#guest_id-1 and got different errors:

rhel8_64Guest:

│ Error: deploy error: An invalid argument "configSpec.guestId" was specified.
│ 
│   with module.Personal.vsphere_virtual_machine.ld-lgonchar-2,
│   on modules/Personal/ld-lgonchar-2.example.local.tf line 2, in resource "vsphere_virtual_machine" "ld-lgonchar-2":
│    2: resource "vsphere_virtual_machine" "ld-lgonchar-2" {

rockylinux_64Guest:

│ Error: cannot find OS family for guest ID "rockylinux_64Guest": could not find guest ID "rockylinux_64Guest"
│ 
│   with module.Personal.vsphere_virtual_machine.ld-lgonchar-2,
│   on modules/Personal/ld-lgonchar-2.example.local.tf line 2, in resource "vsphere_virtual_machine" "ld-lgonchar-2":
│    2: resource "vsphere_virtual_machine" "ld-lgonchar-2" {

However in the docs https://knowledge.broadcom.com/external/article/321876/determine-the-guest-os-from-a-vm-configu.html both of them exist. The next idea was to epoxrt list of guest IDs:

[VMware.Vim.VirtualMachineGuestOsIdentifier].GetEnumValues()

...
rhel8_64Guest
...
rockylinux_64Guest
almalinux_64Guest

and again... both of them exist. Also the windows VM were created without any problems. However, my workmate spotted that the different hardware version is available for customezation in web UI. I also found it in the doc https://registry.terraform.io/providers/hashicorp/vsphere/latest/docs/resources/virtual_machine#hardware_version-1. So, If I set the hardware_version param to 21 it works because rockylinux_64Guest is introduced.

What happens?
1. We used to have the VM hardware compatibility set to version 19. And the templates we created used to be detected as RHEL8. And compatibility for version 19 supported RHEL 8.
2. Weekly template rebuild happens
3. what's happening right now?
- The template is still defined as RHEL 8
- During deployment VMWare automatically detects it as Rocky Linux 8
- Vmware sets the Guest OS to rockylinux_64Guest.
- rockylinux_64Guest is not compatible with VM version 19
- When the VM is startted it get the error The guest operating system 'rockylinux_64Guest' is not supported.
4. The compat level in packer template was increased to 21 and rebuilt.

Why it happens?

1. The first idea was that VMWare tools was updated(we are installing the latest one). but it's the same
2. The Vcenter version is also the same.
3. The code is also the same.
4. Something else has been updated inside the VM template(we also install all updates) like firmware but i've not find the diff yet. also the kernel https://oraclelinux.pkgs.org/8/ol8-baseos-latest-x86_64/kernel-4.18.0-553.46.1.el8_10.x86_64.rpm.html was suspicious but I found nothing interesting in the release notes
5. i also upgraded rocky 8.6->8.10 at a test VM just to exclude the changes in the /etc/redhat-release

so.. I don't know why.. wip..

#vmware #debug #terraform

📂 Photo archive

I have about 400gb of photos for 20+ years. The question is how to store it for the next 40 years? There are some problems:
- Durability - if it's stored on the HDD I lost it or it's damaged?
- Portability - if it's stored in google phots and the service has gone. how to restore data & meta data?
- Indexing - how to find desired photo?

After some iterations I've concluded with that:

💾 Storages
1. Get a synology storage(I have enough fun at work, so i looked for almost zero administration solution).
2. Get a cloud account with webdav interface and configure mirroring to it.
3. Get an external HDD and configure auto backup to it if it's connected to a NAS.
4. Configure phones automatically upload photos to a sort directory

🗄 Structure

.../photos/
  life/
    2012/
      2012-03-23 +tag1+ Very important event
      2012-03-23 +tag2+ not that important event
      2012-03-24 +tag3+ Another important event
    2013/
    ...
    2025/
  sort/
    lev/
      ...
      2025-04-08/
      2025-04-09/
    ...

Finally the structure is timeline & some tags. There are some manual proceses
1. Run a script to move files from sort to the date directories
2. Review the photos in the date directories, add tags and move it to the main timeline

⛳️ Tags
I decided to use + as delimiter because it should be supported by linux, mac, win. Also, I've decided to use 5 chars tags to make aligned. Some of them are self-explained. some not.
- autos - cars
- bikes - bicycles
- craft - DYI
- confs - conferences & meetups
- daily - blog/vlog
- event - just an important event like marriage, graduation, ...
- fixit - apartments renovation
- furry - pets, cats, dogs
- hikes - different hikes
- tours - trips & travels
- lions - I'm collecting, the lions picture. The rules are simple: you send me a picture + GPS, I'll send you later back photo from that place
- party - parties
- sport - sport events
- shows - music perfomances & conciertos
- study - pictures from school or university
- shoot - photo session
- works - work related

🏁 Result
1. I have portable solution. I almost 100% that in 40 years I'll have a way to read JPG files.
2. I have tags in place & can use find for searching.
3. If the hdd, NAS, cloud are dead because of fire/flood/meteor I still have a copy

💡 Plans:
1. Update EXIFs for photos 1950 - 2015 years: GPS & dates are missing
2. Add external indexer to make cool videos be place/person. But it shouldn't touch the master copy
- https://github.com/jabberjabberjabber/ImageIndexer
- https://www.photoprism.app/
- Photos + mac mini
3. Scan old photos. it might be interesting https://estherschindler.medium.com/the-old-family-photos-project-lessons-in-creating-family-photos-that-people-want-to-keep-ea3909129943

#photos

🗣 DevConf

I'm making the speach about GPT 4 IaC at https://www.devconf.info/cz/ . So, If you have a chance to visit it please ping me - will try to meet in person. otherwise, I'll post the important & interesting sumaries here.

June 12-14, 2025
Brno, Czech Republic

upd:
- schedule https://pretalx.devconf.info/devconf-cz-2025/schedule/
- my speach https://pretalx.devconf.info/devconf-cz-2025/talk/R7VQXB/ will be 2025-06-13 at 15:30 in E104

#speach #news

⚡️ Blackout in Spain & Portugal 2025-04-28

At 12:30, the grid lost over 15GW of power(about 60% of its total load).

❓What caused it? There’s no official answer, possible reason:
1. A short circuit triggered by a fire near a high-voltage line
2. This might destabilize a nuclear plant between Bordeaux and Toulouse
3. The domino effect led to a cascade failure & isolated Spain and Portugal from the rest of Europe

💡 Why did it take so long to restore? Because the system went down completely, operators couldn’t just flip a switch. They had to perform a black start

🛠 What is a black start? you need electricity to make electricity. How it works:
1. Tiny generators(eg diesel-based), kick in first-they don’t need the grid to run
2. They bring online slightly bigger power plants(eg hydro or gas turbines)
3. These medium stations help boot up larger stations like nuclear

Step-by-step, the grid is rebuilt. During that process:
- voltage and frequency must be matched
- the generation and consumption must be aligned

🌍 Terraform + OTC + windows

Surprisingly it was easy to create, but tricky to get access. It's not really well describe in the docs https://registry.terraform.io/providers/opentelekomcloud/opentelekomcloud/latest/docs/resources/compute_instance_v2#encrypted_password-1

1. Create the instance

resource "opentelekomcloud_compute_instance_v2" "win" {
  name      = "win"
  flavor_id = "s3.large.4"
  # Enterprise_Windows-Server_2025_STD_amd64_uefi_latest
  image_id          = "66b9843e-f662-40b2-a347-da1f99f314da"
  key_pair          = opentelekomcloud_compute_keypair_v2.pubkey.pub_key_name
...
}
...
output "win_admin_password_encrypted" {
  value     = opentelekomcloud_compute_instance_v2.win.encrypted_password
  sensitive = true
}

2. wait about ~5-10 mins till encrypted_password is reachable
3. Decode the password

echo pass - `terraform output -raw win_admin_password_encrypted | base64 -d | openssl rsautl -decrypt -inkey /path/to/private/key`

Alternative 1:
It's possible to print the password if terraform is able to decrypt

data "opentelekomcloud_compute_instance_v2" "win" {
  id                    = opentelekomcloud_compute_instance_v2.win.id
  ssh_private_key_path  = "/path/to/private/key"
}

output "win_admin_password" {
  value     = data.opentelekomcloud_compute_instance_v2.win.password
  sensitive = true
}

Alternative 2:
If there is cloud init inside, than it can be customized like that:

resource "opentelekomcloud_compute_instance_v2" "win" {
  name      = "win"
  flavor_id = "s3.large.4"
  # Enterprise_Windows-Server_2025_STD_amd64_uefi_latest
  image_id          = "66b9843e-f662-40b2-a347-da1f99f314da"
  user_data         = base64encode(<<-EOF
#ps1
$pwdAdmin = ConvertTo-SecureString "versysecurepassword" -AsPlainText -Force
Set-LocalUser -Name Administrator -Password $pwdAdmin -PasswordNeverExpires:$true
EOF
  )

#terraform #til #otc #windows

🛠 Ansible doesn't support python path with spaces

1️⃣ Jenkins job issue
I have jenkins job to test ansible roles. The job was was moved to the jenkins folder with the space & the error appeared:

<127.0.0.1> EXEC /bin/sh -c 'ANSIBLE_ASYNC_DIR=/home/jenkins_slave/.ansible_async/ /home/jenkins_slave/workspace/Ansible Guild/Snapshot/.venv/bin/python3.12 /home/jenkins_slave/...
failed: [localhost] (item=postgresql-instance) => {
    "ansible_loop_var": "item",
...
    },
    "module_stderr": "/bin/sh: /home/jenkins_slave/workspace/Ansible: No such file or directory\n",
    "module_stdout": "",
    "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error",
    "rc": 127
}

I found that it caused by
1. calling community.docker.docker_container https://github.com/ansible-community/molecule-docker/blob/main/src/molecule_docker/playbooks/destroy.yml which executing
2. the python code https://github.com/ansible-collections/community.docker/blob/main/plugins/module_utils/module_container/module.py which called
3. the python code https://github.com/ansible/ansible/blob/devel/lib/ansible/modules/async_wrapper.py

2️⃣ Reproduce
so it wasn't about the molecule. I decided to reproduce it:

1. Prepare the venv

 mkdir "test dir"
 cd test\ dir/
 export PYTHONPATH=/usr/lib64/python3.6/site-packages
 python3.12 -m venv .venv --system-site-packages
 . .venv/bin/activate
 pip3 install --upgrade pip
 pip3 install -r ../example/requirements.txt 

2. Run debug

ANSIBLE_KEEP_REMOTE_FILES=1 ansible  -vvvv -m stat -c local -i 'asd,' asd
...
python3 ./AnsiballZ_stat.py explode

3. check internals

 echo PLATFORM; uname; echo FOUND; command -v 'python3.12'; command -v 'python3.11'; command -v 'python3.10'; command -v 'python3.9'; command -v 'python3.8'; command -v 'python3.7'; command -v 'python3.6'; command -v '/usr/bin/python3'; command -v '/usr/libexec/platform-python'; command -v 'python2.7'; command -v '/usr/bin/python'; command -v 'python'; echo ENDFOUND
PLATFORM
Linux
FOUND
/home/username/test dir/.venv/bin/python3.12
/home/username/.pyenv/shims/python3.9
/usr/bin/python3.8
/usr/bin/python3.6
/usr/bin/python3
/usr/libexec/platform-python
/home/ms/lgonchar/test dir/.venv/bin/python
ENDFOUND

the module detected symlink and caused the issue.

4️⃣ What is the source?
1. .venv/lib64/python3.12/site-packages/ansible/executor/interpreter_discovery.py:60

try:
...
    shell_bootstrap = "echo PLATFORM; uname; echo FOUND; {0}; echo ENDFOUND".format('; '.join(command_list))

2. .venv/lib64/python3.12/site-packages/ansible/plugins/action/__init__.py

found_interpreters = [interp.strip() for interp in match.groups()[1].splitlines() if interp.startswith('/')]
quoted_interpreters = []
for interpreter in found_interpreters:
    if ' ' in interpreter and not (interpreter.startswith('"') or interpreter.startswith("'")):
        quoted_interpreters.append('"%s"' % interpreter)
    else:
        quoted_interpreters.append(interpreter)
found_interpreters = quoted_interpreters

so the idea was to modify the .venv/lib64/python3.12/site-packages/ansible/plugins/action/__init__.py

if ' ' in interpreter:
    interpreter = shlex.quote(interpreter)
async_cmd = [interpreter, remote_async_module_path, async_jid, async_limit, remote_module_path]

It fixed the python detection but also there are some other places were _low_level_execute_command is called and much work to do.

5️⃣ Final
1. Rename the jenkins folder without space
2. Propose to use the first interpreter without spaces https://github.com/ansible/ansible/issues/82535#issuecomment-2910076878

#ansible #debug #jenkins