Preparing the Windows Fleet for Samba

Off Windows tracks removing every remaining Windows dependency from the lab, starting with the two domain controllers that gave the series its name. The next post in the series covers the actual migration — an in-place replica join that swaps both Windows DCs for a Samba box named sidious without a single workstation noticing. This post covers the evening before that: getting the rest of the Windows fleet — the desktops, laptops, and piett, the Windows Server box running Veeam Backup & Replication and one of the two Windows-licensing reasons this whole series exists — under Ansible management and onto a trust chain that would survive the domain changing underneath them.

A quick note on ordering: this wasn’t originally planned as two posts. The domain migration itself was drafted first, but writing it made clear that the evening of fleet prep it depended on — and the real bugs found doing it — deserved its own post rather than a rushed paragraph of backstory. So this one comes first, chronologically correct, even though it was written second.

Why the fleet needed a channel that didn’t depend on the domain

The idea came from an offhand suggestion, not a plan. Mid-afternoon, while the migration plan document was still being written, I mentioned:

winrm might be enabled on the 3 windows servers. I can also enable ssh if that would help.

— Tod

Claude’s first answer undersold it — it filed SSH access under a separate, unrelated idea about CA inventory, missing the actual point of the suggestion. A few minutes later, still drafting the same plan document, Claude came back with the better answer: standing up SSH and Ansible on the fleet is low-risk, touches no AD, and buys two things this migration genuinely needed — a management and rollback channel to every machine that works independent of domain auth, and a mechanism to push a new root CA certificate everywhere at once.

That independence is the whole point, and it’s worth being specific about why. A Windows logon over a domain account needs a domain controller reachable to build the logon token — including over SSH. If the domain migration goes sideways mid-flight, the moment a rollback channel is needed most is exactly the moment a domain-account login is least likely to work. A local account, authenticating with an SSH key instead of a password, sidesteps that failure mode entirely.

The plan that came out of that conversation broke the work into three steps: enable OpenSSH and switch the default shell to PowerShell on each Windows box; build an Ansible inventory against a local admin account and SSH key; then write a playbook to push the new root CA. Steps two and three are what got committed to the ansible repo. Step one never touched git at all — it was done by hand, machine by machine, before the Ansible session even started. The evening’s own opening line confirms it:

I have gotten ssh running on the 3 windows servers (vader.ad.tod.net, maul.ad.tod.net, and piett.ad.tod.net) also on my windows desktop (todwin01.ad.tod.net)

— Tod

Worth admitting plainly: enabling OpenSSH by hand on seven machines is exactly the kind of manual, un-codified config change this homelab’s own conventions say to avoid — every managed host is supposed to have its state captured somewhere a rerun can reproduce it, not carried only in one person’s memory of which registry keys they clicked through on which box. This isn’t an excuse so much as technical debt taken on knowingly: the tool that would have prevented it (a small PowerShell script kept on a fileshare and run locally on each box, identical every time) wasn’t hard to imagine, it just wasn’t written, because writing it felt like a detour on a night already budgeted for the migration blitz the next morning. That’s a real gap, not a resolved one; it’s on the list for whenever a similar bootstrap problem comes up again.

Building the inventory: a domain controller doesn’t have a local SAM

The obvious way to create a local admin account and drop in an SSH key is a short PowerShell snippet — New-LocalUser, then Add-LocalGroupMember -Group Administrators. Run against three member/desktop machines, it worked. Run against vader, one of the two domain controllers, it didn’t:

Add-LocalGroupMember : Group Administrators was not found.

Claude’s read on it, when the error came back:

Stop — that error is the important part, and it’s specific to vader being a domain controller.

— Claude

A domain controller has no local SAM database — every account on it is a domain object, not a local one. New-LocalUser on a DC doesn’t fail cleanly; it’s liable to create a stray domain user object instead, one that’s about to get replicated straight into the new Samba directory the following morning. That’s genuinely AD 101 to anyone who’s administered a Windows domain day to day — this homelab hasn’t been one of those places. I’m a Linux sysadmin who dabbles in AD, which is a real part of why this migration exists at all: better to run infrastructure I actually understand at that depth than infrastructure I’m perpetually one step behind on. Neither of us — me typing the command, Claude proposing the snippet — was thinking in terms of “domain controller” at that moment; we were both still applying the same workstation-shaped template that had just worked cleanly on three other machines. That’s the more accurate failure to name: not a fact either of us was missing, but a template getting reused past the boundary where it stopped applying. The plan’s original wording had lumped the DCs in with the rest of the fleet for this rollback channel; this is the moment that stopped being viable, live, mid-session, not a design decision made up front. The fix was to exclude vader and maul entirely and scope the local-admin approach to hosts that actually have a local SAM to put an account in — piloting on a single member server first would have caught this before it touched a DC at all, and that’s exactly the lesson that shaped the final inventory group.

Two pieces of Ansible structure below, for anyone new to it: inventory.yml lists which hosts exist and what named groups they belong to (windows, here); group_vars/<group name>/vars.yml holds settings shared by every host in that group, so they don’t have to be repeated per-host. (“Infrastructure to Build Infrastructure”, from the Downsizing series, walks through a first Ansible playbook in more depth if this is unfamiliar territory.) Here’s the group this fleet landed in:

# inventory.yml
    # Windows fleet managed over SSH with a local 'ansible' admin + key
    # (connection settings in group_vars/windows/vars.yml). This is the
    # management/rollback channel for the Samba AD migration, deliberately
    # independent of domain auth. The DCs (vader, maul) are excluded — a DC
    # has no local accounts. Laptops are roaming and may be offline.
    windows:
      hosts:
        piett.ad.tod.net:        # member server
        todwin01.ad.tod.net:     # desktop
        peregrin.ad.tod.net:     # desktop
        msilaptop.ad.tod.net:    # laptop (roaming)
        sabine.ad.tod.net:       # laptop (roaming)

# group_vars/windows/vars.yml
ansible_connection: ssh
ansible_shell_type: powershell
ansible_user: ansible
ansible_ssh_private_key_file: "~/.ssh/ansible_windows"

Two more things went wrong setting up the five hosts that did qualify. piett, meant to be the pilot host, still failed ansible.windows.win_ping (rc=255) after the other four passed. Claude walked through verbose SSH debug output to find the cause: on piett, I’d pasted Claude’s generic PowerShell snippet into the Windows console without filling in the actual public key first, so administrators_authorized_keys on disk literally contained the placeholder text — PASTE ~/.ssh/ansible_windows.pub LINE HERE — instead of a key. Overwriting the file with Set-Content instead of appending with Add-Content fixed it.

That one’s worth sitting with for a second, because the bug isn’t really the interesting part. Claude’s snippet marked the placeholder clearly — PASTE ~/.ssh/ansible_windows.pub LINE HERE isn’t subtle. I pasted it into an elevated console on a live machine without reading it closely enough to notice. That’s not a paste mistake so much as a review habit slipping: by this point in the evening I was treating Claude’s PowerShell output as trustworthy enough to run without a close read, on a box that mattered. It wasn’t wrong that time in any way that broke something — it just didn’t work — but it’s exactly the kind of complacency that would have run something worse than a placeholder string if the snippet had contained one. Worth being honest about, rather than filing this purely as “typo, fixed.”

Separately, vader had been left on cmd.exe as its default shell while every other host had already moved to PowerShell — I asked Claude if we could switch it first so all five hosts matched, and fixed it with a reg add run over the still-cmd SSH session, since there was no PowerShell shell yet to set the PowerShell-shell registry key from.

This channel is also a new attack surface, and worth being explicit about what secures it — and what doesn’t yet. Every fleet host now has a local Administrators-group account that didn’t exist before, which is exactly the kind of standing privileged credential worth being paranoid about. Two things keep it from being a soft spot today: the account’s password is a random 24-character string, generated once per host and saved to LastPass — never typed, never reused, and never actually used for authentication day-to-day, since SSH auth is key-only and the password exists only to satisfy Windows’ local-account requirements. The account-creation script is written to be safe to re-run — it checks whether ansible already exists before generating a new password, so a second pass never silently rotates (or leaks) a credential that’s already in use. The key itself is ed25519, generated specifically for this fleet and not reused from any other purpose.

This is meant as a permanent management channel, not a break-glass account to disable once Samba is stable¹ — which raises the question a migration-focused evening didn’t have time to answer: how do these credentials get rotated? Right now, nothing does it. The password-never-expires flag and the lack of any rotation playbook mean the same key and password, generated once on 2026-06-20, are still what every fleet host trusts, indefinitely, until someone writes the Ansible role to rotate both on a schedule. That’s future work, not a solved problem — the channel is safe as configured, but “safe as configured, forever, with no rotation plan” is its own kind of risk.

With all five hosts responding, the collections that make Windows management possible got pinned alongside the inventory change:

 collections:
   - name: community.general
   - name: community.docker
+  # Windows fleet management over SSH (inventory group 'windows').
+  - name: ansible.windows
+    version: ">=3.0.0"
+  - name: community.windows
+    version: ">=3.0.0"

ansible windows -m ansible.windows.win_ping against all five came back pong.

Root CA distribution: a playbook that looked done and wasn’t

The whole reason this channel needed to exist before the migration, not just as a nice-to-have: the new Samba DC’s LDAPS certificate would eventually come from the homelab’s own internal root CA, and every Windows box that talks to a domain controller over LDAPS needs to trust that root first. distribute_root_ca.yml is the playbook that does it — stage the public root certificate onto each host, then import it into the machine’s trusted root store.

The first version worked on a single run, then failed the promise every Ansible playbook is supposed to keep: idempotency, zero changed on a rerun with nothing left to do. I caught it and said so directly:

the distribute playbook is not idempotent

— Tod

Claude didn’t deflect it onto the modules:

Good catch — that’s a real defect, and it’s my design’s fault, not the modules'.

— Claude

The diagnosis: the first draft staged the certificate to C:\Windows\Temp and then, as a tidiness step, deleted it again once the import finished — the instinct of a one-shot script, clean up after yourself, rather than a state manager, leave things where they belong. That single delete broke the whole thing: every subsequent run found the file missing, re-copied it, and re-deleted it again, guaranteeing a changed result forever. Staging to Temp in the first place wasn’t itself the bug — a copy-then-import into a system cert store from anywhere on disk works fine — the bug was expecting a persistent Ansible fact (0 changes on rerun) out of a directory the play itself had just emptied. The fix was to stop deleting it: stage the certificate to a stable, persistent path instead of a temp one, and let win_copy’s own checksum comparison make reruns genuine no-ops.

- name: Distribute the offline root CA to the Windows fleet
  hosts: windows
  gather_facts: false

  vars:
    root_ca_cert_src: "{{ playbook_dir }}/../pki/root-ca.crt"
    # Stable path the cert lives at on the Windows host. Persistent (not Temp) so
    # win_copy stays idempotent; ProgramData always exists, no parent to create.
    root_ca_remote_path: 'C:\ProgramData\tod-root-ca.crt'

  tasks:
    - name: Stage the root CA cert on the Windows host
      ansible.windows.win_copy:
        src: "{{ root_ca_cert_src }}"
        dest: "{{ root_ca_remote_path }}"

    - name: Import the root CA into LocalMachine\Root
      ansible.windows.win_certificate_store:
        path: "{{ root_ca_remote_path }}"
        state: present
        store_location: LocalMachine
        store_name: Root
      register: _root_ca_import

Once the fix landed, I confirmed it:

much better it ran the second time with 0 changes

— Tod

Leaving the public certificate sitting on disk is harmless — it’s a public cert, never a key — and it’s what makes the checksum comparison meaningful on the next run. On an eventual root rotation, the new file’s checksum differs automatically, so it re-copies and re-imports without anyone needing to remember to force it.

One deprecation warning came up mid-session and, correctly, went nowhere. A run of the playbook printed warnings about a deprecated _text argument spec somewhere in the module chain. Claude started tracing it into community.windows’s own vendored code — laps_password.py, psexec.py, files this playbook doesn’t even use — before I interrupted mid-tool-call:

if this is in a module, we don’t need to update it

— Tod

Claude agreed and dropped it: a deprecation warning inside a third-party collection’s own code isn’t something this playbook’s authors can fix, and it’s cosmetic until upstream does. Chasing warnings inside vendored dependencies is a way to spend an evening solving a problem that was never actually a problem.

A confirmation-gated shutdown, because there’s no module for it

The last piece that evening was a way to power the fleet down cleanly once it was no longer needed — eventually useful for decommissioning old hardware, not something exercised yet at the time it was written. ansible.windows and community.windows were both checked directly for a shutdown module; neither has one, only win_reboot. The playbook falls back to scheduling the built-in shutdown.exe /s directly, and adds a confirmation gate in front of it that aborts the entire play on anything other than a typed yes:

- name: Show which hosts will be shut down
  run_once: true
  ansible.builtin.debug:
    msg: >-
      The following {{ ansible_play_hosts_all | length }} host(s) will be SHUT DOWN
      in {{ shutdown_delay_seconds }}s once confirmed:
      {{ ansible_play_hosts_all | join(', ') }}

- name: Confirm before shutting anything down
  run_once: true
  ansible.builtin.pause:
    prompt: "Type 'yes' to shut down the host(s) listed above, anything else to abort"
  register: _shutdown_confirm

- name: Abort unless explicitly confirmed
  run_once: true
  ansible.builtin.assert:
    that: _shutdown_confirm.user_input | trim | lower == 'yes'
    success_msg: "Confirmed — shutting down."
    fail_msg: "Not confirmed (got '{{ _shutdown_confirm.user_input }}') — aborting, no hosts shut down."

- name: Schedule shutdown
  ansible.windows.win_command: >-
    shutdown.exe /s /t {{ shutdown_delay_seconds }} /c "{{ shutdown_comment }}"
  changed_when: true

The short delay before the actual shutdown (15 seconds, not zero) is deliberate — long enough for the task to return successfully and the SSH connection to close on its own terms, before the box powers off underneath it. A /t 0 shutdown would race the connection teardown.

What this bought, and what it cost

By the end of the evening, five of the seven Windows machines in the fleet had a working, domain-independent management channel: a local admin account, an SSH key, and Ansible modules that could push files, import certificates, and — carefully, with a human typing “yes” — turn the machines off. The two domain controllers were deliberately left out of that channel, for the same reason the very first attempt at it failed against vader: they don’t have the kind of account this model assumes exists.

None of this was strictly required to migrate the domain the next day — the DC join itself doesn’t depend on any of it. What it bought was insurance: a way to reach every other Windows box in the house that didn’t run through the thing actually being replaced, and a proven, idempotent way to get a new root of trust onto all of them once the new DC started issuing LDAPS certificates from it. Small, mostly unglamorous work — an inventory group, two playbooks, five gotchas caught by running the thing rather than by design review — done the night before the part of the story that actually moves the domain.

AI-assistant disclaimer: This post was drafted by Claude Code (Claude Sonnet 5, claude-sonnet-5) from project notes, commit history, decision records, and session transcripts, then reviewed and edited by me. It may contain inaccuracies; verify specifics before relying on them.