Pythonic Ansible

"I should have stuck with shell scripts"

Ansible is an automation platform for "app deployment, configuration management and orchestration - all from one system". Compared to competitors, it's also well-established and (supposedly) comes with the ability to be imported/executed from Python apps. StackOverflow provides a simple example:

import json
import ansible.runner
import ansible.inventory

inventory = ansible.inventory.Inventory('path/to/your/inventory')
pm = ansible.runner.Runner(module_name='debug',module_args='var=hostvars',timeout=5,inventory=inventory,subset='all')
out = pm.run()
print json.dumps(out, sort_keys=True, indent=4, separators=(',', ': '))

Unfortunately, this example only works with Ansible <2.0. Ansible's documentation explains that Ansible 2.0 removed the "ansible.runner.Runner" interface and replaced it with a 45-line monstrosity touching nearly a dozen different Ansible components to simulate the initial example. Mucking through the monstrosity though, it fails on pretty much all counts; it only runs one Play within the Playbook, outputs everything as unicode strings, and barfs those into display with multiple lines per write. Perhaps we can do better?

Peeking Behind the Curtain

Ansible is ultimately a Python program, so it should be possible to hook somewhere and avoid writing an entire program just to call it. /usr/bin/ansible-playbook itself is a Python script, and it calls the ansible.cli.playbook module via some too-clever program name parsing (at least it isn't setuid).

#!/usr/bin/env python
me = os.path.basename(sys.argv[0])
...
target = me.split('-')
sub = target[1]
myclass = "%sCLI" % sub.capitalize()
mycli = getattr(__import__("ansible.cli.%s" % sub, fromlist=[myclass]), myclass)
...
cli = mycli(sys.argv)
cli.parse()
sys.exit(cli.run())

It is slightly concerning that the shell script doesn't appear to output anything directly, but perhaps the global display object is auto-detected (mmm... coupling). mycli(sys.argv) copies the program arguments to a member variable. cli.parse() extracts interesting command-line arguments and stores them in member variable flags. That means ansible.cli.playbook.PlaybookCLI.run should be where all the interesting stuff happens, but reading that reveals a near copy of the documented 2.0+ example - and ultimately just calls ansible.executor.playbook_executor.PlaybookExecutor (followed by a mess of spagetti code building multi-line strings for display). Unfortunately, there aren't really any more paths to follow after PlaybookExecutor; it calls the TaskQueueManager on each Play which dumps (some) data to display as strings.

Conclusion

Digging into Ansible feels a bit like happening upon Enterprise-grade FizzBuzz for the first time; confused, angry, and a little impressed it works. The code is massively coupled and the entire project's "importability" seems to arise more from Python being importable than any deliberate effort. ansible.cli may be particularly at fault here; the middleware anti-pattern essentially disguising the "importability" problems by building more and more cruft onto PlaybookCLI.

In conclusion, this particular adventure reinforced the ease, simplicity, and power of shell scripts. In direct comparison, the shell script took less time to be created from scratch, took far less domain-specific knowledge, and executed an order of magnitude faster than Ansible. At the end of the day, this appears to be the best way to call Ansible 2.0+ scripts from Python:

import os
import subprocess

def run_playbook(path, hosts=[], extra_args={}, debug=True):
    playdir,playname = os.path.split(path)

    if playdir: os.chdir(playdir)
    process_args = ['/usr/bin/ansible-playbook',playname]
    if len(hosts): process_args.extend(['--inventory','%s,'%','.join(hosts)])
    if len(extra_args): process_args.extend(['--extra-vars',json.dumps(extra_args)])
    print 'exec %s'%' '.join(process_args)

    p_stdout=[]
    process = subprocess.Popen(process_args, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
    for line in iter(process.stdout.readline,''):
        if debug: print line.rstrip()
        p_stdout.append(line)
    while process.poll() is None: pass
    return process.returncode,'\n'.join(p_stdout)