Invoking Ansible Inside Python Flask

Pasting sample code that can be used as is to run ansible inside python flask app.
This would be a very useful base implementation when devops need to control system over the web or someone tries to control systems over internet without cli

Prerequisites

Install Python3

#PYTHON3 ( Assuming Centos7 or RHEL Flavor). Change accordingly for your env
yum -y install https://centos7.iuscommunity.org/ius-release.rpm
yum -y install python36u
yum -y install python36u-pip
yum -y install python36u-devel

virtualenv –python=/usr/bin/python3.6 venv

 

from flask import (
    Blueprint
)

import json
import shutil
from collections import namedtuple
from ansible.parsing.dataloader import DataLoader
from ansible.vars.manager import VariableManager
from ansible.inventory.manager import InventoryManager
from ansible.playbook.play import Play
from ansible.executor.task_queue_manager import TaskQueueManager
from ansible.plugins.callback import CallbackBase
import ansible.constants as C


#Blueprint URL Configuration routes
bp = Blueprint('AnsibleController', __name__, url_prefix='/AnsibleController')


# Ansible Return callback
class ResultCallback(CallbackBase):
    """A sample callback plugin used for performing an action as results come in

    If you want to collect all results into a single object for processing at
    the end of the execution, look into utilizing the ``json`` callback plugin
    or writing your own custom callback plugin
    """
    def __init__(self):
        self.resp = []

    def v2_runner_on_ok(self, result, **kwargs):
        """Print a json representation of the result

        This method could store the result in an instance attribute for retrieval later
        """
        host = result._host
        self.resp.append(json.dumps({host.name: result._result}, indent=4))





#print (json.dumps(out, sort_keys=True, indent=4, separators=(',', ': ')))

@bp.route('/ping', methods=('GET', 'POST'))
def ping():
    # since API is constructed for CLI it expects certain options to always be set, named tuple 'fakes' the args parsing options object
    Options = namedtuple('Options',
                         ['connection', 'module_path', 'forks', 'become', 'become_method', 'become_user', 'check',
                          'diff'])
    options = Options(connection='local', module_path=['/etc/swiftui/.swiftui/ansible/mymodules'], forks=100, become=None, become_method=None,
                      become_user=None, check=False, diff=False)

    # initialize needed objects
    loader = DataLoader()  # Takes care of finding and reading yaml, json and ini files
    passwords = dict(vault_pass='secret')

    # Instantiate our ResultCallback for handling results as they come in. Ansible expects this to be one of its main display outlets
    results_callback = ResultCallback()

    # create inventory, use path to host config file as source or hosts in a comma separated string
    inventory = InventoryManager(loader=loader, sources='localhost,')

    # variable manager takes care of merging all the different sources to give you a unifed view of variables available in each context
    variable_manager = VariableManager(loader=loader, inventory=inventory)

    # create datastructure that represents our play, including tasks, this is basically what our YAML loader does internally.
    play_source = dict(
        name="Ansible Play",
        hosts='localhost',
        gather_facts='no',
        tasks=[
            dict(action=dict(module='shell', args='ls'), register='shell_out'),
            dict(action=dict(module='debug', args=dict(msg='{{shell_out.stdout}}')))
        ]
    )

    # Create play object, playbook objects use .load instead of init or new methods,
    # this will also automatically create the task objects from the info provided in play_source
    play = Play().load(play_source, variable_manager=variable_manager, loader=loader)

    # Run it - instantiate task queue manager, which takes care of forking and setting up
    # all objects to iterate over host list and tasks
    tqm = None
    try:
        tqm = TaskQueueManager(
            inventory=inventory,
            variable_manager=variable_manager,
            loader=loader,
            options=options,
            passwords=passwords,
            stdout_callback=results_callback,
            # Use our custom callback instead of the ``default`` callback plugin, which prints to stdout
        )
        result = tqm.run(play)  # most interesting data for a play is actually sent to the callback's methods
    finally:
        # we always need to cleanup child procs and the structres we use to communicate with them
        if tqm is not None:
            tqm.cleanup()

        # Remove ansible tmpdir
        shutil.rmtree(C.DEFAULT_LOCAL_TMP, True)

    return ''.join(results_callback.resp)

 

Linux Networking Fundamentals Part 1

I’ve been working on setting up couple of host and guest machines at my home/office for last couple of years. I wanted to replicate real world scenario where we have multiple hosts on multiple subnet and still interacting with each other. While working on this i  found that lot of people are either not aware of basic concepts or confused because of abundant information available online. I want to simplify networking in a short concise manner all in single post.

I would also show application layers along with network layers to make it more clear. I see that these 2 department don’t have any common knowledge in common everywhere in industry. I would love my developers to understand network connectivity while designing distributed application or even basic non distributed applications.

There are 5 major components of that any company or network admin has to look at

  1. Gateway
  2. Firewall
  3. Switch
  4. Router
  5. Hub

and 2 basic concepts

  1. Nic Cards/ Network Interface Cards
  2. Subnets.

 

Gateway : This is the target applicance/device/machine to which source computer sends all its packets for any networking. Assuming there is only one network interface card in source computer. (Note: A machine can have any number of interface cards as hardware supports). If we use gateway we mostly need to add static routes to all machines

Firewall:  This generally sits between cloud/internet and private network which can control flow of data packets from inside/outside interfaces. Example: Cisco ASA Applicance.  Without a firewall every network is susceptible to brute force attacks and heavy load on systems. (This is funny link which shows how everyone around the world are trying to attach each other digitally  http://map.norsecorp.com)

Hub: This is a device to which all networking devices connect to. This is good for lab testing but not efficient way of connecting devices.Major disadvantage it has is that it broadcasts information sent from one device over common channel that is visible to all other devices connected to that hub. This also creates unnecessary bandwidth issues as it floods information to everyone irrespective of who it is destined to.

Switch: This is an efficient way of creating a minimal network. Switches have multiple sockets to connect different available in network. Switches separate communication channels between all devices which makes it very efficient in connecting devices.

Router:  As the name suggests it routes traffic between different networks. This generally sits coupled with gateway and knows where to send packets to. It stores all information in routing table and decides best possible route for given packet. Router automatically takes care of routing removing need for nodes to add static routes

Below diagram explain a very high level connection scenario’s

networklayout

If you observe in above diagram i’ve one network with subnet 192.168.1.0/24 connected to one switch and other network 10.0.0.1/24 connected to different switch.

Both of them are connected via a  router/gateway

And gateway is connected to internet via Firewall.

This explain a minimalistic design of any company.

We can have multiple layers of switches and router sitting on top of each other in tree fashion but basic concepts remain the same.

Now coming to NIC Card/Network Interface Cards

NIC: This is located in each and every networking device and enable the device to get connected via ethernet. These days we can create Virtual NIC cards on any Operating system which behave same as physical NIC. There are other concepts of NIC called Nic Teaming or Bonding which improve bandwidth and performance of Network IO apart from providing failover connectivity. I would explain all these concepts in details in subsequent series.

Subnet:  Subnets are single most important concept that allow us to segregate different kind of networks. Originally when networks originated in 80’s there was concept of Class Based Networking

Example: for 192.168.1.1

Class A:   Network portion is just 192 and rest  168.1.1 defines the node address

Class B:  192.168 defines network portion and rest 1.1 is node address

Class C: 192.168.1 defines network and remaining 1 represents node address

This approach was good till early 90’s but then issue of ip address exhaustion started.

Then IETF introduced a concept of CIDR (Classless Internet Domain Routing). With introduction of this approach now IP Address is never complete without Network Mask. Example 192.168.1.1 doesn’t represent a node unless we define netmask. So now ip address have to defined as 192.168.1.1/24  where 24 represents 24 bits which is prefix or network portion of above address.

With advent of Virtual Machines/Containerization of networks and Operating system like linux we can replicate all above devices with software code.

I would start explain each concept in detail and how to configure and build multiple computers and network routes in subsequent series.

Technologies i’m going to use and topic coverage in next couple of months. I’m going to write each one fresh using latest version so that my code is not outdated.

  1. Vagrant ( To manage Virtual machines)
  2. Oracle VirtualBox ( Virtual machine provider . We can also use HyperV or VMWare)
  3. Docker (Lightweight replica of VM which can provide similar features)
  4. CentOS/Ubuntu ( I’m going to use both flavors of linux in different devices)
  5. Linux Routing Table for basic
  6. Quagga/Zebra to create powerful router
  7. Iptables used as simple firewall
  8. OpenVPN to setup VPN connections.
  9. Chef/Berkshelf (One of the most versatile OS provisioner )
  10. Shell Scripting ( This is obvious)
  11. Apache/PHP/Mysql For a demo website
  12. Cassandra as demo database to show how data gets across different racks in out virtual data centers (VM’s on different networks).

 

I’ve already have one base version written which i’m going to reuse in bits and pieces.

https://github.com/ashwinrayaprolu1984/chef-lamp

Part 2 of the series is at

https://ashwinrayaprolu.wordpress.com/2016/10/01/linux-networking-fundamentals-part-2/

 

Install Virtualbox Extension Commandline

$(vboxmanage -v)

Gives version like this  5.1.4r110228

We need split above version with “r” and use those to generate our required filename

Then get the file from download. virtualbox. org/virtualbox/5.1.4/

Installing is just issuing this command

sudo VBoxManage extpack install <YOUR FILE> –replace

#!/bin/bash
version=$(vboxmanage -v)
echo $version
var1=$(echo $version | cut -d ‘r’ -f 1)
echo $var1
var2=$(echo $version | cut -d ‘r’ -f 2)
echo $var2
file=”Oracle_VM_VirtualBox_Extension_Pack-$var1-$var2.vbox-extpack”
echo $file
wget ***: download. virtualbox. org/virtualbox/$var1/$file -O /tmp/$file
#sudo VBoxManage extpack uninstall “Oracle VM VirtualBox Extension Pack”
sudo VBoxManage extpack install /tmp/$file –replace