Tutorial: Build vSphere inventory tree

In this tutorial we're going to create a vSphere inventory tree using standard calls of the Perl SDK. The result should be similar to what a vSphere client shows, so that you can mimic its behaviour in your application.

As we all know, vSphere client offers several views — Hosts and Clusters, VMs and Templates, Datastores and Networking, however for the sake of simplicity I'm going to focus on the VMs and Templates view, like the one you can see on the right. You can easily modify my script to get the others.

Routine

Let us start with a clean script that connects to your vCenter or ESXi host. The only difference from Perl SDK sample scripts is that I'll be using Perl 5.16 features for case-insensitive sorting of the resulting tree.

#!/usr/bin/perl -w

use v5.16;
use strict;
use warnings;
use VMware::VIRuntime;

Opts::parse();
Opts::validate();
Util::connect();

Next, we have to specify a list of objects and properties we are interested in and fetch them from vSphere. As we are building a tree of VMs (template is just a form of VM) and folders, it is rather straightforward. We are only interested in the name property and the parent property, which will eventually help us build a tree from flat array.

The only glitch here is that we also need to include Datacenter entities, as they are integral part of the tree. There is no real hierarchy yet. Folders can contain datacenters and vice versa, which is why we can simply merge all entities into a single array.

# objects and properties required for tree assembly
my %vmandfoldertree = (
    Datacenter     => ['name', 'parent'],
    Folder         => ['name', 'parent'],
    VirtualMachine => ['name', 'parent'],
);

# obtain object references from vSphere and merge them into a single array
my @all_views = ();
while (my ($type, $props) = each (%vmandfoldertree)) {
    my $views = Vim::find_entity_views(view_type => $type, properties => $props) 
        || die "Failed to obtain $type list";
    push (@all_views, @$views);
}

The while structure isn't really necessary here, however it will enable you to create different inventory trees simply by modifying the hash of required objects.

What does a view actually look like? Well, we can simply dump one and take a look.

$VAR1 = bless( {
                 'mo_ref' => bless( {
                                      'value' => 'datacenter-2',
                                      'type' => 'Datacenter'
                                    }, 'ManagedObjectReference' ),
                 'parent' => bless( {
                                    'value' => 'group-d1',
                                    'type' => 'Folder'
                                  }, 'ManagedObjectReference' ),
                 'name' => 'Datacenter-lab'
               }, 'Datacenter' );

It is a managed entity of type Datacenter, which contains a name and references to its parent and itself. Folders and VMs are very similar. Entity's type can be determined simply from its mo_ref->type or using Perl's ref function.

Tree growing

We have a flat array of all required entities. As a first step of creating a tree we are going to use the parent references as keys to map our array into a hash. It is possible to use Perl's map function, but I find a simple foreach loop more readable.

# create a map of parentref -> [children], if there is no parent, use 'root' string
my %view_map = ();
push (@{$view_map{($_->parent) ? $_->parent->value : 'root'}}, $_) foreach (@all_views);

You may notice some folders like datastore or network which do not appear in vSphere client. These will always be empty, unless you fetch datastores and networks from vSphere as well. So it makes sense to exclude them from our tree. Also, it would be prettier to have the output sorted. Let us alter the foreach loop like this:

# create a map of parentref -> [children], if there is no parent, use 'root' string
my %view_map = ();
foreach my $view (sort {fc($a->name) cmp fc($b->name)} @all_views) {
    unless ($view->name ~~ ['network', 'datastore', 'host']) {
        push (@{$view_map{($view->parent) ? $view->parent->value : 'root'}}, $view);
    }
}

We now have a Perl hash with parent references as keys and arrays of children as values. The only thing left is to walk through the hash and recursively match grandchildren with each child, starting with the root entity.

sub maketree {
    my @results = ();

    # recursively build tree for all children
    push @results, {
        name     => $_->name,
        type     => $_->get_property('mo_ref.type'),
        mo_ref   => $_->get_property('mo_ref.value'),
        children => maketree($view_map{$_->get_property('mo_ref.value')}),
    } foreach (@{$_[0]});

    return \@results;
}

my $inventory_tree = maketree($view_map{'root'});

And that's it. Resulting $inventory_tree variable is an array reference, but under normal circumstances, there should be only one root. The resulting tree can be easily converted to JSON and used in any application or stored in memcached.

Performance

This script is capable of fetching an entire inventory, however that comes with a price. Personally, I heavily rely on caching and only call this script once in a while to refresh the cache. This way, your frontend will work even when there is some glitch in vSphere. In my environment it takes about 50 seconds to fetch 23000 items and sort them in a tree. On the other hand, it will process a single ESXi host in a couple seconds.

If your inventory is too large, it should be possible to add some filtering to Vim::find_entity_views and only limit the scope to useful items. But in my opinion users can't be bothered to wait even a second and caching is the way to go.

Have fun with the result. You can see my lab inventory fetched into a JSON and then displayed in my browser using zTree jQuery plugin.