Tuesday, February 19, 2013

Exploring VMware vSphere PowerCLI with F#

I was recently asked to vet some VMware cluster capacity numbers. It seemed like a task that might be repeated in the future and I really hate to manually transcribe the data and compare them. So that means I need to write a script to automate it. Fortunately, VMware has vSphere PowerCLI for this job. Unfortunately, the documentation for it was rather sparse. I also looked at the book VMware vSphere PowerCLI Reference: Automating vSphere Administration, but really did not want to script in PowerShell simply because I'm not that familiar with PowerShell. However, PowerCLI is accessible from .NET also, so I can write my script in F#.

The following lines of script simply drills down from Datacenter object to the VirtualMachine object. Once you grab all the objects, you can explore the properties of each object.

#r @"C:\pkg\VMware\Infrastructure\vSphere PowerCLI\VMware.Vim.dll"

open System
open VMware.Vim
open System.Collections.Specialized

let serviceUrl = "https://myVSphereHost/sdk"
let userId = "someUserId"
let password="somePassword"

let client = new VimClient()
let service = client.Connect(serviceUrl)

// Must login to do anything - if you are getting null values, it means the session automatically timed out

// Let us get all the datacenters
let dataCenters = client.FindEntityViews(typeof<Datacenter>,null,null,null)

// Drill down into the first datacenter
let dc = dataCenters |> Seq.cast<Datacenter> |> Seq.head

// Get a cluster
let cluster = client.GetView(dc.Parent,null) :?> ClusterComputeResource

// Get the first host in the cluster
let host = client.GetView(cluster.Host |> Seq.head,null) :?> HostSystem

// Get the first VM on the physical host
let vm = client.GetView(host.Vm |> Seq.head,null) :?> VirtualMachine

Let's do something interesting with PowerCLI. In the following scripts, I wanted to grab the capacity information at the VMware cluster level and the combine allocation/utilization info of all the virtual machines hosted on the VMware cluster.

// Utility function to help use get vSphere Entities
let getEntityViews viewType searchParams =
    match searchParams with
    | Some(searchParams0) ->
        let filters = new NameValueCollection()
        searchParams0 |> Seq.iter (fun (k,v) -> filters.Add(k,v))

    | None ->

// Get vSphere Entities with specific properties (reduces returned data)
// Don't know how to define a function with multiple arities in F# - clumsy workaround
let getEntityViews2 viewType searchParams props =
    match searchParams with
    | Some(searchParams0) ->
        let filters = new NameValueCollection()
        searchParams0 |> Seq.iter (fun (k,v) -> filters.Add(k,v))
    | None ->

// Get Cluster usage summary!
let getUsageSummary clusterName =

    let clusterProps = [|"Summary"; "Host"|]
    let hostProps = [|"Vm";"Name";"Hardware";"Runtime"|]
    let vmProps = [|"Name";"Config";"Runtime";"Summary";"ResourceConfig"|]

    let toMB memoryInBytes = memoryInBytes / (1024L*1024L)

    // Get cluster - Expect only one result
    printfn "Getting cluster data..."
    let cluster = 
        let filters = Some([("name",clusterName)])
        getEntityViews2 typeof<ClusterComputeResource> filters clusterProps
            |> Seq.cast<ClusterComputeResource>
            |> Seq.head

    // Get all Hosts for this cluster
    printfn "Getting host list..."
    let hostList =
        |> Seq.map(fun moRef -> client.GetView(moRef,hostProps))
        |> Seq.cast<HostSystem>
        |> Seq.cache

    printfn "Getting VM list..."
    let vmList =
        |> Seq.map (fun host -> host.Vm)
        |> Seq.concat
        |> Seq.map (fun moRef -> client.GetView(moRef,vmProps))
        |> Seq.cast<VirtualMachine>
        |> Seq.cache

    let clusterCores = cluster.Summary.NumCpuCores
    let clusterCPU = cluster.Summary.TotalCpu
    let clusterMemory = cluster.Summary.TotalMemory

    // Utility function to get summation results from selected fields
    let inline total (extractor:VirtualMachine -> Nullable<'b>) =
        |> Seq.map extractor
        |> Seq.map (fun x -> x.GetValueOrDefault())
        |> Seq.sum

    printfn "Getting Cluster Summary Info..."
    let cpuUsed        = total (fun vm -> vm.Summary.Config.NumCpu)
    let memoryUsed     = total (fun vm -> vm.Summary.Config.MemorySizeMB)
    let cpuReserved    = total (fun vm -> vm.Summary.Config.CpuReservation)
    let memoryReserved = total (fun vm -> vm.Summary.Config.MemoryReservation)
    let maxMemoryUsage = total (fun vm -> vm.Runtime.MaxMemoryUsage)
    let maxCpuUsage    = total (fun vm -> vm.Runtime.MaxCpuUsage)

    let cpuReservation    = total (fun vm -> vm.ResourceConfig.CpuAllocation.Reservation) 
    let cpuLimit          = total (fun vm -> vm.ResourceConfig.CpuAllocation.Limit) 
    let memoryReservation = total (fun vm -> vm.ResourceConfig.MemoryAllocation.Reservation)
    let memoryLimit       = total (fun vm -> vm.ResourceConfig.MemoryAllocation.Limit) 

    printfn "Cluster Name               : %s" clusterName
    printfn "Number of Hosts in Cluster : %i" (Seq.length hostList)
    printfn "Number of VMs in Cluster   : %i" (Seq.length vmList)
    printfn "Cluster Total Cores        : %i" clusterCores
    printfn "Cluster Total CPU (MHz)    : %i" clusterCPU
    printfn "Cluster Total Memory       : %i" clusterMemory
    printfn "Cluster Total Memory (MB)  : %i" (toMB clusterMemory)
    printfn "CPU Used by VMs            : %i" cpuUsed
    printfn "Memory Used by VMs         : %i" memoryUsed
    printfn "CPU Reserved by VMs        : %i" cpuReserved
    printfn "Memory Reserved by VMs     : %i" memoryReserved
    printfn "Max Memory Usage by VMs    : %i" maxMemoryUsage
    printfn "Max CPU Usage by VMs (MHz) : %i" maxCpuUsage
    printfn "Total Allocated CPU Reservations    : %i" cpuReservation
    printfn "Total Allocated CPU Limits          : %i" cpuLimit
    printfn "Total Allocated Memory Reservations : %i" memoryReservation
    printfn "Total Allocated Memory Limits       : %i" memoryLimit
    printfn "Memory Used / Cluster Total Memory  : %f" (double(memoryUsed) / double(toMB clusterMemory))
    printfn "vCPU Allocated / Cluster Total Cores : %f" (float(cpuUsed) / float(clusterCores))    
    printfn "Done!"

// Invoke getUsageSummary to get the summary info on my cluster
getUsageSummary "MyTestCluster"

Sample results:

Number of Hosts in Cluster : 5
Number of VMs in Cluster   : 10
Cluster Total Cores        : 120
Cluster Total CPU (MHz)    : 276000
Cluster Total Memory       : 1374369136640
Cluster Total Memory (MB)  : 1310700
CPU Used by VMs            : 38
Memory Used by VMs         : 253952
CPU Reserved by VMs        : 0
Memory Reserved by VMs     : 0
Max Memory Usage by VMs    : 253952
Max CPU Usage by VMs (MHz) : 87400
Total Allocated CPU Reservations    : 0
Total Allocated CPU Limits          : -10
Total Allocated Memory Reservations : 0
Total Allocated Memory Limits       : -10
Memory Used / Cluster Total Memory  : 0.193753
vCPU Allocated / Cluster Total Cores : 0.316667

I wish VMware had PowerCLI class documentation similar to those for the .NET library hosted on MSDN. If VMware does have those documentation, I can't seem to find them. Lack of documentation has forced me to interactively explore the PowerCLI with F#. Thankfully, I can do this in F# and shudder at the thought of exploring PowerCLI in C#.


Blogger said...

Bluehost is one of the best hosting provider for any hosting services you might need.

Nannie Co Pam said...

Very much useful article. Kindly keep blogging

Java Training in Chennai

Java Online Training India