avi Module

Inmanta adapter for the AVI / VMware NSX Advanced Load Balancer controller, built on top of restbase.

The DSL model - the resource entities and their attributes - is generated from the controller’s own API schema, so an entity carries every field AVI documents for that object type. Only a thin layer is hand written: the session client, the registration token plugin and the CRUD handler. Of the model, only model/base.cf (the credentials and the base resource) is. Objects are read by name and created, updated and deleted by uuid; an update overlays the fields the model sets on the object currently on the controller, so fields nobody manages are left alone.

Scope

Feature	Entity / plugin
Tenant	avi::Tenant
Service engine group	avi::ServiceEngineGroup
Virtual service VIP	avi::VsVip
VRF context (route domain), BGP peers, static routes	avi::VrfContext
Service engine registration token	avi::create_token_reference (a Reference[str]) / avi::generate_token
Service engine (wait for / SE group / remove)	avi::service_engine::ServiceEngine
Service engine data interface (VRF context)	avi::service_engine::Interface
Service engine image (download)	avi::se_image::ServiceEngineImage

Configuration

The model reads no environment variables. Everything a resource needs comes from the avi::base::Credentials instance it is given: the controller URL, the credentials, the API version and the default tenant. The agent a resource is deployed on is derived from the controller host, so no std::AgentConfig is required. (AVI_URL is only used when regenerating the model, see below.)

Every resource is scoped to a tenant through tenant_scope (default admin). To manage an object inside another tenant, set it and add a requires on the avi::Tenant, which orders the two.

Example

import avi
import avi::base
import avi::service_engine
import avi::tenant::definitions
import avi::vrf_context::definitions
import avi::vs_vip::definitions

credentials = avi::base::Credentials(
    host="https://172.25.223.252",
    username="admin",
    password="secret",
)

tenant = avi::Tenant(
    credentials=credentials,
    name="my-tenant",
    local=true,
    # Give the tenant its own routing domain, so it can hold VRF contexts.
    config_settings=avi::tenant::definitions::TenantConfiguration(tenant_vrf=true),
)

seg = avi::ServiceEngineGroup(
    credentials=credentials,
    name="my-seg",
    tenant_scope="my-tenant",   # X-Avi-Tenant scope for this object
    max_se=4,
    min_scaleout_per_vs=2,
    ha_mode="HA_MODE_SHARED_PAIR",
    requires=[tenant],          # the group is created after the tenant
)

# Nested objects (here the vip list and its address) are generated entities too.
avi::VsVip(
    credentials=credentials,
    name="my-vip",
    tenant_scope="my-tenant",
    vip=[avi::vs_vip::definitions::Vip(
        vip_id="1",
        enabled=true,
        ip_address=avi::vs_vip::definitions::IpAddr(addr="42.42.42.42", type="V4"),
    )],
    requires=[tenant],
)

avi::VrfContext(
    credentials=credentials,
    name="my-vrf",
    tenant_scope="my-tenant",
    cloud="Default-Cloud",
    bgp_profile=avi::vrf_context::definitions::BgpProfile(
        ibgp=false,             # eBGP
        local_as=64979,
        num_as_path_prepend=2,
        community=["64979:19200", "64979:20"],
        send_community=true,
        # One peer per anchor leaf of the ACI floating l3out.
        peers=[avi::vrf_context::definitions::BgpPeer(
            peer_ip=avi::vrf_context::definitions::IpAddr(
                addr="10.1.1.3", type="V4"),
            subnet=avi::vrf_context::definitions::IpAddrPrefix(
                ip_addr=avi::vrf_context::definitions::IpAddr(
                    addr="10.1.1.0", type="V4"),
                mask=29,
            ),
            remote_as=65001,
            md5_secret="s3cr3t",
            advertise_vip=true,
            label="aci",        # the routing options to apply to this peer
        )],
        # What the peers carrying that label learn and advertise. The four
        # booleans form two pairs that can not be mixed (see below).
        routing_options=[avi::vrf_context::definitions::BgpRoutingOptions(
            label="aci",
            learn_routes=true,
            advertise_default_route=true,
            max_learn_limit=100,
        )],
    ),
    requires=[tenant],
)

# The engine that registered itself with the token it booted with, moved into
# the service engine group of its service - the last step of the load balancer
# creation. Registration takes a while, so this waits for the engine to show up.
avi::service_engine::ServiceEngine(
    credentials=credentials,
    name="my-engine",           # identifies the resource
    engine_name="10.0.37.27",   # the engine as the controller names it
    tenant_scope="my-tenant",
    cloud="Default-Cloud",
    se_group_ref="/api/serviceenginegroup?name=my-seg",
    requires=[tenant, seg],     # the group it is moved into has to exist
)

# The data interface the cloud attached to a service engine, in that VRF. Only
# the mac address of the port identifies it; the engine and the interface both
# show up on the controller a while after the port was attached, so this waits.
avi::service_engine::Interface(
    credentials=credentials,
    engine="10.0.37.27",        # the engine as the controller names it
    tenant_scope="my-tenant",
    cloud="Default-Cloud",
    name="my-data-port",        # identifies the interface on that engine
    mac_address="fa:16:3e:cd:ad:e3",
    vrf="my-vrf",               # null puts it back in "my-tenant-default"
)

Things worth knowing per resource

• avi::Tenant takes the service engines registered in it along when it is purged. The controller refuses to delete a tenant anything still refers to (400 Cannot delete Tenant ... is referred by:  <engine>), and an engine is not created through the API - it registers itself into the tenant whose token it was given - so nothing guarantees the model holds one for every engine that ended up there. Purge the objects the model does hold before the tenant all the same: only the engines are cleaned up here.

• avi::VrfContext is scoped to a cloud, not only to a tenant: every cloud has VRF contexts of its own and names only have to be unique within one, so cloud is mandatory and part of the resource identity. The tenant needs config_settings.tenant_vrf set, otherwise the controller refuses the write. Purge a VRF after everything that references it (a VIP’s vrf_context_ref, a service engine’s data vnics) and before its tenant. Its bgp_profile.peers, its bgp_profile.routing_options and its static_routes are what a connectivity adds to a route domain: a peer is identified by peer_ip/peer_ip6, a routing options object by its label and a route by route_id, so all three can be added and removed one at a time without rewriting the rest. A peer’s md5_secret is written but never read back (the controller answers "<sensitive>"), so it is not compared: a secret that changes and nothing else is only written along with the next change to the VRF. A peer’s label picks the routing options that apply to it. Those four booleans form two pairs that can not be mixed: learn_routes goes with advertise_default_route, and learn_only_default_route with advertise_learned_routes. The controller refuses any other combination with 400 "<one> cannot be enabled with <other>", and it holds max_learn_limit to the 50-250 range.

  Known limitation: taking the last item of a managed list away does not clear it on the controller, and neither does dropping the bgp_profile instance: an empty relation is left out of the desired state altogether, so no change is detected, no request is sent and the resource reports success. Adding, changing and removing items while at least one remains is correct, as is creating a VRF with or without BGP; only the transition to nothing is not.

• avi::service_engine::ServiceEngine is not created through the API: the engine VM boots with a single use token and registers itself. The resource covers the two ends of that lifecycle, driven by purged - wait for the engine to show up (polling, which gates everything in its provides), or remove it from the controller. Order it after its VM in both directions, and before the service engine group it sits in. In between, the attributes the model sets on the engine are written to it, and se_group_ref is the one the load balancer creation needs: an engine registers into the default group of its cloud, and its own group is the last step. Point at that group the way a model points at any AVI object, by name (se_group_ref="/api/serviceenginegroup?name=my-seg"); the controller resolves it in the cloud the engine registered into and refuses a group that is not there, which is the scoping a group name needs - it is only unique within a cloud. Moving an engine between groups makes the controller deactivate and reactivate it, so it refuses to move an unlicensed engine (400 "Cannot perform deactivate/activate operation since SE is not licensed."). The engine is replaced as a whole on a write, so attributes nobody sets are left alone, and its name, its interfaces (data_vnics/mgmt_vnic, which have avi::service_engine::Interface resources of their own) and its resources are never written.

• avi::service_engine::Interface is one data interface of an engine, found by its mac_address, and the VRF context it belongs to. It is not created either - the cloud attaches a port to the engine’s VM and the controller discovers the interface - so the resource waits for it to show up and then configures it. vrf names the VRF context, looked up in the cloud the engine registered into; leaving it null (or purging the resource) puts the interface back in the tenant’s default VRF <tenant_scope>-default, which is where the controller puts a newly discovered one. The interface is a member of its engine, so writing it replaces the whole engine object: interfaces of one engine are written one at a time, and everything else on it is left alone.

• avi::se_image::ServiceEngineImage is a file the controller serves, not an API object. It downloads the image (1.8 GB) onto the agent for whatever uploads it to the hypervisor, re-downloads it when the controller serves another one, and resumes an interrupted download.

• avi::create_token_reference yields a registration token as a Reference[str] resolved on the agent, to feed to an SE VM as its AVICNTRL_AUTHTOKEN metadata. Tokens are single use and expire after an hour, so they are generated at deploy time rather than at compile time.

Regenerating the model from the controller

The entities under model/ and the cached schemas under files/schemas/ are generated. To refresh them against a controller:

pip install -e .[generator]
AVI_URL=https://172.25.223.252 make generate

This fetches the swagger document of each managed object type from <AVI_URL>/swagger/<ObjectType>.json (one document per type - the controller exposes no combined schema), caches it under files/schemas/ and rewrites the generated .cf files in place. Which object types those are is decided by the @generated_avi_resource classes in inmanta_plugins/avi/resources.py.

The same run emits the entities no swagger describes - how an object is found on the controller and how long it is waited for, the cloud a VRF context lives in, and the service engine image, which is a file rather than an API object. Those are declared with the same module factory in the last section of resources.py, so model/base.cf is the only model file that is not generated.

Running tests

pip install -e . -c requirements.txt -r requirements.dev.txt

pytest -m "not integration" tests   # offline
pytest -m integration tests         # live, against a controller

There is one test file per resource, each with offline tests and one live test_integration. The integration tests need the controller connection parameters, through environment variables or pytest arguments (see tests/conftest.py); CI provides them through .ci-integration-tests.yml.

• Module avi
• Changelog