Allocation V2¶

Allocation V2 is a new framework, similar to allocation (v1). It happens in the same lifecycle stage and serves the same purpose: filling up read-only values of a service instance.

It comes to fill some gaps in the functionalities of allocation (v1) and takes advantages of the experience and learnings that using allocation (v1) taught us. It is a more complete, functional, and elegant framework.

Example¶

The example below show you the use case where a single allocator is used the same way on both the service instance and an embedded entity.

main.cf¶

import lsm
import lsm::fsm


entity ValueService extends lsm::ServiceEntity:
    string                      name
    lsm::attribute_modifier     name__modifier="rw"

    int?                        first_value
    lsm::attribute_modifier     first_value__modifier="r"
end
ValueService.embedded_values [0:] -- EmbeddedValue

entity EmbeddedValue extends lsm::EmbeddedEntity:
    string                      id
    lsm::attribute_modifier     id__modifier="rw"

    int?                        third_value
    lsm::attribute_modifier     third_value__modifier="r"

    string[]? __lsm_key_attributes = ["id"]
end

index EmbeddedValue(id)

implement ValueService using parents
implement EmbeddedValue using std::none

binding = lsm::ServiceEntityBinding(
    service_entity="__config__::ValueService",
    lifecycle=lsm::fsm::simple,
    service_entity_name="value-service",
    allocation_spec="value_allocation",
    strict_modifier_enforcement=true,
)

for assignment in lsm::all(binding):
    attributes = assignment["attributes"]

    service = ValueService(
        instance_id=assignment["id"],
        entity_binding=binding,
        name=attributes["name"],
        first_value=attributes["first_value"],
    )

    for embedded_value in attributes["embedded_values"]:
        service.embedded_values += EmbeddedValue(
            **embedded_value
        )
    end
end

plugins/__init__.py¶

"""
    Inmanta LSM

    :copyright: 2022 Inmanta
    :contact: code@inmanta.com
    :license: Inmanta EULA
"""

from inmanta.util import dict_path
from inmanta_plugins.lsm.allocation import AllocationSpecV2
from inmanta_plugins.lsm.allocation_v2.framework import AllocatorV2, ContextV2, ForEach


class IntegerAllocator(AllocatorV2):
    def __init__(self, value: int, attribute: str) -> None:
        self.value = value
        self.attribute = dict_path.to_path(attribute)

    def needs_allocation(self, context: ContextV2) -> bool:
        try:
            if not context.get_instance().get(self.attribute):
                # Attribute not present
                return True
        except IndexError:
            return True
        return False

    def allocate(self, context: ContextV2) -> None:
        context.set_value(self.attribute, self.value)


AllocationSpecV2(
    "value_allocation",
    IntegerAllocator(value=1, attribute="first_value"),
    ForEach(
        item="item",
        in_list="embedded_values",
        identified_by="id",
        apply=[
            IntegerAllocator(
                value=3,
                attribute="third_value",
            ),
        ],
    ),
)

Allocation V2 features¶

The two main additions to allocation v2 when compared to v1 are:

The new ContextV2 object (replacement for AllocationContext object), which goes in pair with AllocatorV2 and AllocationSpecV2
The support for allocating attributes in embedded entities.

Setting a read-only attribute on an embedded entity, like done in the above-mentioned example, is only possible when strict_modifier_enforcement is enabled. On legacy services, where strict_modifier_enforcement is not enabled, read-only attributes can be set on embedded entities using the workaround mentioned the Section Legacy: Set attributes on embedded entities.

Warning

To use allocation safely, allocators should not keep any state between invocations, but pass all state via the ContextV2 object.

ContextV2¶

A context object that will be passed to each allocator and that should be used to set values. This context always shows the attributes the allocator should have access to, based on its level in the allocators tree. This means a top level allocator will see all the attributes, but an allocator used on embedded entities will only see the attributes of such embedded entity (as if it was a standalone entity). The context object can also be used to store values at each “level of allocation”, reachable by all allocators at the same level.

class inmanta_plugins.lsm.allocation_v2.framework.ContextV2(instance: ServiceInstance, plugin_context: Context)[source]¶

All interactions between allocators happen via this context.

The context is responsible for

exposing the current instance to the allocator
allow the allocator to set values
allow allocators to be used in the same way on embedded instances as on top-level instances
carry shared state between allocators (such as connections to the inventory)

Each Context exposes a ServiceInstance via the get_instance() method. It exposes this instance as if it is the service itself. I.e. embedded entities are presented in the same way as top level services.

To allow the context to carry additional state (such as a connection to an inventory), make a subclass of this class.

Variables:

plugin_context – The plugin context attached to this context
instance – The instance this context is attached to
flush_set – A dict containing a trace of all the set_value call that where made on this context object. The key is the :param path: that was passed to the method The value is the :param value: that was passed to the method

build_child(path: DictPath, wild_path: WildDictPath) → ChildContextV2[source]¶

Produce a new child instance of this context

(this method acts as a polymorphic constructor)

Parameters:

path – the path for the new context, relative to the parent
wild_path – the path for the new context, or any context at the same level, relative to the parent

classmethod clone(instance: ServiceInstance, plugin_context: Context) → TC[source]¶

Produce a new instance of this context

(this method acts as a polymorphic constructor)

Parameters:: instance – the instance this new context should be attached to

property environment: str¶: Return the environment id

get_child(relation: str) → ChildContextV2[source]¶

Not intended for external usage

Idempotent method

Get a context for the child that matches the path relation

get_full_path() → DictPath | None[source]¶: Get the full path up to the instance in this context. The root context should return None as there is no path required to reach it.

get_instance() → ServiceInstance[source]¶

Returns:: Get the service instance bound to this context

get_list_children(relation: str, key_attribute: str) → List[ChildContextV2][source]¶

Not intended for external usage

Idempotent method

Get a context for every child that matches the path relation[key_attribute=*] in the attributes set of the instance

get_wild_full_path() → WildDictPath | None[source]¶: Get the full path up to the instance in this context, replacing all KeyedList by WildKeyedList. The root context should return None as there is no path required to reach it.

set_value(path: DictPath, value: object) → None[source]¶

Set the value at the given path and make sure this allocation is written back to the LSM database.

This call will update the ServiceInstance and the LSM database.

The flushing to database can either be when allocation succeeds, when it fails or immediately. This behavior is determined by the context.

Parameters:

path – the path at which to set the value
value – the value to set at the given path

In the example at the beginning of this page, the same allocator can be used to set a value on the service entity and an embedded entity. In needs_allocation, when calling context.get_instance(), we receive as dict the full service entity when allocating first_value and the embedded entity when allocating third_value.

AllocatorV2¶

A base class for all v2 allocators, they are provided with a ContextV2 object for those two methods: needs_allocation and allocate. The main difference with v1, is that the allocate method doesn’t return any value to allocate, it sets them using the context object: context.set_value(name, value).

class inmanta_plugins.lsm.allocation_v2.framework.AllocatorV2[source]¶

Base class for all V2 Allocators

abstract allocate(context: ContextV2) → None[source]¶

Perform the actual allocation

Allocated values will be written back to the context using ContextV2.set_value()

abstract needs_allocation(context: ContextV2) → bool[source]¶: Determine if this allocator has any work to do or if all values have already been allocated correctly.

post_allocate() → None[source]¶: Called after allocation, can be used e.g., to teardown a connection to the external inventory

pre_allocate() → None[source]¶: Called before allocation, can be used e.g., to establish a connection to the external inventory

AllocationSpecV2¶

The collector for all AllocatorV2.

class inmanta_plugins.lsm.allocation.AllocationSpecV2(name: str, *allocators: AllocatorV2)[source]¶

__init__(name: str, *allocators: AllocatorV2) → None[source]¶

Create the allocation spec and register it in the global collector

Parameters:

name – The name of the allocator, must be unique
allocators – The allocators contained in this aggregator

Legacy: Set attributes on embedded entities¶

The server doesn’t have support to set read-only attributes on embedded entities when strict_modifier_enforcement is disabled. Thanks to the allocator ContextV2Wrapper and the plugin lsm::context_v2_unwrapper a workaround exists to do allocation on an embedded entity’s attributes with strict_modifier_enforcement disabled. This workaround saves all the allocated values in a dict, in an attribute of the service instance (added to the instance for this single purpose). That way, the server accepts the update.

class inmanta_plugins.lsm.allocation_v2.framework.ContextV2Wrapper(fallback_attribute: str, *allocators: AllocatorV2, track_deletes: bool = False)[source]¶

This wrapper allows to use allocators for attributes of embedded entities. It performs the allocation for all the provided allocators, and save/allocate all of their allocated values in a single key.

This wrapper can be used in pair with the context_v2_unwrapper plugin, which allows to place all those allocated values back where they are supposed to go.

__init__(fallback_attribute: str, *allocators: AllocatorV2, track_deletes: bool = False) → None[source]¶

This wrapper allows to use allocators for attributes of embedded entities. It will condense all the changes in a dict: - whose keys are dict_path to the attribute of the embedded entities - whose values are the one to allocate

This dict is itself saved at the root of the changed attributes dict, saved in the fallback attributes specified in the constructor.

Parameters:

fallback_attributes – The attribute in which all the allocated values should be saved
allocators – The allocators whose values should be saved
track_deletes – when this is set, when an embedded entity is deleted during an update, it will also be dropped from the fallback_attribute. This also implies that embedded entities which are created by allocation alone (all attributes of type ‘r’) are not supported.

e.g:

lsm.AllocationSpecV2(
    "service_allocator_spec",
    allocation.ContextV2Wrapper(
        "allocated",
        allocation.ForEach(
            item="subservice",
            in_list="subservices",
            identified_by="id",
            apply=[
                ValueAllocator(
                    into="to_allocate",
                ),
            ],
        ),
    ),
)

This would generate as update attribute dict something like

{
    "allocated": {
        "subservices[id=1].to_allocate": "a",
        "subservices[id=2].to_allocate": "b",
    }
}

allocate(context: ContextV2) → None[source]¶: Run allocation for all the children, then aggregate all the allocated values into a single value, in the parent context.

needs_allocation(context: ContextV2) → bool[source]¶: Needs allocation if any of the children allocators need allocation.

inmanta_plugins.lsm.context_v2_unwrapper(assignments: dict[], fallback_attribute: string, track_deletes: bool = False) → dict[][source]¶

This plugin can be used to wrap the instances coming out of lsm::all and place all allocated values in :param fallback_attribute: where they should go. The returned value is what has been given as input, except for the allocated values being set where they should. :param track_deletes: drop deleted embedded entities, even if they still exist in the fallback set.

This should be used together with ContextV2Wrapper allocator.

Each assignment is an attribute dict containing a fallback attribute assigned with allocated values as produced by the ContextV2 (one-level deep, keys are string representation of the paths, values are allocated values) and update the dict placing values where their key-path would reach them.

Parameters:

assignments – The list of service instance dictionaries as returned by lsm::all
fallback_attributes – The attribute name at the root of the instance attributes that contains all the allocated values

e.g.:

context_v2_unwrapper(
    [
        {
            "environment": "8f7bf3a5-d655-4bcb-bbd4-6222407be999",
            "id": "f93acfad-7894-4a12-9770-b27cbdd85c74",
            "service_entity": "carrierEthernetEvc",
            "version": 4,
            "config": {},
            "state": "allocating",
            "attributes": {
                "allocated": {
                    "evcEndPoints[identifier=my-evc-ep-1].uni": {
                        "uniref": "inmanta:456-852-789",
                        "test_value": "test_value",
                    },
                    "evcEndPoints[identifier=my-evc-ep-2].uni": {
                        "uniref": "inmanta:123-852-456",
                        "test_value": "test_value",
                    },
                },
                "another_key": "any value",
            },
            "candidate_attributes": {
                "allocated": {
                    "evcEndPoints[identifier=my-evc-ep-1].uni": {
                        "uniref": "inmanta:456-852-789",
                        "test_value": "test_value",
                    },
                    "evcEndPoints[identifier=my-evc-ep-2].uni": {
                        "uniref": "inmanta:123-852-456",
                        "test_value": "test_value",
                    },
                },
                "another_key": "any value",
            },
            "active_attributes": {},
            "rollback_attributes": {},
        }
    ],
    "allocated",
)

will return:

[
    {
        "environment": "8f7bf3a5-d655-4bcb-bbd4-6222407be999",
        "id": "f93acfad-7894-4a12-9770-b27cbdd85c74",
        "service_entity": "carrierEthernetEvc",
        "version": 4,
        "config": {},
        "state": "allocating",
        "attributes": {
            "allocated": {
                "evcEndPoints[identifier=my-evc-ep-1].uni": {
                    "uniref": "inmanta:456-852-789",
                    "test_value": "test_value",
                },
                "evcEndPoints[identifier=my-evc-ep-2].uni": {
                    "uniref": "inmanta:123-852-456",
                    "test_value": "test_value",
                },
            },
            "evcEndPoints": [
                {
                    "identifier": "my-evc-ep-1",
                    "uni": {
                        "uniref": "inmanta:456-852-789",
                        "test_value": "test_value",
                    },
                },
                {
                    "identifier": "my-evc-ep-2",
                    "uni": {
                        "uniref": "inmanta:123-852-456",
                        "test_value": "test_value",
                    },
                },
            ],
            "another_key": "any value",
        },
        "candidate_attributes": {
            "allocated": {
                "evcEndPoints[identifier=my-evc-ep-1].uni": {
                    "uniref": "inmanta:456-852-789",
                    "test_value": "test_value",
                },
                "evcEndPoints[identifier=my-evc-ep-2].uni": {
                    "uniref": "inmanta:123-852-456",
                    "test_value": "test_value",
                },
            },
            "evcEndPoints": [
                {
                    "identifier": "my-evc-ep-1",
                    "uni": {
                        "uniref": "inmanta:456-852-789",
                        "test_value": "test_value",
                    },
                },
                {
                    "identifier": "my-evc-ep-2",
                    "uni": {
                        "uniref": "inmanta:123-852-456",
                        "test_value": "test_value",
                    },
                },
            ],
            "another_key": "any value",
        },
        "active_attributes": {},
        "rollback_attributes": {},
    }

]

The ContextV2Wrapper, which has to be used at the root of the allocation tree, will collect and save all the allocated value in a single dict. And when getting all the service instances in your model, with lsm::all, you can simply wrap the call to lsm::all with a call to lsm::context_v2_unwrapper, which will place all the allocated values saved in the dict, directly where they belong, in the embedded entities.

When using the ContextV2Wrapper and the lsm::context_v2_unwrapper plugin, you will have to specify in which attributes all the allocated values should be saved.

main.cf¶

import lsm
import lsm::fsm


entity ValueService extends lsm::ServiceEntity:
    string                      name
    lsm::attribute_modifier     name__modifier="rw"

    int?                        first_value
    lsm::attribute_modifier     first_value__modifier="r"

    dict?                       allocated
    lsm::attribute_modifier     allocated__modifier="r"
end
ValueService.embedded_values [0:] -- EmbeddedValue

entity EmbeddedValue extends lsm::EmbeddedEntity:
    string                      id
    lsm::attribute_modifier     id__modifier="rw"

    int?                        third_value
    lsm::attribute_modifier     third_value__modifier="r"
end

implement ValueService using parents
implement EmbeddedValue using std::none

binding = lsm::ServiceEntityBinding(
    service_entity="__config__::ValueService",
    lifecycle=lsm::fsm::simple,
    service_entity_name="value-service",
    allocation_spec="value_allocation",
)

for assignment in lsm::context_v2_unwrapper(
    assignments=lsm::all(binding),
    fallback_attribute="allocated",
):
    attributes = assignment["attributes"]

    service = ValueService(
        instance_id=assignment["id"],
        entity_binding=binding,
        name=attributes["name"],
        first_value=attributes["first_value"],
        allocated=attributes["allocated"],
    )

    for embedded_value in attributes["embedded_values"]:
        service.embedded_values += EmbeddedValue(
            **embedded_value
        )
    end
end

plugins/__init__.py¶

"""
    Inmanta LSM

    :copyright: 2022 Inmanta
    :contact: code@inmanta.com
    :license: Inmanta EULA
"""

from inmanta.util import dict_path
from inmanta_plugins.lsm.allocation import AllocationSpecV2
from inmanta_plugins.lsm.allocation_v2.framework import (
    AllocatorV2,
    ContextV2,
    ContextV2Wrapper,
    ForEach,
)


class IntegerAllocator(AllocatorV2):
    def __init__(self, value: int, attribute: str) -> None:
        self.value = value
        self.attribute = dict_path.to_path(attribute)

    def needs_allocation(self, context: ContextV2) -> bool:
        try:
            if not context.get_instance().get(self.attribute):
                # Attribute not present
                return True
        except IndexError:
            return True
        return False

    def allocate(self, context: ContextV2) -> None:
        context.set_value(self.attribute, self.value)


AllocationSpecV2(
    "value_allocation",
    IntegerAllocator(value=1, attribute="first_value"),
    ContextV2Wrapper(
        "allocated",
        ForEach(
            item="item",
            in_list="embedded_values",
            identified_by="id",
            apply=[
                IntegerAllocator(
                    value=3,
                    attribute="third_value",
                ),
            ],
        ),
    ),
)

To facilitate allocation on embedded entities, the ForEach allocator can be used.

class inmanta_plugins.lsm.allocation_v2.framework.ForEach(item: str, in_list: str, identified_by: str, apply: List[AllocatorV2])[source]¶

Allocator to apply a given set of allocators to each instance in a list of instances.

The instances are uniquely identified by an attribute

__init__(item: str, in_list: str, identified_by: str, apply: List[AllocatorV2]) → None[source]¶

Parameters:

item – name of the loop variable (not used, to improve readability and error reporting)
in_list – the relation from which to get the items
identified_by – the identifying attribute of each item
apply – the allocators to apply to each item

e.g:

entity Service extends lsm::ServiceInstance:
end

entity SubService extends lsm::EmbeddedEntity:
    string id
    string to_allocate
    lsm::attribute_modifier to_allocate__modifier="r"
end

Service.subservices [0:] -- SubService

lsm.AllocationSpecV2(
    "service_allocator_spec",
    allocation.ContextV2Wrapper(
        "allocated",
        allocation.ForEach(
            item="subservice",
            in_list="subservices",
            identified_by="id",
            apply=[
                ValueAllocator(
                    into="to_allocate",
                ),
            ],
        ),
    ),
)

allocate(context: ContextV2) → None[source]¶

Perform the actual allocation

Allocated values will be written back to the context using ContextV2.set_value()

needs_allocation(context: ContextV2) → bool[source]¶: Determine if this allocator has any work to do or if all values have already been allocated correctly.

Deleting of Embedded entities¶

When you want to support deletion of embedded entities during updates, a slightly different configuration is needed. Because all allocated values are stored in a single attribute, the deleted entities will be recreated when unwrapping.

To prevent this, use track_deletes=true on both the the allocator ContextV2Wrapper and the plugin lsm::context_v2_unwrapper

Additionally, to re-trigger allocation when an item is deleted, use SetSensitiveForEach instead of ForEach.

main.cf¶

import lsm
import lsm::fsm


entity ValueService extends lsm::ServiceEntity:
    string                      name
    lsm::attribute_modifier     name__modifier="rw"

    int?                        first_value
    lsm::attribute_modifier     first_value__modifier="r"

    dict?                       allocated
    lsm::attribute_modifier     allocated__modifier="r"
end
ValueService.embedded_values [0:] lsm::__rwplus__ EmbeddedValue

entity EmbeddedValue extends lsm::EmbeddedEntity:
    string                      id
    lsm::attribute_modifier     id__modifier="rw"

    int?                        third_value
    lsm::attribute_modifier     third_value__modifier="r"

    string                      other_value
    lsm::attribute_modifier     other_value__modifier="rw"
end

index EmbeddedValue(id)

implement ValueService using parents
implement EmbeddedValue using std::none

binding = lsm::ServiceEntityBindingV2(
    service_entity="__config__::ValueService",
    lifecycle=lsm::fsm::simple,
    service_entity_name="value-service",
    allocation_spec="value_allocation",
)

for assignment in lsm::context_v2_unwrapper(
    assignments=lsm::all(binding),
    fallback_attribute="allocated",
    track_deletes=true,
):
    attributes = assignment["attributes"]

    service = ValueService(
        instance_id=assignment["id"],
        entity_binding=binding,
        name=attributes["name"],
        first_value=attributes["first_value"],
        allocated=attributes["allocated"],
    )

    for embedded_value in attributes["embedded_values"]:
        service.embedded_values += EmbeddedValue(
            **embedded_value
        )
    end
end

plugins/__init__.py¶

"""
    Inmanta LSM

    :copyright: 2020 Inmanta
    :contact: code@inmanta.com
    :license: Inmanta EULA
"""

from inmanta.util import dict_path
from inmanta_plugins.lsm.allocation import AllocationSpecV2
from inmanta_plugins.lsm.allocation_v2.framework import (
    AllocatorV2,
    ContextV2,
    ContextV2Wrapper,
    SetSensitiveForEach,
)


class IntegerAllocator(AllocatorV2):
    def __init__(self, value: int, attribute: str) -> None:
        self.value = value
        self.attribute = dict_path.to_path(attribute)

    def needs_allocation(self, context: ContextV2) -> bool:
        try:
            if not context.get_instance().get(self.attribute):
                # Attribute not present
                return True
        except IndexError:
            return True
        return False

    def allocate(self, context: ContextV2) -> None:
        if self.needs_allocation(context):
            context.set_value(self.attribute, self.value)


AllocationSpecV2(
    "value_allocation",
    IntegerAllocator(value=1, attribute="first_value"),
    ContextV2Wrapper(
        "allocated",
        SetSensitiveForEach(
            item="item",
            in_list="embedded_values",
            identified_by="id",
            apply=[
                IntegerAllocator(
                    value=3,
                    attribute="third_value",
                ),
            ],
        ),
        track_deletes=True,
    ),
)