Multi-version LSM

Multi-version lsm allows you to have multiple api versions for the same service.

Why use multi-version LSM?

You should use multi-version LSM when you want to:

  • Offer multiple api schema versions to the same service

  • Upgrade a service in a way that is not supported by the automated upgrade mechanism

Using multi-version LSM

Using the model to create InterfaceIPAssignments as an example, we will see how we can turn this entity into a versioned one. We just need to:

  • Change our existing lsm::ServiceEntityBinding into a lsm::ServiceBinding.

  • Set the default_version to 0.

  • Create a lsm::ServiceBindingVersion with the information that we had on our previous binding.

When unrolling using lsm::all, we use lsm::get_service_binding_version to fetch the correct entity binding version for each instance.

 1import lsm
 2import lsm::fsm
 3import ip
 4
 5entity InterfaceIPAssignment extends lsm::ServiceEntity:
 6    """
 7        Interface details.
 8
 9        :attr service_id: A unique ID for this service.
10
11        :attr router_ip: The IP address of the SR linux router that should be configured.
12        :attr router_name: The name of the SR linux router that should be configured.
13        :attr interface_name: The name of the interface of the router that should be configured.
14        :attr address: The IP-address to assign to the given interface.
15    """
16    string service_id
17
18    string router_ip
19    string router_name
20    string interface_name
21
22    string address
23    lsm::attribute_modifier address__modifier="rw+"
24
25end
26
27index InterfaceIPAssignment(service_id)
28
29implement InterfaceIPAssignment using parents
30
31binding = lsm::ServiceBinding(
32    service_entity_name="service_simple",
33    default_version=0,
34    versions=[
35        lsm::ServiceBindingVersion(
36            service_entity="__config__::InterfaceIPAssignment",
37            lifecycle=lsm::fsm::service,
38            version=0,
39        )
40    ]
41)
42
43for instance in lsm::all(binding):
44    InterfaceIPAssignment(
45        instance_id=instance["id"],
46        entity_binding=lsm::get_service_binding_version(binding, instance["service_entity_version"]),
47        **instance["attributes"],
48    )
49end

Adding or removing versions

To add a new version of our service we can either create a new entity (if we want to modify the attributes of a previously created version) or just use the same entity but with different binding attributes (i.e. different lifecycle).

 1import lsm
 2import lsm::fsm
 3import ip
 4
 5entity InterfaceIPAssignment extends lsm::ServiceEntity:
 6    """
 7        Interface details.
 8
 9        :attr service_id: A unique ID for this service.
10
11        :attr router_ip: The IP address of the SR linux router that should be configured.
12        :attr router_name: The name of the SR linux router that should be configured.
13        :attr interface_name: The name of the interface of the router that should be configured.
14        :attr address: The IP-address to assign to the given interface.
15    """
16    string service_id
17
18    string router_ip
19    string router_name
20    string interface_name
21
22    string address
23    lsm::attribute_modifier address__modifier="rw+"
24
25end
26
27entity InterfaceIPAssignmentV2 extends lsm::ServiceEntity:
28    """
29        Interface details. With a few less attributes
30
31        :attr router_ip: The IP address of the SR linux router that should be configured.
32        :attr interface_name: The name of the interface of the router that should be configured.
33        :attr address: The IP-address to assign to the given interface.
34        :attr description: A description to associate to this interface.
35    """
36    string router_ip
37    string interface_name
38    string description
39
40    string address
41    lsm::attribute_modifier address__modifier="rw+"
42
43end
44
45index InterfaceIPAssignmentV2(router_ip, interface_name)
46
47implement InterfaceIPAssignment using parents
48implement InterfaceIPAssignmentV2 using parents
49
50binding = lsm::ServiceBinding(
51    service_entity_name="service_simple",
52    default_version=0,
53    versions=[
54        lsm::ServiceBindingVersion(
55            service_entity="__config__::InterfaceIPAssignment",
56            lifecycle=lsm::fsm::service,
57            version=0,
58        ),
59        lsm::ServiceBindingVersion(
60            service_entity="__config__::InterfaceIPAssignment",
61            lifecycle=lsm::fsm::simple,
62            version=1,
63        ),
64        lsm::ServiceBindingVersion(
65            service_entity="__config__::InterfaceIPAssignmentV2",
66            lifecycle=lsm::fsm::simple,
67            version=2,
68        ),
69
70    ]
71)
72
73for instance in lsm::all(binding, min_version=1, max_version=2):
74    InterfaceIPAssignmentV2(
75        instance_id=instance["id"],
76        entity_binding=lsm::get_service_binding_version(binding, instance["service_entity_version"]),
77        router_ip=instance["attributes"]["router_ip"],
78        interface_name=instance["attributes"]["interface_name"],
79        description=instance["service_entity_version"] == 1 ? "old version" : instance["attributes"]["description"],
80        address=instance["attributes"]["address"],
81    )
82end
83
84for instance in lsm::all(binding, max_version=0):
85    InterfaceIPAssignment(
86        instance_id=instance["id"],
87        entity_binding=lsm::get_service_binding_version(binding, instance["service_entity_version"]),
88        **instance["attributes"]
89    )
90end

In this example we add two new versions. Version 1 links to the same entity model but has a different lifecycle while Version 2 is a different entity altogether. When unrolling using lsm::all we can separate into groups of versions and unroll them as we wish. In this example, version 1 and 2 will be unrolled together into InterfaceIPAssignmentV2 regardless of their original version, but version 0 will be unrolled separately into InterfaceIPAssignment).

To remove a version we can delete the corresponding lsm::ServiceBindingVersion from lsm::ServiceBinding.versions and recompile and export the new model.

Note

We cannot remove service entity versions that have active instances.

API endpoints

The following API endpoints were added in order to manage versioned services:

  • GET lsm/v2/service_catalog : List all service entity versions of each defined services entity type in the service catalog

  • GET lsm/v2/service_catalog/<service_entity>/<version>/schema: Get the json schema for a service entity version.

  • GET lsm/v2/service_catalog/<service_entity>: Get all versions of the service entity type from the service catalog.

  • GET lsm/v2/service_catalog/<service_entity>/<version>: Get one service entity version from the service catalog.

  • DELETE lsm/v2/service_catalog/<service_entity>/<version>: Delete an existing service entity version from the service catalog.

  • GET lsm/v2/service_catalog/<service_entity>/<version>/config: Get the config settings for a service entity version.

  • POST lsm/v2/service_catalog/<service_entity>/<version>/config: Set the config for a service entity version.

  • PATCH lsm/v1/service_inventory/<service_entity>/<service_id>/update_entity_version: Migrate a service instance from one service entity version to another.

The v1 endpoints are still supported. When we use the v1 endpoints to make operations on a service (i.e. get, create, update), we always target the default version of the service entity. (i.e. GET lsm/v1/service_catalog/<service_entity> will return the default version of this service entity).

Updating a service entity version is not supported for versioned entities. The new workflow is to create a new version with the required changes. However, updates to a version 0 of a service entity are allowed, if it is the only version of that service.

The endpoint to create a new service instance (POST lsm/v1/service_inventory/<service_entity>) now has an optional service_entity_version argument. If left empty, the service instance will be created using the default version of the service entity. Most of the other endpoints that manage service instances remain unchanged, the only exception being the endpoint to list the service instances of a given service entity (GET lsm/v1/service_inventory/<service_entity>) which received an update to the filter argument to make it possible to filter by service entity version (i.e. GET lsm/v1/service_inventory/<service_entity>?filter.service_entity_version=ge:2 to filter for instances with service entity version greater than or equal to 2).

Migrating instances between service entity version

With the introduction of versions to service entities we might want to migrate existing instances to newer versions. This can only be done with the PATCH /lsm/v1/service_inventory/<service_entity>/<service_id>/update_entity_version endpoint or by calling the lsm_services_update_entity_version API method on the Inmanta client.

To do this we need to provide at least 1 of the 3 attribute sets that we want the instance to have on the new entity version. These attribute sets are validated against the schema of the new version. We also need to provide the target state that we want to set the instance to.

Note

This change is impossible to rollback since we override each attribute set. And each attribute set needs to be compatible with the target entity version.

Below is a simple script that migrates existing instances of our service that have service_entity_version 0 or 1 and that are on the up or failed states.

We modify the existing active attribute set of each instance that qualifies for migration to add a generic description field. We only need to set the candidate set on this example because we are moving each instance to the start state where this set will be validated and eventually promoted.

 1"""
 2    Inmanta LSM
 3
 4    :copyright: 2024 Inmanta
 5    :contact: code@inmanta.com
 6    :license: Inmanta EULA
 7"""
 8
 9import argparse
10
11import requests
12
13
14def main(args):
15    environment = args.env
16    address = args.host
17    port = args.port
18    service_name = "service_simple"
19
20    # Make sure that our service is created
21    url = f"http://{address}:{port}/lsm/v1/service_catalog/{service_name}"
22    response = requests.get(
23        url=url,
24        headers={"X-Inmanta-tid": environment, "Content-Type": "application/json"},
25    )
26    assert response.status_code == 200
27
28    # Fetch every instance of our service that has version 0 or 1
29    url = f"http://{address}:{port}/lsm/v1/service_inventory/{service_name}?filter.service_entity_version=lt:2"
30    response = requests.get(
31        url=url,
32        headers={"X-Inmanta-tid": environment, "Content-Type": "application/json"},
33    )
34    assert response.status_code == 200
35    d = response.json()
36    # Iterate over every retrieved instance and migrate to version 2
37    for instance in d["data"]:
38        # Only migrate instances on failed and up states because they are the only ones that we know for sure that have an
39        # active attribute set. We will:
40        # 1) Take this active attribute set, modify it to include a description and remove the missing fields.
41        # 2) Set this set as the new candidate attribute set
42        # 3) Set the state to 'start' so that our new candidate attribute set is validated and, eventually, promoted.
43        if instance["state"] not in ["failed", "up"]:
44            print(
45                f"Instance with id {instance['id']} is being skipped because it is on state {instance['state']} "
46                f"and not on state 'up' or 'failed'."
47            )
48            continue
49
50        description = "migrated instance"
51
52        url = f"http://{address}:{port}/lsm/v1/service_inventory/{service_name}/{instance['id']}/update_entity_version"
53
54        active_attributes = instance["active_attributes"]
55        attributes = {
56            "router_ip": active_attributes["router_ip"],
57            "interface_name": active_attributes["interface_name"],
58            "address": active_attributes["address"],
59            "description": description,
60        }
61        response = requests.patch(
62            url=url,
63            headers={"X-Inmanta-tid": environment, "Content-Type": "application/json"},
64            json={
65                "current_version": instance["version"],
66                "target_entity_version": 2,
67                "state": "start",
68                "candidate_attributes": attributes,
69                "active_attributes": None,
70                "rollback_attributes": None,
71            },
72        )
73        assert response.status_code == 200, response.text
74
75
76if __name__ == "__main__":
77    parser = argparse.ArgumentParser(
78        description="The goal of this script is to migrate service instances (in the up and failed states) "
79        "from version 0 or 1 of service entity `service_simple` to version 2"
80    )
81    parser.add_argument(
82        "--host",
83        help="The address of the server hosting the environment",
84        default="172.25.102.2",
85    )
86    parser.add_argument(
87        "--port", help="The port of the server hosting the environment", default=8888
88    )
89    parser.add_argument(
90        "--env",
91        help="The environment to execute this script on",
92        default="8404deea-3621-4bca-9076-6a612115f810",
93    )
94    parser_args = parser.parse_args()
95    main(parser_args)

Multi-version Inter-Service Relations

Previously, an entity in the model corresponded to a service, with the introduction of multi-version lsm, that is no longer the case, a service can have multiple versions and each version can have the same, or different entities. So what does this mean for inter-service relations?

An inter-service relation still is a relation between 2 services, meaning that one service is related to another, regardless of version and the entity that each version uses in the model. It is up to the developer to make sure that the model remains functional and coherent.

Here is an example of an inter-service relation using multi-version lsm. In this example we have a service with 2 versions each with a different entity in the model (Service and ChildService) and another service (RefService) that refers to them.

Both services are still supported, but, when unrolling the referred services we only create ChildService instances (although we could realistically support both).

 1import lsm
 2import lsm::fsm
 3
 4entity Service extends lsm::ServiceEntity:
 5    string name
 6end
 7
 8index Service(instance_id)
 9implement Service using parents
10
11entity ChildService extends Service:
12    string description
13end
14
15implement ChildService using parents
16
17entity RefService extends lsm::ServiceEntity:
18    string ref_name
19end
20
21implement RefService using parents
22
23RefService.ref [1] lsm::__service__, lsm::__rw__ Service
24
25child_service_binding = lsm::ServiceBinding(
26    service_entity_name="child_service",
27    default_version=1,
28    versions=[
29        lsm::ServiceBindingVersion(
30            service_entity="__config__::Service",
31            lifecycle=lsm::fsm::simple,
32            version=0
33        ),
34        lsm::ServiceBindingVersion(
35            service_entity="__config__::ChildService",
36            lifecycle=lsm::fsm::simple,
37            version=1
38        ),
39    ]
40)
41
42main_service_binding = lsm::ServiceBinding(
43    service_entity_name="main_service",
44    default_version=0,
45    versions=[
46        lsm::ServiceBindingVersion(
47            service_entity="__config__::RefService",
48            lifecycle=lsm::fsm::simple,
49            version=0,
50        )
51    ]
52)
53
54for instance in lsm::all(child_service_binding):
55    ChildService(
56        instance_id=instance["id"],
57        name=instance["attributes"]["name"],
58        description=instance["service_entity_version"] > 0 ? instance["attributes"]["description"] : "old service",
59        entity_binding=lsm::get_service_binding_version(child_service_binding, instance["service_entity_version"]),
60    )
61end
62
63for instance in lsm::all(main_service_binding):
64    RefService(
65        instance_id=instance["id"],
66        ref_name=instance["attributes"]["ref_name"],
67        ref=ChildService[instance_id=instance["attributes"]["ref"]],
68        entity_binding=lsm::get_service_binding_version(main_service_binding, 0)
69    )
70end

In this example we create a new version that uses the DifferentService entity in the model. This entity does not inherit from Service so we also changed RefService.ref to expect DifferentService instead.

Now when we unroll the services that RefService refers to, we have to create instances of DifferentService, otherwise the model will not compile. Another alternative would be to create a super class that all of the entities that our service uses would inherit from.

 1import lsm
 2import lsm::fsm
 3
 4entity Service extends lsm::ServiceEntity:
 5    string name
 6end
 7
 8index Service(instance_id)
 9implement Service using parents
10
11entity ChildService extends Service:
12    string description
13end
14
15implement ChildService using parents
16
17entity DifferentService extends lsm::ServiceEntity:
18    string diff_name
19    int original_version
20end
21
22index DifferentService(instance_id)
23implement DifferentService using parents
24
25entity RefService extends lsm::ServiceEntity:
26    string ref_name
27end
28
29implement RefService using parents
30
31RefService.ref [1] lsm::__service__, lsm::__rw__ DifferentService
32
33
34child_service_binding = lsm::ServiceBinding(
35    service_entity_name="child_service",
36    default_version=2,
37    versions=[
38        lsm::ServiceBindingVersion(
39            service_entity="__config__::Service",
40            lifecycle=lsm::fsm::simple,
41            version=0
42        ),
43        lsm::ServiceBindingVersion(
44            service_entity="__config__::ChildService",
45            lifecycle=lsm::fsm::simple,
46            version=1
47        ),
48        lsm::ServiceBindingVersion(
49            service_entity="__config__::DifferentService",
50            lifecycle=lsm::fsm::simple,
51            version=2
52        ),
53    ]
54)
55
56main_service_binding = lsm::ServiceBinding(
57    service_entity_name="main_service",
58    default_version=0,
59    versions=[
60        lsm::ServiceBindingVersion(
61            service_entity="__config__::RefService",
62            lifecycle=lsm::fsm::simple,
63            version=0,
64        )
65    ]
66)
67
68
69for instance in lsm::all(child_service_binding):
70    DifferentService(
71        instance_id=instance["id"],
72        diff_name=instance["service_entity_version"] > 1 ? instance["attributes"]["diff_name"] :  instance["attributes"]["name"],
73        original_version=instance["service_entity_version"] > 1 ? instance["attributes"]["original_version"] : instance["service_entity_version"],
74        entity_binding=lsm::get_service_binding_version(child_service_binding, 1),
75    )
76end
77
78for instance in lsm::all(main_service_binding):
79    RefService(
80        instance_id=instance["id"],
81        ref_name=instance["attributes"]["ref_name"],
82        ref=DifferentService[instance_id=instance["attributes"]["ref"]],
83        entity_binding=lsm::get_service_binding_version(main_service_binding, 0)
84    )
85end

Our migration script is also a bit different. We can’t just set the state to start because it is a non-exporting state, which means that our child entities would not be unrolled and the index lookup on the main_service unrolling would fail, causing the compilation to fail.

 1"""
 2    Inmanta LSM
 3
 4    :copyright: 2024 Inmanta
 5    :contact: code@inmanta.com
 6    :license: Inmanta EULA
 7"""
 8
 9import argparse
10
11import requests
12
13
14def main(args):
15    environment = args.env
16    address = args.host
17    port = args.port
18    service_name = "child_service"
19
20    # Make sure that our service is created
21    url = f"http://{address}:{port}/lsm/v1/service_catalog/{service_name}"
22    response = requests.get(
23        url=url,
24        headers={"X-Inmanta-tid": environment, "Content-Type": "application/json"},
25    )
26    assert response.status_code == 200
27
28    # Fetch every instance of our service that has version 0 or 1
29    url = f"http://{address}:{port}/lsm/v1/service_inventory/{service_name}?filter.service_entity_version=lt:2"
30    response = requests.get(
31        url=url,
32        headers={"X-Inmanta-tid": environment, "Content-Type": "application/json"},
33    )
34    assert response.status_code == 200
35    d = response.json()
36    # Iterate over every retrieved instance and migrate to version 2
37    for instance in d["data"]:
38        # Only migrate instances on failed and up states because they are the only ones that we know for sure that have an
39        # active attribute set. We will:
40        # 1) Take this active attribute set, modify it to include a description and remove the missing fields.
41        # 2) Set this set as the new candidate attribute set.
42        # 3) Set this set as the new active attribute set (in order to not break the compilation of our model).
43        # 4) Set the state to 'update_start' so that our new candidate attribute set is validated and, eventually, promoted.
44        if instance["state"] not in ["failed", "up"]:
45            print(
46                f"Instance with id {instance['id']} is being skipped because it is on state {instance['state']} "
47                f"and not on state 'up' or 'failed'."
48            )
49            continue
50
51        url = f"http://{address}:{port}/lsm/v1/service_inventory/{service_name}/{instance['id']}/update_entity_version"
52
53        active_attributes = instance["active_attributes"]
54        attributes = {
55            "diff_name": f"{active_attributes['name']} - {active_attributes.get('description', 'old service')}",
56            "original_version": instance["service_entity_version"],
57        }
58        response = requests.patch(
59            url=url,
60            headers={"X-Inmanta-tid": environment, "Content-Type": "application/json"},
61            json={
62                "current_version": instance["version"],
63                "target_entity_version": 2,
64                "state": "update_start",
65                "candidate_attributes": attributes,
66                "active_attributes": attributes,
67                "rollback_attributes": None,
68            },
69        )
70        assert response.status_code == 200, response.text
71
72
73if __name__ == "__main__":
74    parser = argparse.ArgumentParser(
75        description="The goal of this script is to migrate service instances (in the up and failed states) "
76        "from version 0 or 1 of service entity `child_service` to version 2"
77    )
78    parser.add_argument(
79        "--host",
80        help="The address of the server hosting the environment",
81        default="localhost",
82    )
83    parser.add_argument(
84        "--port", help="The port of the server hosting the environment", default=8888
85    )
86    parser.add_argument(
87        "--env",
88        help="The environment to execute this script on",
89        default="f499500c-36b1-4690-bf28-f18d299c7fc0",
90    )
91    parser_args = parser.parse_args()
92    main(parser_args)