Testing¶
This guide includes:
By following the steps here, you should have a virtual environment
and a clone of connect
module locally.
Constructing A LAB File¶
All the tests are parametrized so that multiple developers can execute them on the same LAB without interfering with each other.
The user specific data is presented in the LAB file, under tests/labs/user/
directory.
To select a LAB when running the tests, use the --lab
option, specifying in argument the name of the file, except the .yaml
extension. All LAB files should end with the .yaml
extension.
Example
Here is an example of the LAB file, you can copy it in this directory, and modify the values.
environment:
id: 8404deea-3621-4bca-9076-6a612115f810 # Change this
name: ci-1 # Change this
project:
id: 3fa85f64-5717-4562-b3fc-2c963f66afa6
name: connect
prefix: ci-1 # Change this
service_id_range:
start: 10000 # Change this
stop: 10099 # Change this
vlan_id_range:
start: 600 # Change this
stop: 649 # Change this
LAB Topology Constraints¶
In the test cases some assumptions are made on the structure of the lab. Those assumptions are checked when the tests using the LAB manager are setup. Here are the assumptions:
The topology file contains at least four subscribers, named:
subscriber-1
,subscriber-2
,subscriber-3
andsubscriber-4
.Side A (
subscriber-1
andsubscriber-2
) are connected to the same provider router.Side B (
subscriber-3
andsubscriber-4
) are connected to the same provider router.Side A and B do not have a direct connection
All routers in the LAB have the same vendor.
Here is a visual representation of the topology:
Vendor specific tests¶
Some tests can only be executed with a specific topology file because they test the behavior of the module against a specific vendor. To avoid running this test case in unsupported configurations, pytest’s markers can be used.
Example¶
This is already the case for two test cases:
tests/test_model/test_carrier_ethernet_evc_nokia/test_LDP
tests/test_model/test_carrier_ethernet_evc_nokia/test_EVPN
These verify that the generated yang config contains the desired information. In order to mark those tests as suitable only for Nokia topology, they get marked with the marker nokia_only
:
@pytest.mark.nokia_only
def test_LDP(project: Project, lab_config: LabConfig) -> None:
This marker can be reused for any test that requires a Nokia topology file.
By default those tests will be skipped. If you are running the tests with a topology file containing only Nokia devices, you can either set the environment variable INMANTA_CONNECT_LAB_KIND
or the cli parameter --connect-lab-kind
to Nokia. In that case, and in that case only, those two tests will be executed.
Adding New Markers¶
To add some additional markers, i.e. when other vendor-specific tests are added apart from Nokia, there are three things to do:
Extend the
pytest_configure
function in tests/conftest.py to support the additional marker.Extend the
pytest_runtest_setup
function in tests/conftest.py to skip the test if the marker is detected but the vendor is not the one we want.Add the marker on top of the test you wish to limit to the specific vendor.
Topology File Structure¶
The topology file should contain four main entries:
routers
: A dictionary containing the routers of the LAB.subscribers
: A dictionary containing the subscribers of the LAB.links
: A list containing the links between different element (routers and subscribers) in the LAB.inventory
: A dictionary containing a valid inventory that can be given to the module for this LAB.
Here is an example of the topology file:
routers:
nokia-east:
vendor: Nokia
mgmt_address: 192.168.2.33
netconf_port: 20830
ssh_port: 20022
mgmt_username_env: NETCONF_DEVICE_USER
mgmt_password_env: NETCONF_DEVICE_PASSWORD
nokia-west:
vendor: Nokia
mgmt_address: 192.168.2.33
netconf_port: 21830
ssh_port: 21022
mgmt_username_env: NETCONF_DEVICE_USER
mgmt_password_env: NETCONF_DEVICE_PASSWORD
subscribers:
subscriber-1:
mgmt_address: 192.168.2.33
api_port: 2001
uni: inmanta:456-852-789
subscriber-2:
mgmt_address: 192.168.2.33
api_port: 2002
uni: inmanta:456-985-752
subscriber-3:
mgmt_address: 192.168.2.33
api_port: 2003
uni: inmanta:123-852-456
subscriber-4:
mgmt_address: 192.168.2.33
api_port: 2004
uni: inmanta:652-784-963
links:
- endpoints:
- type: router
device: nokia-east
interface: eth1
- type: subscriber
device: subscriber-1
interface: eth1
namespace: east1
- endpoints:
- type: router
device: nokia-east
interface: eth2
- type: subscriber
device: subscriber-2
interface: eth1
namespace: east1
- endpoints:
- type: router
device: nokia-west
interface: eth1
- type: subscriber
device: subscriber-3
interface: eth1
namespace: west1
- endpoints:
- type: router
device: nokia-west
interface: eth2
- type: subscriber
device: subscriber-4
interface: eth1
namespace: west1
inventory:
devices:
dev-1:
mgmt_ip: 172.20.20.31
mgmt_port: 22
vendor: Nokia
model: 7750 SR
os: TiMos
version: "20.10"
username_env: NETCONF_DEVICE_USER
password_env: NETCONF_DEVICE_PASSWORD
dev-2:
mgmt_ip: 172.20.20.21
mgmt_port: 22
vendor: Nokia
model: 7750 SR
os: TiMos
version: "20.10"
username_env: NETCONF_DEVICE_USER
password_env: NETCONF_DEVICE_PASSWORD
ne:
- id: "1"
name: Nokia_west
router_ip: 10.255.255.2
device: ref#devices.dev-1
- id: "2"
name: Nokia_east
router_ip: 10.255.255.1
device: ref#devices.dev-2
uni:
- id: inmanta:456-985-752
port: 1/1/c3/1
network_element: ref#ne[id=1]
- id: inmanta:456-852-789
port: 1/1/c2/1
network_element: ref#ne[id=1]
- id: inmanta:652-784-963
port: 1/1/c3/1
network_element: ref#ne[id=2]
- id: inmanta:123-852-456
port: 1/1/c2/1
network_element: ref#ne[id=2]
Here are some notes about parametrized values:
Some values in the file can be parametrized using the prefix
opt:
prefix.The remaining part of the value will be extracted and resolved.
The remaining value should match the key of one of the
TestParameter
object set in the testsconftest.py
file.
For instance, if you set opt:inm_con_lab_mgmt_ip
as a value for routers.nokia-east.mgmt_address
, when the topology file is loaded, this value will be replaced by the value passed to the --lab-mgmt-ip
argument or set in INMANTA_CONNECT_LAB_MGMT_IP
. Please note that CLI arguments have more priority than env variables
.
Assembling A Basic Test¶
We also need to make a temporary directory for the tests and utilize it by environment variables:
mkdir /tmp/env
export INMANTA_TEST_ENV=/tmp/env
The most basic type of testing is a compile
test which can be conducted like:
import uuid
import pytest
from pytest_inmanta.plugin import Project
def test_basics(project: Project) -> None:
"""
Simple example of instantiation of a CarrierEthernetEvc entity. We just try to compile it.
"""
model = f"""
import connect
import connect::infra as infra
import connect::mock
connect::CarrierEthernetEvc(
identifier="my-evc-001",
connectionType="POINT_TO_POINT",
ext_networkBackend=connect::Ext_networkBackend(type="EVPN"),
evcEndPoints=[
connect::CarrierEthernetEvcEndPoint(
identifier="my-evc-ep-1",
egressBandwidthProfilePerEndPoint=[
connect::EgressBwpFlow(
cir=1,
),
connect::EgressBwpFlow(
cir=2,
),
],
evcEndPointMap=connect::VlanIdListOrUntag(
type="LIST",
vlanIdList=[
connect::VlanId(vlanId=200),
],
),
carrierEthernetSubscriberUni=connect::CarrierEthernetSubscriberUniRef(
href="inmanta:456-985-752",
),
_uni=infra::UserNetworkInterface(
id="inmanta:456-985-752",
port="1/1/c3/1",
network_element=infra::NetworkElement(
id="1",
name="ne-1",
router_ip="1.2.3.4",
device=infra::Device(
mgmt_ip="1.2.3.4",
mgmt_port=22,
vendor="Nokia",
model="7750 SR",
os="TiMos",
version="20.10",
username_env="NETCONF_DEVICE_USER",
password_env="NETCONF_DEVICE_PASSWORD",
)
)
),
),
connect::CarrierEthernetEvcEndPoint(
identifier="my-evc-ep-2",
egressBandwidthProfilePerEndPoint=[
connect::EgressBwpFlow(
cir=1,
),
connect::EgressBwpFlow(
cir=2,
),
],
evcEndPointMap=connect::VlanIdListOrUntag(
type="LIST",
vlanIdList=[
connect::VlanId(vlanId=201),
],
),
carrierEthernetSubscriberUni=connect::CarrierEthernetSubscriberUniRef(
href="inmanta:652-784-963",
),
_uni=infra::UserNetworkInterface(
id="inmanta:652-784-963",
port="1/1/c3/1",
network_element=infra::NetworkElement(
id="2",
name="ne-2",
router_ip="1.2.3.5",
device=infra::Device(
mgmt_ip="1.2.3.5",
mgmt_port=22,
vendor="Nokia",
model="7750 SR",
os="TiMos",
version="20.10",
username_env="NETCONF_DEVICE_USER",
password_env="NETCONF_DEVICE_PASSWORD",
)
)
),
),
],
purge_resources=false,
instance_id="{uuid.uuid4()}",
)
"""
project.compile(model, no_dedent=False)
Running Tests¶
# There are multiple env files with the names of vendors. These match a specific topology file
# in tests/labs/topology
# Sourcing one of this files will also run the common-env.sh script. This script loads some
# environment variables common to any developer of the module.
# If you wish to load some env variables automatically, this script looks for the existence
# of personal_env.sh file and executes it, if found. You can add all private envs you require there.
source <topology>-env.sh
export INMANTA_CONNECT_LAB_USER="" # The name of the lab file you created (if not added to personal_env.sh)
export INMANTA_LSM_ENVIRONMENT="" # An environment on mentioned orchestrator (if not added to personal_env.sh)
pytest tests \
--use-module-in-place \
--log-cli-level=debug \
tests/
Running Mypy Type Check¶
The use of Mypy is encouraged since it makes the code easier to read and debug should a problem occur.
Mypy is an optional static type checker for Python that aims to combine the benefits of dynamic or "duck"
typing and static typing.
# Check typing in the plugins
make mypy-plugins
# Check typing in the tests
make mypy-tests
# Check typing in both plugins and tests
make mypy
To see if you improved the typing, do
make mypy-save
before you make changes andmake mypy-diff
afterwards