Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 24 additions & 3 deletions ansible/doc/README.testbed.Config.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,29 @@
# Testbed Configuration

```testbed.csv``` is the topology configuration file for the testbed.
## Testbed Inventory

# ```testbed.csv```
- [```ansible/lab```](../lab): Include all lab DUTs, fanout switches and testbed server topologies

- [```ansible/veos```](../veos): all servers and VMs

## Testbed Physical Topology

- [```ansible/files/sonic_lab_devices.csv```](../files/sonic_lab_devices.csv): Helper file helps you create lab_connection_graph.xml, list all devices that are physically connected to fanout testbed (all devices should be in ansible/lab)

- [```ansible/files/sonic_lab_links.csv```](../files/sonic_lab_links.csv): Helper file helps you to create lab_connection_graph.xml, list all physical links between DUT, Fanoutleaf and Fanout root switches, servers and vlan configurations for each link

- [```ansible/files/lab_connection_graph.xml```](../files/lab_connection_graph.xml): This is the lab graph file for library/conn_graph_facts.py to parse and get all lab fanout switch connections information. If you have only one fanout switch, you may go head manually modify the sample lab_connection_graph.xml file to set bot your fanout leaf and fanout root switch management IP point to the same fanout switch management IP and make sure all DUT and Fanout name and IP are matching your testbed.

- ```ansible/files/creategraph.py```: Helper file helps you generate a lab_connection_graph.xml based on the device file and link file specified above.

Based on ansible_facts, you may write ansible playbooks to deploy fanout switches or run test which requires to know the DUT physical connections to fanout switch


## Testbed Logical Topology

[```testbed.csv```](../testbed.csv) is the topology configuration file for the testbed.

### ```testbed.csv``` format
```
# conf-name,group-name,topo,ptf_image_name,ptf_mgmt_ip,server,vm_base,dut,comment
ptf1-m,ptf1,ptf32,docker-ptf-sai-mlnx,10.255.0.188/24,server_1,,str-msn2700-01,Tests ptf
Expand All @@ -21,7 +42,7 @@ vms-t1-lag,vms1-1,t1-lag,docker-ptf-sai-mlnx,10.255.0.178/24,server_1,VM0100,str
- dut – target dut name
- comment – any text here

# ```testbed.csv``` consistency rules
### ```testbed.csv``` consistency rules
```
# uniq-name,testbed-name,topo,ptf_image_name,ptf_ip,server,vm_base,dut,owner
vms2-2-b,vms2-2,t1,docker-ptf-sai-brcm,10.0.10.7/23,server_1,VM0100,str-d6000-05,brcm test
Expand Down
7 changes: 7 additions & 0 deletions ansible/doc/README.testbed.FAQ.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,12 @@
# FAQ

## Minimum hardware requirement

- One 'Fanout' layer-2 switch to connect all SONiC Switch ports and Linux server NIC.
you may need more 'Fanout' Switches connects to multiple SONiC switches as DUTs.

- Linux Server with minimum 92G memory.

## How to find IP addresses of VMs and PTF
- IP address of testbed PTF container could be found in testbed.csv
- To find some VM IP address:
Expand Down
72 changes: 45 additions & 27 deletions ansible/doc/README.testbed.Setup.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,55 +30,73 @@ iface br1 inet static
```

- Installed python 2.7 (required by ansible).
- Add Dockers official GPG key
- Add Docker's official GPG key
```
$ curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
```

## Build and run ```sonic-mgmt``` docker
## Setup docker registry for *PTF* docker

ansible playbook in ```sonic-mgmt``` repo requires to setup ansible and various dependencies.
We have built a ```sonic-mgmt``` docker that installs all dependencies, and you can simply
build that docker and ansible playbook inside the docker.
PTF docker is used to send and receive packets to test data plane.

- Build ```sonic-mgmt``` docker
- Build PTF docker
```
git clone --recursive https://github.com/Azure/sonic-buildimage.git
make target/docker-ptf.gz
```

- Setup [docker registry](https://docs.docker.com/registry/) and upload *docker-ptf* to the docker registry.

## Build and run *sonic-mgmt* docker

ansible playbook in *sonic-mgmt* repo requires to setup ansible and various dependencies.
We have built a *sonic-mgmt* docker that installs all dependencies, and you can build
the docker and run ansible playbook inside the docker.

- Build *sonic-mgmt* docker
```
git clone --recursive https://github.com/Azure/sonic-buildimage.git
make target/docker-sonic-mgmt.gz
```

- Run ```sonic-mgmt``` docker
Pre-built *sonic-mgmt* can also be downloaded from [here](https://sonic-jenkins.westus.cloudapp.azure.com/job/common/job/docker-sonic-mgmt/lastSuccessfulBuild/artifact/target/docker-sonic-mgmt.gz).

- Run *sonic-mgmt* docker
```
docker load -i target/docker-sonic-mgmt.gz
docker run -it docker-sonic-mgmt bash
cd ~/sonic-mgmt
```

From now on, all steps are running inside the ```sonic-mgmt``` docker.
From now on, all steps are running inside the *sonic-mgmt* docker.

## Prepare testbed configurations

Prepare various configuration files for your testbed.

- Add/Update your testbed server management IP in veos file. Example:'STR-ACS-SERV-01 ansible_host=10.0.0.5' where 10.0.0.5 your server mgmt ip
- Add testbed server credentials in ```ansible/group_vars/vm_host/creds.yml```
- Check that ansible could reach this device by command ```ansible -m ping -i veos vm_host_1```
- Put files: ```Aboot-veos-serial-8.0.0.iso``` and ```vEOS-lab-4.15.9M.vmdk``` to /home/{your_username from step 3}/veos-vm/images on your testbed server
- Edit ```ansible/host_vars/STR-ACS-SERV-01.yml```. You need to change ```external_iface```,```mgmt_gw``` and ```mgmt_prefixlen```. These settings define network parameters for VM/ptf management interfaces. Example:

```
external_iface: p4p1   <--- trunk port of the server (connected to the fanout switch)
mgmt_gw: 10.250.0.1    <--- ip of gateway for VM mgmt interfaces
mgmt_prefixlen: 24     <--- prefixlen for management interfaces
```

- Add ip addresses for your VMs in veos inventory file ```ansible/veos``` inventory file. These IP addresses should be in the management subnet defined in above file.
- Update VM credentials in ```ansible/group_vars/eos/creds.yml```. Use root:123456 as credentials
- Add information about your docker registry here: ```vars/docker_registry.yml```
Latest *sonic-mgmt* repo is cloned into *sonic-mgmt* docker under '/var/[your-login-username]/sonic-mgmt`.
Once you are in the docker, you need to modify the testbed configuration files to reflect your lab setup.

- Server
- Update server management IP in [```ansible/veos```](../veos).
- Update testbed server credentials in [```ansible/group_vars/vm_host/creds.yml```](../group_vars/vm_host/creds.yml).
- Update server network configuration for VM and PTF management interface in [```ansible/host_vars/STR-ACS-SERV-01.yml```](../host_vars/STR-ACS-SERV-01.yml).
- ```external_iface```: server trunk port name (connected to the fanout switch)
- ```mgmt_gw```: ip of gateway for VM mgmt interfaces
- ```mgmt_prefixlen```: prefixlen for management interfaces
- Check that ansible could reach this device by command ```ansible -m ping -i veos vm_host_1```.

- VM
- Download vEOS image from [arista](https://www.arista.com/en/support/software-download).
- Copy below image files to ```~/veos-vm/images``` on your testbed server.
- ```Aboot-veos-serial-8.0.0.iso```
- ```vEOS-lab-4.15.9M.vmdk```
- Update VM IP addresses [```ansible/veos```](../voes) inventory file. These IP addresses should be in the management subnet defined above.
- Update VM credentials in [```ansible/group_vars/eos/creds.yml```](../group_vars/eos/creds.yml).

- ```PTF``` docker
- Update docker registry information in [```vars/docker_registry.yml```](../vars/docker_registry.yml).

## Setup VMs in the server


```
./testbed-cli.sh start-vms server_1 password.txt
```
Expand All @@ -87,7 +105,7 @@ Check that all VMs are up and running: ```ansible -m ping -i veos server_1```

## Deploy topology

- Update testbed.csv with your data. At least update PTF mgmt interface settings
- Update ```testbed.csv``` with your data. At least update PTF mgmt interface settings
- To deploy PTF topology run: ```./testbed-cli.sh add-topo ptf1-m ~/.password```
- To remove PTF topology run: ```./testbed-cli.sh remove-topo ptf1-m ~/.password```
- To deploy T1 topology run: ```./testbed-cli.sh add-topo vms-t1 ~/.password```