Automated Physical Testbeds for Emulation of Wireless Networks

Institute of Reliable Embedded Systems and Communication Electronics, Offenburg University of Applied Sciences, Germany has developed an automated testing environment, Automated Physical TestBeds (APTB), for analyzing the performance of wireless systems and its supporting protocols. Wireless physical networking nodes can connect to this APTB and the antenna output of this attaches with the RF waveguides. To model the RF environment this RF waveguides then establish wired connection among RF elements like splitters, attenuators and switches. In such kind of set up it’s well possible to vary the path characteristics by altering the attenuators and switches. The major advantage of using APTB is the possibility of isolated, well controlled, repeatable test environment in various conditions to run statistical analysis and even to execute regression tests. This paper provides an overview of the design and implementation of APTB, demonstrates its ability to automate test cases, and its efficiency.


Introduction
Over the last decade, wireless networks have been deployed at almost all the areas.The exciting capabilities made possible by the inexpensive ubiquity of wireless networking technology, has led to a massive amount of research activity to improve the performance of existing wireless networks, and develop new applications based on this technology such as wireless sensor networks and internet of things.Even though many wireless standards are available, to meet the specific requirements in terms of low energy, scalability etc, we need to develop optimum solutions.Conducting such type of wireless network research is, none the less, a challenging effort due to the distributed and short lived nature of wireless signal propagation.There are many challenges in wireless network research which are nicely listed in [1], [18].The major approaches used for performance evaluation of wireless networks are simulation, virtualization, outdoor field test (test beds), and emulation.
Simulation is used to test the behaviour of the protocol and its implementation, i.e. the firmware on hardware abstraction layer level.The advantage of simulations is its scalability, but simulation is not helpful in debugging the underlying hardware and is totally missing the effects that hardware has on RF characteristics [1].
Virtualization of networks also used to analyse the protocol stack in a virtual environment of intended network.Network virtualization has the advantage that enables to emulate connection between applications, services, dependencies, and end users in a test environment without the need to test in real hardware.But, distinctive properties of wireless environment such as time variant channels, mobility, broadcast, attenuation, and specific access technologies makes convergence, sharing and abstraction difficult to achieve [10].
Outdoor field tests play a vital role for testing the capabilities and reliabilities of wireless systems, but the major disadvantages are irreproducible tests due to the uncontrollable nature of the outdoor environment.Also, outdoor field tests are time consuming and therefore expensive [1].
Emulation is the combination of simulation and testbed.As in the simulation approach emulators can generate predefined emulated network conditions and traffic dynamics for different use cases.Hardware-based emulation clearly achieves the most physical layer realism; practical considerations such as ease of development, control, and experimental repeatability have made emulation the dominant experimental technique [1].The requirements of the development of an emulator can be seen in [2].
Authors have developed first generation of our automated RF emulator for distributed wireless sensor networks for analysing EnOcean Radio Protocol (ERP) [2], and the second generation developed to test Ko-TAG subsystem [1].In this paper we describe the architecture and implementation details our third generation fully automated emulator for wireless networks.This includes the physical networking nodes, RF elements to allow the controlling of RF environment between the participating nodes, and automated control software.We call this set up as Automated Physical Testbeds (APTB) for emulation of wireless networks.
APTB is an emulator to perform reproducible tests with static and dynamic attenuation between the participating wireless nodes as real laboratory set up.The nodes are connected over their antenna outputs to RFwaveguides.It is also possible to cut or switch the connections of the network, in this way many different topologies can be set up.APTB promises to achieve much of the flexibility of wireless emulators while maintaining great realism of real wireless networks [4].
This contribution is structured as follows: Ch. 2 gives a review on the previously developed emulator platforms by other researchers and the previous generation of emulator by author's Ch. 3 presents the APTB architecture and implementation details, whereas Ch. 4 gives an overview of experience with APTB, Ch. 5 concludes the paper.

State of the art
An overview of the previously proposed emulators by other work groups and authors team is given in this chapter.

Review of emulators
In the field of wireless network emulation we observed different approaches like a).Central control based wireless emulators, in which the wireless nodes connect to central server as seen in ONE [5], MIT's Click modular router [6], Dummynet [7] and NIST NET [8].b).Simulation integrated wireless emulators, which combines the network emulation with existing network simulators which has rich models and libraries as VINT/ns [9].c).Trace based wireless emulation, which collect the traces from the target network and then construct a wireless network model with collected traces, and then reproduces the traced network effect in a wired network and provide a reproducible environment [3].
Clearly these emulators provide a controllable and reproducible environment, but it generally lacks the support for network topology, mobility and an interactive interface for users.In some of the emulator setup the node mobility implemented using remotely controlled robots as in CMU [11], APE [12], Mobile Emulab [13], m-MiNT [14].Associated with the mobility features a tracking system [14]- [16] is used to determine the position and orientation of each node.But such type of emulator platforms is not flexible and efficient to test the wireless system in the development stage.Using a wired connection of RF elements we overcome these drawbacks.EMWIN [3] is an IP based mobile wireless network emulation system; provide a unique feature of emulating multiple mobile hosts in one single emulator node, also to emulate the mobility using a wired network of computers.
To the best of our knowledge, we have not seen a system which uses wired connection of RF elements like attenuators, splitters, switches etc., to imitate the RF environment and manipulate the path characteristics.So, we decided to develop such type of emulator for wireless networks.Next section reviews the details of our previous generation of emulators.

Review of our previous emulators
The first generation consists of elements like network nodes, management nodes, a client computer, a deployment support network (DSN), with an embedded web server running web 2.0 technology for remote monitoring and control of its main elements.The emulator originated form the implementation of the EnOcean Radio protocol (ERP), the details of this emulator and experience are described in [2].This presented an approach of management and sniffing tools for distributed wireless networks.The emulator was designed flexible and generic to extent this to other wireless protocols.
The second generation of our emulator developed for highly scalable IEEE 802.11p communication and localization subsystem in the Ko-FAS project [1].This includes physical networking nodes, but models the RF environment using RF-waveguides.The RF emulator allows the controlling of path losses and connectivity between any of the nodes with the help of RF attenuator and programmable RF switches, while it's shielded against its surrounding RF environments in the lab.Details and experience of the system design can be seen [1].
These two versions of our emulator was designed for specific projects, we extend this development to easily reuse it for other wireless network research projects.

Automated physical testbeds (APTB)
We have developed a wireless emulator, APTB that enables both realistic and repeatable wireless experimentation by supporting accurate wireless signal transmission, propagation, and reception in an emulated physical space.APTB uses physical networking nodes, but where the RF environment is still modelled using RFwaveguides and RF elements like attenuators, splitters, switches, cables, terminators etc.In this environment, it is well possible e.g. to control the path loss between any of the nodes with the help of RF attenuators and to control the connectivity between any of the nodes with the help of programmable RF switches.APTB maintains the repeatability, configurability, the ability to modify wireless device behaviour, automated tests, support for a large number of nodes, manageability of simulation while retaining the support for real applications, and much of the realism of hardware testbeds.As a result, this APTB emulator can be a superior platform for wireless network experimentation [4].

Architecture
APTB consists of both custom and commodity hardware that must act in concert to enable users to accurately emulate arbitrary signal propagation environments and efficiently execute experiments.Developing hardware to achieve this has been a challenging task.Figure 1 describes the architecture that enables our physical layer APTB to achieve accurate emulation while enabling    In the author's Lab APTB is developed in two identical physical boards, the rear side of the testbed is shown in Figure 2 with splitters, attenuators, switches, SMA cables and APTB control unit.The front side of the APTB is shown in Figure 3 consist of shield boxes where we can encapsulate the APTB nodes.

APTB hardware
The hardware elements of the APTB are APTB nodes, RF elements and controller units.
x APTB Nodes Any type of wireless nodes with antenna output can be used with APTB.For example, @ANY900-2 modules used as APTB nodes, which contains Atmel microcontroller ATmega 1281V (128 bytes of FLASH memory, 8K bytes SRAM, 4K bytes EEPROM), IEEE 802.15.4 radio transceiver AT86RF212 (868/915 MHz band) and 2Mbit serial data flash (AT25F2048), which can store the flash images of the used MCU [19].We encapsulate these APTB nodes in to shield boxes.The RF nodes are shielded from each other so that no communication occurs over air.Such types of 12 shield boxes are present in the APTB emulator.
¾ Three way splitters/combiners (ZB3PD-63+): are three-way, 0 degree, in-phase splitter and combiner covers a wide frequency range (150-6000 MHz), making this splitter suitable for GPS, GSM applications, in addition to WiFi, Bluetooth and 802.11a uses.This model also features low insertion loss, amplitude and phase unbalance [21].
¾ Variable attenuators (ZX76-15R5-SP+): is a 50Ω RF digital step attenuator that offers an attenuation range up to 15.5 dB in 0.5 dB steps.The control is a 5-bit serial interface.The model operates on a single +3 volt supply [23].
¾ SMA cables (141-6SM+): low loss precision test, 141 series hand-flex coaxial cables are used for interconnection RF elements.It has wideband frequency coverage from DC to 18GHz and low Loss, 0.43dB, at 18GHz.Also it has excellent return loss, 23 dB at 18GHz [25].
¾ RF shield boxes (STE2200): is an RF shielded test enclosure [26] used to connect the APTB nodes and have a shield RF environment for test set up.In STE2200 heavy, rugged 0.090 and 0.125 aluminium is used throughout and double lip woven Monel flexible gaskets are used at all joint locations assuring a reliable RF tight closure.Oversized hinges and latches are used to provide a physically tight seal every time.RF absorbent foam lines the interior to provide a typical RF attenuation of -90dB @ 3GHz.
¾ Terminators (ANNE-50+): SMA Male, 50Ω terminator [27] used for test set up of APTB.All the communication between RF nodes occurs through the signal propagation environment modelled using these different RF elements within the APTB emulator.
x Controller Units APTB Controller unit is an ARM Cortex-M3 based MCU (STM32F107VCT6) running together with a web interface on the PC to control paths and attenuation between different nodes.With these we can vary the topology and attenuation between the APTB nodes.
Based on the experience from earlier emulator setups [2] [1], all elements have additional shielding and are carefully mounted whereas links were fixed with a torque wrench.All elements and connections are characterized after they have been built-in with regard to attenuation, reflection, and phase shift (delay).

APTB software
The main software implementations for APTB are control software for RF elements and MATLAB scripts to automate, reproduce the tests.All the involved devices like APTB nodes, RF elements and controller units are automatically controlled by MATLAB program from the PC, integration with the APTB controller unit.
To control the RF elements of the APTB emulator uses web interface running together with APTB controller.In experiment setup the APTB consists of two parts, which helps on one side to control data traffic flow better and on other side to extend physical topology as much as it required for the test case.Two identical parts create a local network which is managed externally by an Ethernet based control interface.Used infrastructure with such devices helps to add different controlling extensions which may automatically perform tests and grab output data [1].The controlling devices are equipped with an embedded TCP/IP stack including an embedded webserver.This provides basic access mechanisms e.g. using an embedded website as shown in Figure 4.In APTB Low layer commands are used to directly access RF elements of the APTB.For example, x APTB:RADELements:ATTenuator:SETAttenuati on <id>,<val> x APTB:RADELements:SWitch:SETEnable <id>,<val> x APTB:NETements:NOde:SENDData <id>,<data> Furthermore the software of the controlling devices have been extended with a software module able to interpret the SCPI protocol commonly used to control and to monitor measurement devices.This makes it possible to control and to automate the network topology e.g. with MATLAB that is using Java-IO libraries to send the SCPI commands to the controlling devices.Self-defined configuration scripts can be used to create a timecontrolled sequence of different topologies and therefore also provides the possibility to reproduce scenarios.MATLAB programs are used for high level user interface to describe the use cases and topologies.A sample MATLAB script for a scenario of link loss add reconnection after a specific amount of time can be Figure 4 shows the automated test setup together with webserver using MATLAB, the green colour between the nodes shows the enabled paths and red shows the disabled path.

Experiences with APTB
To analyse the accuracy of the emulator we have performed many tests like creating different topologies using automatic control of the attenuator and switches using MATLAB, and we have measured the attenuation between different nodes, compared this with the calculated attenuation.These measurement results confirmed the calibration accuracy of the APTB.We have extensively used APTB for testing and verification of the protocol stacks developed by author's team.These testing involve various steps: test case definition, configuration of APTB nodes, test execution and result analysis.The measurements of different test cases were performed for different topologies like chain, mesh and tree.Each test executed many times in a time span of 24 hours to one week.More details related to these experiments and results are seen in [4], [17].

Conclusion and outlook
In this paper we presented the state of art of our wireless network emulator APTB, which helps the wireless network researchers to set up different wireless time varying topologies and path characteristics automatically.The major advantage of using APTB against traditional mount and measure approach is concentrated in striving for isolated, well controlled, repeatable test environment which may help to prove a sustainable and reliable behaviour of wireless network protocols in various conditions.The authors would be happy to share more details about the implemented APTB, so that other work groups can build identical or extended testbeds, keeping compatibility especially on the software side.
The hardware elements of the APTB are APTB nodes, RF elements and Controller units.

Figure 1 .
Figure 1.Architecture of APTB.Any of the wireless modules with an antenna output used as APTB nodes.We encapsulate these APTB nodes in to shield boxes.The RF nodes are shielded from each other using RF shield boxes so that less communication occurs over air.Such types of 12 shield boxes are present in the APTB emulator as shown Figure1.The RF elements used in the APTB are two way splitters/combiners (SP2), three way splitters/ combiners (SP3), variable and fixed attenuators (ATT), high isolation switches (SW), SMA cables, terminators and RF shielded test enclosure.All the communication between RF nodes occurs through the signal propagation environment modelled using these different RF elements within the APTB emulator.The controller units present in the APTB are two ARM Cortex-M3 based MCU to control the RF elements.

Figure 4 .
Figure 4. Screenshots for the programming and the web interface for APTB automated tests These commands are interpreted and executed by the APTB Controller which includes an according "Standard Commands for Programmable Instruments" (SCPI) parser/handler.The RF Switch Control is fitted with the STZEDN's embedded TCP/IP protocol suite (emBetter) which is expanded by an SCPI module.Furthermore the software of the controlling devices have been extended with a software module able to interpret the SCPI protocol commonly used to control and to monitor measurement devices.This makes it possible to control and to automate the network topology e.g. with MATLAB that is using Java-IO libraries to send the SCPI commands to the controlling devices.Self-defined configuration scripts can be used to create a timecontrolled sequence of different topologies and therefore also provides the possibility to reproduce scenarios.MATLAB programs are used for high level user interface to describe the use cases and topologies.A sample MATLAB script for a scenario of link loss add reconnection after a specific amount of time can be