Stratos Keranidis, Wei Liu, Michael Mehari, Pieter Becue, Stefan Bouckaert, Ingrid Moerman, Thanasis Korakis, Iordanis Koutsopoulos and Leandros Tassiulas

CONCRETE: A benchmarking framework to CONtrol and Classify REpeatable Testbed Experiments

Abstract

The global research community has been recently directed towards implementation approaches and evaluation through experimentation in networking testbeds. A unique problem that arises during experimentation in such environments is that several different factors may impact the monitored performance of networks under consideration. More specifically, the major factor that significantly impacts performance is the effect of interference that is generated either from testbed nodes that simultaneously transmit or from collocated commercial devices belonging to external networks. Moreover, as the level of interference significantly varies over time, the monitored performance may present high variation among several executions of the same experiment. In order to ensure stable experimental conditions, specific tools have to be developed that would be able to detect whether the gathered results have been affected by external factors. In this work, we present a novel benchmarking framework, developed within the collaboration of CREW and OpenLab EU FP7 projects. The developed framework provides for proper evaluation of experimentally gathered results and thus enables the experimenter to arrive at solid conclusions regarding the validity of the conducted experiments.

Authors

Stratos Keranidis, Wei Liu, Michael Mehari, Pieter Becue, Stefan Bouckaert, Ingrid Moerman, Thanasis Korakis, Iordanis Koutsopoulos and Leandros Tassiulas

Keywords

Monitoring, Benchmarking, Testbed experiments

Participates in following conferences:
FIRE Engineering Conference