SDRE: Selective data redundancy elimination for resource constrained hosts

Data redundancy elimination (DRE), also known as data de-duplication, reduces the data amount to be transferred or stored by identifying and eliminating both intra-object and inter-object duplicated data elements. It is one of the key content delivery acceleration techniques over wide area networks (WANs) to reduce delivery latency and bandwidth consumptions by reducing the amount of data to be transferred. Deploying DRE at the end hosts maximizes the bandwidth savings and latency reductions, because the amount of content sent to the destination hosts is minimized. However, standard DRE used to identify redundant content chunks is very expensive in terms of memory and processing capability especially on resource constrained hosts. By analyzing the web application traffic traces, we find out that some types of contents have more redundant contents than others. Thus, it is possible to apply DRE selectively and opportunistically on those contents with more redundant data elements than other content types to save the memory and processing resources at the hosts. In this paper, we propose content-type based selective DRE (SDRE), which deploys DRE selectively on the contents which have the most opportunities for redundant content identification. We explore the benefits of deploying SDRE on smartphone traffic traces. The results show that SDRE can achieve almost the same bandwidth savings as that of standard DRE with less computation and smaller memory.