Very bad network simulation for testing of mobile applications [PART 2]

In the previous post we talked about the need for platform independent, scriptable solution for testing of your mobile applications in a poor internet conditions. To complement the theory with something executable, this post will introduce scripts (Debian Linux like), and a guide to setup your own WIFI access point, which would simulate slow, unreliable mobile internet. You will be able to connect with your Android, iOS, Windows, whatever devices and see from your office, how your apps adapt.

This tutorial will be divided into following sections:

1. Failure when firstly attempting to solve this problem.
2. Obtaining the right USB WIFI dongle.
3. Tutorial for creating an AP from your Linux based workstation.
4. Script for changing the Quality of service (QoS) characteristics of your AP.
5. Script for setting particular QoS, simulating GRPS, EDGE, 3G, LTE, whatever networks.
6. Example usage

Failure, when firstly attempting to solve this problem.

My first attempt did not end up successfully. I am not saying it is a wrong way, but I was just not able to go this way. The plan was to:

• Buy WIFI dongle with ability to be in AP mode.
• Virtualize OpenWRT (a small Linux distro, usually run on routers) in the VirtualBox.
• Install on that virtual machine a Cellular-Data-Network-Simulator - which is capable running on OpenWRT, and is established on well known technologies: tc, iptables and CloudShark.
• Connect with devices to that AP, and use CloudShark to sniff the network in order to see particular packets.

It looked promising. It would be just an integration of already existing parts, not reinventing a wheel. A fairy tail. It worked. Even when I found out that it would require some work to do in order to script the way, how the devices are connected to the Cellular-Data-Network-Simulator, and the way how the QoS characteristics are changed in order to switch among 2G, 3G, etc. networks. But it was nothing impossible to overcome. The biggest problem which I encountered after I set it up, was the stability of the AP. It switched off the WIFI dongle at random intervals. I studied various OpenWRT log files, but have not found the root cause, hence I was not able to fix it. I needed to think out a different way. Following describes my second attempt, which finally worked.

Obtaining the right USB WIFI dongle.

First things first. Before buying the WIFI dongle, checkout its chipset, and see, whether it is supported by some Linux driver. I am using TP-Link TL-W22N. Its chipset AR9271 is supported by ath9k_htc driver.

Tutorial for creating an AP from your Linux based workstation.

Next you will need to setup various things properly: hostapd, DHCP server, firewall. I followed this great post (automated in the install script for Debian like systems here). In that install script, you can also spot a part (wifi_access_point), which would enable you to start the AP as a service.

Script for changing of Quality of service (QoS) characteristics of your AP.

Now, you should be able to connect to the created AP with your devices. It should provide you a similar Internet connection quality as you have on your workstation. To simulate various cellular data networks we need to limit it somehow.

Following script does it by setting various firewall rules. You will need to alter it a bit before using it.

1. Set IF_IN to network interface name which is dedicated to the created AP.
2. Set IF_OUT to network interface name by which is your workstation connected the Internet.
3. Set IP_IN to a IP address space which will be assigned to your connected devices (you chose this when setting up a DHCP server).
4. Set IP_OUT to the IP address of your application backend server.

Save the following script, and named it e.g. qos.sh.

#!/bin/bash
#
#  tc uses the following units when passed as a parameter.
#  kbps: Kilobytes per second 
#  mbps: Megabytes per second
#  kbit: Kilobits per second
#  mbit: Megabits per second
#  bps: Bytes per second 
#       Amounts of data can be specified in:
#       kb or k: Kilobytes
#       mb or m: Megabytes
#       mbit: Megabits
#       kbit: Kilobits
#  To get the byte figure from bits, divide the number by 8 bit
#

#
# Name of the traffic control command.
TC=/sbin/tc

# The network interface we're planning on limiting bandwidth.
IF_IN=wlan1
IF_OUT=wlan0

# IP address of the machine we are controlling
IP_IN=10.10.0.0     # Host IP
IP_OUT=0.0.0.0 #the address of your backend server

# Filter options for limiting the intended interface.
U32_IN="$TC filter add dev $IF_IN protocol ip parent 1: prio 1 u32"
U32_OUT="$TC filter add dev $IF_OUT protocol ip parent 2: prio 1 u32"

start() {
    ping -c 1 $IP_OUT >/dev/null 2>&1
    if [ $? -ne 0 ]; then
 echo "Error:"
        echo "The IP address: $IP_OUT is not reachable!"
 echo "Check out the backend server address!"
 exit -1
    fi

    $TC qdisc add dev $IF_IN root handle 1: htb default 30
    # download bandwidth
    $TC class add dev $IF_IN parent 1: classid 1:1 htb rate "$1"
    $U32_IN match ip dst $IP_IN/24 flowid 1:1
    # in delay
    $TC qdisc add dev $IF_IN parent 1:1 handle 10: netem delay "$3" "$4" distribution normal
    # in packet loss
    $TC qdisc add dev $IF_IN parent 10: netem loss "$7" "$8"

    # upload bandwidth
    $TC qdisc add dev $IF_OUT root handle 2: htb default 20
    $TC class add dev $IF_OUT parent 2: classid 2:1 htb rate "$2"
    $U32_OUT match ip dst $IP_OUT/32 flowid 2:1
    # out delay
    $TC qdisc add dev $IF_OUT parent 2:1 handle 20: netem delay "$5" "$6" distribution normal
    $U32_OUT match ip dst $IP_OUT/32 flowid 20:
}

stop() {

# Stop the bandwidth shaping.
    $TC qdisc del dev $IF_IN root
    $TC qdisc del dev $IF_OUT root
}

show() {

# Display status of traffic control status.
    echo "Interface for download:"
    $TC -s qdisc ls dev $IF_IN
    echo "Interface for upload:"
    $TC -s qdisc ls dev $IF_OUT

}

case "$1" in

  start)
    if [ "$#" -ne 8 ]; then
        echo "ERROR: Illegal number of parameters"
        echo "Usage: ./qos.sh start [downloadLimit] [uploadLimit] [inDelayMax] [inDelayMin] [outDelayMax] [outDelayMin] [packetLossPercentage] "
        echo "[downloadLimit]  See man page of tc command to see supported formats, e.g. 1mbit."
 echo "[uploadLimit] The same as for downloadLimit applies here."
 echo "[inDelayMax] Max delay in miliseconds for requests outgoing from AP."
 echo "[inDelayMin] Min in delay."
 echo "[outDelayMax] Max Delay in miliseconds for requests outgoing to servers."
 echo "[outDelayMin] Min out delay."
 echo "[packetLossPercentage] The percentage of packet lost"
 echo "Example: /qos.sh 1mbit 1mbit 50ms 20ms 30ms 10ms 5%"
        exit -1
    fi

    echo "Starting  shaping quality of service: "
    start $2 $3 $4 $5 $6 $7 $8
    echo "done"
    ;;

  stop)

    echo "Stopping shaping quality of service: "
    stop
    echo "done"
    ;;

  show)

    echo "Shaping quality of service status for $IF_IN and $IF_OUT:"
    show
    echo ""
    ;;

  *)

    pwd=$(pwd)
    echo "Usage: qos.sh {start|stop|show}"
    ;;

esac

exit 0

Script for setting particular QoS, simulating GRPS, EDGE, 3G, LTE, whatever networks.

Now when you have a script to limit the QoS characteristic of your created AP, you will need to do some measurements, in order to have a clue what bandwidth, what latency, and what packet loss various cellular data networks have. You will need to find out a way how to measure these characteristics in the environment where your customers use your application.

The reason is that the same data network type (e.g. 3G) can have different QoS characteristics on different places. There are other factors in play as well: mobile Internet provider, hour of the day, city vs. village, weather and like. For the measurement I used handy mobile applications Speedtest.net (for bandwidth and latency) and Fing for a double check up, as it is able to ping any server you like.

Off-topic: Would not be awesome, if there is a web service which would give me the average QoS characteristics of any place in the world for a particular hour of the day, for a particular data carrier, for particular weather and other conditions? I submitted a bachelor thesis assignment, but so far no enrollment :) And it would be IMO quite easy to setup: a mobile application with gamification characteristics to find out the particular statistics, store them, and make them available via some REST endpoints.

Save the following script as simulate.sh, into the same directory as previous script was saved. The measured values, you can see, are valid for average morning in Brno, Czech republic. The average was made after one week of measuring.

#!/bin/bash

QOS="./qos.sh"
echo -n "Shaping WIFI to "

case "$1" in

    GPRS)
 echo "GRPRS"
 $QOS stop > /dev/null 2>&1
 $QOS start 80kbit 20kbit 200ms 40ms 200ms 40ms 5%
 ;;

    EDGE)
 echo "EDGE"
 $QOS stop > /dev/null 2>&1
 $QOS start 200kbit 260kbit 120ms 40ms 120ms 40ms 5%
 ;;

    HDSPA)
 echo "HDSPA"
 $QOS stop > /dev/null 2>&1
 $QOS start 2400kbit 2400kbit 100ms 100ms 100ms 100ms 5%
 ;;

    LTE)
 echo "LTE"
 ;;

    FULL)
 echo "FULL"
 $QOS stop > /dev/null 2>&1
 ;;

    DISABLED)
 echo "DISABLED"
 $QOS stop > /dev/null 2>&1
 $QOS start 1kbit 1kbit 5000ms 5000ms 5000ms 5000ms 5%
 ;;

    *)
        pwd=$(pwd)
        echo "Usage: simulate.sh {GPRS|EDGE|HDSPA|LTE|FULL|DISABLED}"
        ;;

esac

exit 0

Example usage

So if you followed the steps, you should be able now to:

1. Start the AP by: service wifi_access_point start.
2. Simulate e.g. EDGE, by issuing: simulate.sh EDGE

Ideas have no limits. Use these scripts to e.g. stress network test your application (write a bash script which would randomly switch among all types of the network in a random intervals), or use WireShark to go deeper, to see actual packets being transmitted. Your development team would love you, if you attach to your bug report a saved transmission with a packet level information. Fixing of tough, non-deterministic network issues becomes more easy.

Disclaimer: I am still improving the scripts, use them on your own danger :) Any feedback on how you utilized these scripts, or your improvements would be deeply appreciated.