Merge pull request #10 from banoris/up/EventV2_fuzz_report

Dashboard + report parsing script + new rpc in umbra-scenario to query topology

build/build.sh

@@ -7,10 +7,13 @@ echo "###################################"
 sudo apt update &&
     sudo apt install -y software-properties-common &&
     sudo add-apt-repository -y ppa:deadsnakes/ppa &&
-    sudo apt install -y python3.7 python3.7-dev python3-dev python3-pip ansible git aptitude
+    sudo apt install -y python3.7 python3.7-dev python3-dev python3-pip ansible git aptitude cpanminus
 
 sudo pip3 install setuptools
 
+# used by report generator to parse DOT file to ascii art
+sudo cpanm Graph::Easy
+
 echo "###################################"
 echo "Installing Umbra"
 echo "###################################"
@@ -38,4 +41,10 @@ cd ..
 sudo python3.7 setup.py install
 cd ..
 
+echo "##################################################"
+echo "Setup dockprom v3.17.1 for UI monitoring dashboard"
+echo "##################################################"
+
+git clone -b v3.17.1 https://github.com/stefanprodan/dockprom.git
+
 sudo usermod -aG docker $USER

docs/source/architecture.rst

@@ -14,8 +14,10 @@ Umbra was designed having in mind the following guidance:
 Components
 **********
 
-Umbra has three independent components:
+Umbra has five independent components:
 
 * design: defines APIs to implement the topology and events that will be respectively deployed and triggered when testing a blockchain platform. 
-* broker: main component, responsible for the orchestration and management of the scenario (topology and events)  
+* broker: main component, responsible for the orchestration and management of the scenario (topology and events)
 * scenario: the actual interface that deploys the topology (i.e. network, containers, virtual switches)
+* agent: runs as one of the "peer" in the blockchain network. It can be used to generate stimulus to the network like interrupting a running blockchain transaction (via ``iperf``) and replay packets (via ``tcpreplay``).
+* monitor: runs on the host machine. Used to monitor the status/metrics of both host and containers

docs/source/example.rst

@@ -4,6 +4,7 @@ Example
 To examplify how Umbra can be utilized, an extension was coded to support Hyperledger Fabric v1.4.
 The details of how that was coded are exposed in the Extensions section. Here is just the explanation of what the Fabric example realizes.
 
+.. image:: /imgs/imgs/example_fabric_topology.JPG.JPG
 
 Building
 ********
@@ -47,11 +48,11 @@ The executable file named run.sh in the umbra/examples/ folder contains the comm
 
 Having that executed, all the instantiation of components from the saved FabricTopology will take place (i.e., peers, orderers, links, etc) and events will be called on them. 
 
-The commands bellow respectivelly start and stop the example experiment with Fabric.
+The commands below respectively start and stop the example experiment with Fabric.
 
 .. code-block:: bash
 
-    $ sudo -H ./run.sh start -c ./fabric/fabric_configs/config_fabric_simple.json 
+    $ sudo -H ./run.sh start -c ./fabric/fabric_configs/Fabric-Simple-01.json
 
     $ sudo -H ./run.sh stop
 
@@ -72,4 +73,133 @@ Modifying
 *********
 
 To modify the Fabric experiment, it is just needed to modify the build_configs.py file, changing how the FabricTopology instance is built, besides changing how the events are scheduled.
-If new orgs are added, the file fabric.py inside base_configtx folder needs to be modified accordingly to define the policies in configtx needed for the creation of the Fabric requirements.
+If new orgs are added, the file fabric.py inside base_configtx folder needs to be modified accordingly to define the policies in configtx needed for the creation of the Fabric requirements.
+
+
+Changing environment during runtime
+***********************************
+
+We will use the broker's environment plugin (``umbra/broker/plugins/env.py``) to generate event that modifies the environment behavior. Refer `build_configs <https://github.com/hyperledger-labs/umbra/blob/master/examples/fabric/build_configs.py>`_.
+
+Following are example usecases:
+
+.. code-block:: python
+
+	# 1. Kill a container
+	ev_kill_container = {
+	    "command": "environment_event",
+	    "target_node": <peer_name>, # e.g. "peer0.org2.example.com"
+	    "action": "kill_container",
+	    "action_args": {},
+	}
+
+	# 2. Set mem limit
+	ev_mem_limit_peer1_org1 = {
+	    "command": "environment_event",
+	    "action": "update_memory_limit",
+	    "action_args": {
+	        "mem_limit": <amount_in_bytes>, # e.g. 256000000 for 256MB memory
+	        "memswap_limit": -1
+	    },
+	    "target_node": <peer_name>, # e.g. "peer0.org2.example.com"
+	}
+
+	# 3. Set cpu limit. More info at
+	# https://docs.docker.com/config/containers/resource_constraints/#cpu
+	# https://www.kernel.org/doc/Documentation/scheduler/sched-bwc.txt
+	ev_cpu_limit_peer1_org2 = {
+	    "command": "environment_event",
+	    "target_node": <peer_name>, # e.g. "peer0.org2.example.com"
+	    "action": "update_cpu_limit",
+	    "action_args": { # refer Docker docs for these values
+	        "cpu_quota": 10000,
+	        "cpu_period": 50000,
+	        "cpu_shares": -1,
+	        "cores": {}
+	    },
+	}
+
+	# 4. Update link resources
+	# Here, we change the resources of s0<-->peer1.org1 interface
+	# to bandwidth of 3Mbps, with 4ms delay, and packet loss rate of 10%
+	ev_update_link_res = {
+	    "command": "environment_event",
+	    "action": "update_link",
+	    "action_args": {
+	        "events": [
+	            {
+	                "group": "links",
+	                "specs": {
+	                    "action": "update",
+	                    "online": True,
+	                    "resources": {
+	                        "bw": 3, # Mbps
+	                        "delay": "4ms",
+	                        "loss": 10, #
+	                    }
+	                },
+	                "targets": ("s0", "peer1.org1.example.com")
+	            },
+	        ]
+	    },
+	}
+
+	# 5. Change link state, e.g. UP or DOWN
+	# Beginning of test all link should be up.
+	# Set the "online" key to either True or False
+	# Example below set the orderer interface to DOWN
+	ev_update_link_orderer_down = {
+	    "command": "environment_event",
+	    "action": "update_link",
+	    "action_args": {
+	        "events": [
+	            {
+	                "group": "links",
+	                "specs": {
+	                    "action": "update",
+	                    "online": <True|False>, # True=UP, False=DOWN
+	                    "resources": None
+	                },
+	                "targets": ("s0", "orderer.example.com")
+	            },
+	        ]
+	    },
+	}
+
+
+Creating stimulus in the network via agent
+******************************************
+
+`umbra-agent` currently supports `ping` to send ICMP echo, `iperf` to simulate heavy traffic, and `tcpreplay` to replay pcap packet. Only `ping` is tested thus far, other examples coming soon.
+
+.. code-block:: python
+
+	# Ping peer0.org1.example.com at 1 packet per second
+	# for 4 seconds
+	ev_agent_ping_peer0org1 = {
+	    "agent_name": agent_name,
+	    "id": "100",
+	    "actions": [
+	        {
+	            'id': "1",
+	            "tool": "ping",
+	            "output": {
+	                "live": False,
+	                "address": None,
+	            },
+	            'parameters': {
+	                "target": <target_node>, # e.g. "peer0.org1.example.com"
+	                "interval": "1",
+	                "duration": "4",
+	            },
+	            'schedule': {
+	                "from": 1,
+	                "until": 0,
+	                "duration": 0,
+	                "interval": 0,
+	                "repeat": 0
+	            },
+	        },
+	    ],
+	}
+

docs/source/starting.rst

@@ -1,3 +1,5 @@
+.. sectnum::
+
 Getting Started
 ===============
 
@@ -11,7 +13,7 @@ Umbra works on Ubuntu 18.04.
 To run the getting started example it is recommended a machine with Ubuntu 18.04 installed and containing 4 vCPUs (cores), 4 GB of memory, and at least 5GB of disk available.
 
 
-1. Install the Main Components
+Install the Main Components
 ******************************
 
 Umbra contains 3 python components (design, broker, scenario), the build script below installs the requirements and the components themselves.
@@ -32,7 +34,7 @@ Umbra contains 3 python components (design, broker, scenario), the build script
 Please note, the script above (build.sh) install docker-ce and adds the $USER to the docker group (no need to user sudo before the docker command). To enable this setting, logout and login again, or execute `su $USER` (to start a new session). You can test if it works simply running `docker ps -a`.
 
 
-2. Install the Fabric Requirements
+Install the Fabric Requirements
 **********************************
 
 As Umbra is plugin oriented (i.e., each Hyperledger project needs its own umbra-plugin), the build_fabric script below installs all the Fabric (v1.4) components needed to run the Umbra Fabric plugin.
@@ -48,7 +50,7 @@ As Umbra is plugin oriented (i.e., each Hyperledger project needs its own umbra-
     $ cd -
 
 
-3. Create the Fabric Configs
+Create the Fabric Configs
 ****************************
 
 The build_configs script below creates the config files for the Fabric scenario.
@@ -63,21 +65,32 @@ Open this file to see what is the scenario created, the topology and its events.
     $ cd -
 
 
-4. Run the Test
+Run the Test
 ***************
 
+**Optional Grafana Dashboard UI**: Skip this step if not needed. Before running the test, start the monitoring stack first using the command below.
+
+.. code-block:: bash
+
+    $ cd build/dockprom
+    $ docker-compose up -d
+    # If the command is succesful, then the dashboard should be up by now.
+    # Open a browser and point to <IP>:3000 to view the grafana dashboard.
+
+.. image:: /imgs/grafana_dashboard.JPG
+
 The run.sh script below executes the Fabric scenario (topology and events).
 In order to run the Mininet, a sudo password will be asked to run the Umbra scenario component.
 
+To run the test with more debug log, add (``-d``) flag. This is required to generate a proper report when stopping the simulation.
+
 .. code-block:: bash
 
     $ cd umbra/examples/
-
-    $ ./run.sh start -c ./fabric/fabric_configs/config_fabric_simple.json
+    $ ./run.sh start -c ./fabric/fabric_configs/Fabric-Simple-01.json <-d>
 
 
-
-4. Check the Test Logs
+Check the Test Logs
 **********************
 
 As the broker and scenario components save logs during their execution, they can be seen by the commands below.
@@ -89,11 +102,12 @@ As the broker and scenario components save logs during their execution, they can
     $ tail -f logs/scenario.log
 
 
-4. Stop the Test
+Stop the Test
 ****************
 
-The command below stops all the Umbra processes and clean their breadcrumbs. 
+The command below stops all the Umbra processes and clean their breadcrumbs. To generate the simulation report, add -r flag. Note that to generate a proper report, the simulation needs to run in debug mode (``-d``). Refer the previous section "Run the Test".
 
 .. code-block:: bash
 
-    $ ./run.sh stop
+    $ ./run.sh stop <-r>
+

docs/source/workflow.rst

@@ -77,4 +77,6 @@ Directory Structure
 * **umbra/common**: contains all the common application models (i.e., protocol buffer messages and services) implemented to be used by the other Umbra components.
 * **umbra/broker**: contains all the component source code to install and run the orchestration logic of umbra, i.e., how to receive a scenario configuration request, deploy it and trigger the events programmed in the scenario logic. The executable umbra-broker contains plugins, each one specified for a different blockchain platform it supports.
 * **umbra/design**: contains all the component source code to install and enable APIs for the configuration logic of umbra, i.e., how to specify different topology APIs to build the configuration needed for each blockchain platform to be executed by umbra-broker. 
-* **umbra/scenario**: contains all the component source code to install and run the plugin that enables Containernet to deploy the topology needed to execute a blockchain platform. 
+* **umbra/scenario**: contains all the component source code to install and run the plugin that enables Containernet to deploy the topology needed to execute a blockchain platform.
+* **umbra/agent**: contains source code related to umbra-agent executable
+* **umbra/monitor**: contains source code related to umbra-monitor executable