Running a Federation Simulation (MNIST Example)

Here we will use an example flplan (bin/federations/plans/keras_cnn_mnist_10.yaml) that will work in conjunction with an example collaborators list (bin/federations/collaborator_lists/cols_10.yaml) for which entries already exist in the example local data config (bin/federations/local_data_config.yaml) enabling a predetermined sharding of the MNIST public dataset across 10 collaborators.

Setup and Installation

  1. Clone the repository onto a linux machine the has Python 3.5 or greater, and the virtualenv library installed.

  2. Enter the project root directory, and install the project with support for Keras models.

$ make install_openfl install_openfl_tensorflow

  1. Make an exact copy of the example network configuration to ensure one exists (it needs to be there, but in this case it’s contents are not important).

$ cp bin/federations/plans/defaults/network.yaml.example bin/federations/plans/defaults/network.yaml

Creation of Initial Weights

  1. Create the initial weights file by running the following command from the bin directory:

$ ../venv/bin/python create_initial_weights_file_from_flplan.py -p keras_cnn_mnist_10.yaml -c cols_10.yaml

Launch the Simulated Federation

  1. Again from the bin directory, kick off the simulation by running the following:

$ ../venv/bin/python run_simulation_from_flplan.py -p keras_cnn_mnist_10.yaml -c cols_10.yaml

Monitor the Progress

  1. You’ll find the output from the aggregator in bin/logs/aggregator.log. Grep this file to see results (one example below). You can check the progress as the simulation runs, if desired.

$ pwd                                                                                                                                                                                                                            msheller@spr-gpu01
  /home/<user>/git/openfl/bin
$ grep -A 2 "round results" logs/aggregator.log
  2020-03-30 13:45:33,404 - openfl.aggregator.aggregator - INFO - round results for model id/version KerasCNN/1
  2020-03-30 13:45:33,404 - openfl.aggregator.aggregator - INFO -        validation: 0.4465000107884407
  2020-03-30 13:45:33,404 - openfl.aggregator.aggregator - INFO -        loss: 1.0632034242153168
  --
  2020-03-30 13:45:35,127 - openfl.aggregator.aggregator - INFO - round results for model id/version KerasCNN/2
  2020-03-30 13:45:35,127 - openfl.aggregator.aggregator - INFO -        validation: 0.8630000054836273
  2020-03-30 13:45:35,127 - openfl.aggregator.aggregator - INFO -        loss: 0.41314733028411865
  --

Note that aggregator.log is always appended to, so will include results from previous runs.

Explore Modifications

  1. Perform a new simulation using 32 collaborators instead of 10 (using the plan, ‘keras_cnn_mnist_32.yaml’) to see how this effects the learning curve. Explore further modifications by copying and editing existing plans.