0.4.0

New Python API!

Following feedback from our users, we have updated our Python API to be more "pythonic". What this means is that we have removed the following constructs:

• StateBuilder classes: these have been removed completely form the API. Instead, where you would have passed an instance of your StateBuilder in the pipeline, you will bass your state class.
• Computation classes: these have been replaced by a computation function wrapped by an appropriate computation decorator
• PartitionFunction classes: these have been by a partition function wrapped by the partition_function decorator
• Encoder classes: these have been replaced by an encode function wrapped in the encoder decorator
• Decoder classes: these have been replaced by an encode function wrapped in the decoder decorator

This results in more concise code, in a format that is more familiar to regular Python programmers. Unfortunately this is a breaking change, which is not backwards-compatible. For a more comprehensive guide on how to upgrade your application to the new version of the API, look at How to update your Wallaroo Python applications to the new API.

Autoscaling

First public release!

Add workers to and remove workers from a running cluster. Wallaroo will automatically repartition your application state to match available capacity, without having to restart. Learn more on our website. And check out the documentation in "Running A Wallaroo Application".

Change log

Fixed

• Do not force shrink count to a minimum of 1 (PR #1931)
• Fix bug that caused worker joins to fail after the first successful round. (PR #1927)

Added

• Add "Running Wallaroo" section to book (PR #1914)

Changed

• New version of Python API based on decorators (PR #1833)

0.3.3

Wallaroo version 0.3.3 the first release of our Go API. The current version of the API should be considered a preview release. We're looking to hear from you. We are going to iterate and improve the API based on your feedback. So, give it a try and let us know what you think. To get started with Wallaroo Go API, please visit our Go API introduction.

0.3.3 also contains a fix to work around a bug in LLVM that can prevent you from building our Python runner application Machida. If you're using our Python API and building from source, we recommend you upgrade.

Fixed

• Fix shrink autoscale query reply (PR #1862)

Added

• Initial Go API (PR #1866)

0.3.2

Release Notes

Fixed

• Includes several updates to our documentation.

0.3.1

Release Notes

Fixed

• Includes several updates to our documentation.

0.3.0

Release Notes [0.3.0] - 2017-12-18

As part of this release we've introduced Wallaroo in Docker in order to provide an easy way for new users to get started with Wallaroo quickly. Visit our Docker setup instructions to get started with Wallaroo in Docker.

MacOS Users

If you are currently on MacOS Sierra or High Sierra, we recommend using Wallaroo in Docker. There have been a number of reports of several applications, including Wallaroo, causing kernel panics. This is currently under investigation.

Fixed

• Get ctrl-c to shutdown cluster after autoscale (PR #1760)
• Send all unacked messages when resuming normal sending at OutgoingBoundary (PR #1766)
• Fix bug in Python word count partitioning logic (PR #1723)
• Add support for chaining State Partition -> Stateless Partition (PR #1670)
• Fix Sender to properly dispose of files (PR #1673)
• Create ProxyRouters to all required steps during initialization

Added

• Add join for more than 1 worker simultaneously (PR #1759)
• Add stateless partition shrink recalculation (PR #1767)
• Add full support for partition routing to newly joined worker (PR #1730)
• Shutdown cluster cleanly when SIGTERM or SIGINT is received (PR #1705)

Changed

• Don't report a cluster as ready to work until node connection protocol has completed (PR #1771)
• Add Env as argument to source/sink builders (PR #1734)

Upgrading from Wallaroo 0.2.x

In order to upgrade to Wallaroo 0.3.0, you will need to upgrade ponyc and Wallaroo itself.

You will need to upgrade ponyc if you have a version older than 0.21.0. Below we have sections dedicated to upgrading on Ubuntu.

Upgrading on Ubuntu

Upgrading ponyc to 0.21.0

ponyc can be upgraded with the following command:

sudo apt-get install --only-upgrade ponyc

Verify you are now on the correct version of ponyc by running:

ponyc --version

You should get the following output:

0.21.0 [release]

Upgrading Wallaroo

Once you're on the latest ponyc, you're ready to switch over to Wallaroo 0.3.0. To do so, run the following command:

cd ~/wallaroo-tutorial/wallaroo
git fetch --tags
git checkout 0.3.0

In order to make sure you aren't running any binaries from the previous release, run the following commands:

cd ~/wallaroo-tutorial/wallaroo
make clean

You can then run the following commands to build the necessary tools to continue developing using Wallaroo 0.3.0:

cd ~/wallaroo-tutorial/wallaroo
make build-machida build-giles-all build-utils-cluster_shutdown

You're now set to continue developement with Wallaroo!

0.2.2

Release Notes

When we did the 0.2.1 release we missed didn't fully resolve the problem with building Machida on High Sierra. This release should fully resolve the problem.

Fixed

• Machida doesn't compile on MacOS High Sierra (PR #1726)

0.2.1

Release Notes

Apple moved some header files around when they released High Sierra and our Python runner application, Machida, no longer compiled. If you are a High Sierra user, you should update. Everyone else can skip this release.

Fixed

• Machida doesn't compile on MacOS High Sierra (PR #1698)

0.2.0

Release Notes

Fixed

• There were several fixes made as part of this release, we strongly recommend upgrading.

Changed

• Several documentation changes were made to create a better user experience.

• We switched over to the official ponyc compiler as opposed to our custom fork as part of this release. There are several updates that are required as part of this switch if you have previously installed Wallaroo 0.1.2.

Upgrading from Wallaroo 0.1.2

In order to upgrade to Wallaroo 0.1.2, you will need to upgrade ponyc, pony-stable, and wallaroo itself.

Due to our switch to the official ponyc compiler, you will need to update ponyc if you have our custom fork version or a version older than 0.20.0. We also recommend updating pony-stable as we have done all of our testing with the latest pony-stable release. Below we have sections dedicated to upgrading on OSX or Ubuntu.

Upgrading on OSX

Upgrading ponyc

If you installed ponyc via the commands provided in our documentation, below we provide the necessary information for you to get the latest version installed.

If you still have the ponyc directory from which you installed, run the following:

cd ~/ponyc-wallaroolabs-19.2.14
sudo make uninstall

You can then follow the official OSX install instructions for ponyc found here to complete the installation.

Verify you are now on the correct version of ponyc by running:

ponyc --version

You should get the following output:

0.20.0 [release]

---

If you no longer have the ponyc directory but still have ponyc installed, run the following commands:

sudo rm -rf /usr/local/include/pony
sudo rm /usr/local/include/pony.h
sudo rm /usr/local/lib/libponyc.a
sudo rm /usr/local/lib/libponyrt.a

You can then follow the official OSX install instructions for ponyc found here to install. To complete the installation you'd need to run the following to override the previous symlink:

brew link --overwrite --force ponyc

Verify you are now on the correct version of ponyc by running:

ponyc --version

You should get the following output:

0.20.0 [release]

---

Upgrading pony-stable

If you installed pony-stable via the commands provided in our setup instructions, below we have instructions for getting the latest version.

You can follow the official OSX install instructions for pony-stable found here to install. To complete the installation you'd need to run the following to override the previous symlink:

brew link --overwrite --force pony-stable

Confirm you have the latest version by running the following command:

stable version

You should see the following output:

0.1.1 [release]

Upgrading Wallaroo

Once you're on the latest ponyc and pony-stable, you're ready to switch over to Wallaroo 0.2.0. To do so, run the following command:

cd ~/wallaroo-tutorial/wallaroo
git fetch --tags
git checkout 0.2.0

In order to make sure you aren't running any binaries from the previous release, run the following commands:

cd ~/wallaroo-tutorial/wallaroo
make clean

You can then run the following commands to build the necessary tools to continue developing using Wallaroo 0.2.0:

cd ~/wallaroo-tutorial/wallaroo
make build-machida
make build-giles-sender
make build-giles-receiver
make build-utils-cluster_shutdown

You're now set to continue developement with Wallaroo!

Upgrading on Ubuntu

Upgrading ponyc

Remove the previous binary for ponyc via the following command:

sudo rm /usr/local/bin/ponyc

Run the following command to refresh the shell:

exec bash

You can then follow the official Ubuntu install instructions for ponyc found here to complete the installation.

Verify you are now on the correct version of ponyc by running:

ponyc --version

You should get the following output:

0.20.0 [release]

Upgrading pony-stable

If you installed pony-stable via the commands provided in our setup instructions, below we have instructions for getting the latest version.

Remove the previous binary for pony-stable via the following command:

sudo rm /usr/local/bin/stable

Run the following command to refresh the shell:

exec bash

You can then follow the official Ubuntu install instructions for pony-stable found here to complete the installation.

Confirm you have the latest version by running the following command:

stable version

You should see the following output:

0.1.1 [release]

Upgrading Wallaroo

Once you're on the latest ponyc and pony-stable you're ready to switch over to Wallaroo 0.2.0. To do so, run the following command:

cd ~/wallaroo-tutorial/wallaroo
git fetch --tags
git checkout 0.2.0

In order to make sure you aren't running any binaries from the previous release, run the following commands:

cd ~/wallaroo-tutorial/wallaroo
make clean

You can then run the following commands to build the necessary tools to continue developing using Wallaroo 0.2.0:

cd ~/wallaroo-tutorial/wallaroo
make build-machida
make build-giles-sender
make build-giles-receiver
make build-utils-cluster_shutdown

You're now set to continue developement with Wallaroo!

0.1.2

First announced public release

0.1.1

Updates based on soft-launch tester feedback.