Kubernetes on AWS with kops

I evaluated ways to deploy an Kubernetes cluster on AWS and settled on kops, since it is well supported by the Kubernetes community and can set up HA clusters.

Extending Spark's Accumulators

Spark’s accumulators can be used for much more than just adding up numbers. Spark only ships with int/double accumulators, but you can always create your own by implementing AccumulatorParam. Let’s say we’re processing log files with Spark Streaming and we would like to create a running tally of the most used browsers.