“I never castle” - Tom, chess player
“I never expand” - printf, Starcraft pro-player
Constraints lead to innovation. Without constrains, the road is paved and the best you can do is optimize and innovate on top of existing paradigms. That’s how, without having the option to implement a proper server-side service component, I was forced to design a way to implement partial rollout on the client side.
I teamed up with one of my colleagues and laid out the specs we wanted for our partial rollout feature: as a delivery squad, we wanted to give early access to a feature to a group of a limited amount of customers, so that we can monitor incidents while we increase the amount of customer that have access, or roll back the feature completely, in case of an unforeseen major failure:
allocate a configurable percentage of total traffic that would experience the new feature - a percentage that we can change between 0 to 100, without redeploying the application
don’t shuffle the users between the groups on a given configuration. Allocation should be sticky, so that users don’t see different features every time they log in
because of organizational constraints, we couldn’t build any backend service components. We had to figure out a quick, good-enough solution, in the client side code.
We were lucky enough to find some piece of customer data that’s monotonic: unique and increasing - uniformly distributed. Once we had that, we were able to use it to distribute the individual users in a fixed amount of buckets (sharding), i.e. by modulo-operation. Imagine you have 200.000 users, which are allocated 200000 unique ID-s, integers, in consecutive sequence. You can choose to allocate them to 100 buckets, by computing their Modulo 100:
We then found a problem with this design - for every new feature we wanted to partially roll out, it was always the same users that got to be targeted:
If your user identifiers are not countable and uniformly distributed, then you will need to come up with an additional stateless transformation that would directly convert the identifiers to a monotonic sequence. If the hash function does not distribute the users evenly into the buckets, you’d risk having significantly more traffic in one or more of the buckets, and thus, the percentage you configure can be a bad underestimation or overestimation.
The crux of this design is that it only works if the traffic is evenly distributed in the buckets, and you are comfortable having the frontend decide the configuration of your features or version of the features of your app.
For more advanced designs, you’d want to include this in your load balancer and handle the allocation logic there. Then, the frontend app can request the feature configuration, which may change based on how the user session is assigned to the control groups of your new features. This approach is not only more powerful but also more secure, as you can verify the feature configuration on the server, preventing malicious users from manually accessing a feature.
Creativity isn't always about revolutionary solutions. In this case, it was about a simple, easy-to-implement change in the frontend that helped us roll out gradually without much development effort.
