Here’s an interesting talk by Aaditya Ramdas on “Sequential Estimation of Quantiles with Applications to A/B-testing and Best-arm Identification”
From the description:
Consider the problem of sequentially estimating quantiles of any distribution over a complete, fully-ordered set, based on a stream of i.i.d. observations. We propose new, theoretically sound and practically tight confidence sequences for quantiles, that is, sequences of confidence intervals which are valid uniformly over time. We give two methods for tracking a fixed quantile and two methods for tracking all quantiles simultaneously. Specifically, we provide explicit expressions with small constants for intervals whose widths shrink at the fastest possible rate, as determined by the law of the iterated logarithm (LIL).