DrAFTS -- Durability Agreements From Time Series for AWS Spot Instances
DrAFTS is an acronym for Durability Agreements From Time Series.
Using historical pricing data from AWS and Quantile Bounds Estimation from Time Series (QBETS) -- a
non-parametric time series forecasting methodology for predicting bounds on
future values -- this site provides predicted bid prices (in US dollars) and
time durations (in hours) with probabilistic guarantees. Each prediction is
of the form
- time stamp, bid price (probability of bid success), duration
(probability of longevity)
where
- time stamp is the time
at which the prediction was made,
- bid price is the upper bound on
price that will likely be higher than the next bid,
- probability of bid
success is the probability that the bid will succeed (based on previous
predictions)
- duration is the lower bound on the future time frame
(beginning at the time stamp) over which the bid will continue to be greater
than other bids
- probability of longevity is the probability that the
duration will exceed the quoted duration (based on previous predictions)
For example, a prediction of the form
- c4.4xlarge Wed Feb 25 05:30:42 2015 $1.1041 (0.96) 1.72 hrs (0.95)
should be interpreted
to mean that at Wed Feb 25 05:30:42 2015 a bid of $1.1041 an
c4.4xlarge instance type would be above the "spot price" (with
probability 0.96) at which AWS can terminate the instance for at least
1.72 hours (with probability 0.95).
NEW: Minimum Bid Calculator
DrAFTS computes the minimum value that can be specified as the "AWS Maximum Bid" such that the instance will start and run for the predicted duration
with probability that is greater than or equal to 0.95. Any bid above the DrAFTS bid will also start and run with probability greater than or equal to 0.95 but for (potentially) a longer duration.
The Bid Calculator graph (accessed by clicking on "What Should I Bid") shows the relationship between maximum AWS bid and the duration that is statistically guaranteed with
probability at least 0.95. Using the graph it is possible to select the bid value to specify that assures a specific execution duration with probability 0.95 or greater.
Because Amazon obscures the global Availability Zone identities, the minimum bid calculator cannot determine what Availability Zone you
should use -- only that the bid that it computes will be correct for which is ever zone is currently least expensive.
To use the bid, request an instance in the Availability Zone with the lowest current price.
The site recomputes its
predictions periodically. Note that the the historical pricing data provided
by AWS is not necessarily complete. The algorithm attempts to compensate when it can, but some times are not represented or
have insufficient data necessary to make a reliable prediction. Note also that
AWS does not necessarily terminate a spot instance when the spot price exceeds
the bid price -- it is just entitled to do so. For more information on spot
instances, please see the
AWS
documentation for spot instances.
Using the Predictions as the basis for SLAs
The predictions are intended to
provide a statistical "Service Level Agreement" guarantee
associated with each spot
instance type in each availability zone. That is, bidding the predicted price
at the time stamp shown will "guarantee" that the spot price
does not exceed
the bid for at least the duration shown with a probability that is
the product (assuming
independence) of the two probabilities shown in each entry. Thus they do not reflect the instantaneous spot price or the duration it persists but rather a
prediction of the upper bound on the spot price and the duration that the
upper bound will persist.
Minimum Predicted Bids by Region
These predictions represent the minimum bid that guarantees the listed duration with probability at least 0.95.
Predicted Bid Prices by Instance Type for each Region
Predicted Bid Prices by Region for each Instance Type
If you are running an instance in a launch group and there is an
active instance in that group, AWS will attempt to co-locate the new instance
in the same availability zone with the running instance if the bid
price is high enough for that availability zone. These prices reflect the
bid price necessary to achieve an SLA per availability zone.
HOWEVER, AWS does not use the same mapping of availability zone names to
physical resources for each user. Thus these predictions reflect what a
"random" user sees in terms of availability zone names. If you wish to use
them, you'll need to try and figure out how they map to the
availability zone names AWS shows to your specific user ID.
This work is part of the Aristotle Federated Cloud Project sponsored by NSF.