Posted On: Sep 21, 2021
We’re excited to announce that in Amazon Forecast, you can now select the accuracy metric of your choice to direct AutoML to optimize training a predictor for the selected accuracy metric. Additionally, we have added three more accuracy metrics to evaluate your predictor – average weighted quantile loss (Average wQL), mean absolute percentage error (MAPE), and mean absolute scaled error (MASE).
Depending on the business operations and the accuracy metric that had traditionally been used in evaluating forecasts, customers preferred using different accuracy metrics for evaluating their predictors. Previously, customers understood the strength of their predictor by evaluating three accuracy metrics: weighted quantile loss (wQL) metric for each selected distribution point, weighted absolute percentage error (WAPE), and root mean square error (RMSE), but did not have control over the metric that AutoML optimizes model accuracy.
With today’s launch, you can direct AutoML to optimize the predictor for a specific accuracy metric of your choosing and Forecast will provide customers five different model accuracy metrics for you to assess the strength of your forecasting models. These are: average weighted quantile loss (Average wQL) of all selected distribution points, weighted absolute percentage error (WAPE), mean absolute percentage error (MAPE), mean absolute scaled error (MASE), and root mean square error (RMSE), calculated at the mean forecast. For each metric, a lower value, which are non-negative, indicates a smaller error and therefore a more accurate model.
To get started with this capability, read our blog to learn more about each accuracy metric and see Evaluating Predictor Accuracy. You can use this capability in all Regions where Forecast is publicly available. For more information about Region availability, see Region Table.