How to evaluate TemporalFusionTransformer (TFT)

[NhutTien0905]

Hi everyone,

I’m currently working on a project using Temporal Fusion Transformer (TFT) for time series forecasting. My team is trying to determine which evaluation metrics are the most suitable for assessing TFT performance, especially considering its ability to output probabilistic and multi-horizon forecasts.

We would really appreciate insights on questions like:

• Which metrics work best for point forecast accuracy (e.g., MAE, RMSE, MAPE)?
• Which metrics are recommended for quantile / probabilistic forecasts (e.g., Quantile Loss / Pinball Loss, CRPS)?
• Are there any business-oriented or practical metrics that you’ve found useful when deploying TFT in production?
• Any pitfalls or common mistakes when evaluating TFT models?

If you’ve used TFT in production or research, we’d love to hear:

• What metrics you ended up using
• Why you chose them
• Any lessons learned

Thanks in advance!