[ENH] AutoETS numerical stability issues

[TonyBagnall]

Describe the feature or idea you want to propose

Several design choices were made to make AutoETS stable, and these may well make it less efficient. This is from @alexbanwell1, previously in the docstring but moving it here. We can discuss, recreate and try improvements on this thread

Attempts to make this forecaster stable:
- Issues with Zero division Errors:
    - If any data points are non-positive, multiplicative options are excluded.
    - With numba fastmath=true, the compiler moves the operations around. This means
      zero division guards are ineffective. As such I have tried putting the guards
      in a separate fastmath=false function, with inline=true. This slows it down a
      lot though.
    - Need to make sure initialisation function never assigns slices of the data
    array
    - fixed bug where the first few values were being changed in the seasonality
      calculation array
- Issues with the nelder mead not finding good parameters,
  usually ending up with alpha approx 1:
    - Tested updating the initial conditions (initial level, trend,
      seasonality array) to be heuristically calculated across the whole
      array at the start.
      This seemed to fix some issues, but cause others.
    - Tested optimising over the initial conditions in the nelder-mead array.
      This didn't really help, although is how statsmodels does it.
    - Added guards to the nelder-mead algorithm to reflect points back in when they
      go above or below (0,1) to ensure parameters stay valid.
    - Added sigmoid function to output of nelder-mead to ensure parameters stay
      in (0,1).
    - Initialised the simplex array with reasonable starting values
      (a=0.4, b=0.25, phi=0.95, g=0.35)
- The algorithms sometimes produce really extreme forecasts of the order of 1e100
  larger than the data. I assume this is due to the guards on the zero division,
  but haven't been able to work out how to fix it.

Describe your proposed solution

work around short comings to improve speed without catastrophic fails

Describe alternatives you've considered, if relevant

No response

Additional context

No response

[alexbanwell1]

No longer unstable, just slower than we'd like, does match statsforecast in terms of performance currently

[TonyBagnall]

to try sort out #3297, I'm going to debug here. There are several possible causes for poor performance.

1. Starting conditions, particularly with short series
2. Differences in performance metric between multiplicative and additive
3. Always damping
4. Lack of constraints on the parameters
5. Problems with iterative

I think the problem is with the multiplicative models, so I will start just comparing additive features with ETS set to difference configurations. We do this with simulated data, for overall, but it could also be an issue with short series

First model is additive trend and error, no seasonality, one step ahead forecasts

aeon = ETS(error_type=1,trend_type=1, seasonality_type=0).fit(train)

sf = StatsForecast(
        models=[SF_AutoETS(model="AZN", season_length=1, damped=True)],  # <- disable multiplicative error
        freq="D",
        n_jobs=1,
    )

statsmodels=ETSModel(
                y,
                error="add",
                trend="add",
                damped_trend=True,
                seasonal=None,
                initialization_method="estimated",
            )
            res = mod.fit(disp=False)

            res = mod.fit(disp=False)

[TonyBagnall]

this experiment we get big errors with the first run short. Rerun 200 per length, it could be a thing, I believe SF and SM have specific algorithms for starting conditions, which for short series may be a thing.

[TonyBagnall]

run 300 to smooth more, this looks like a thing

[TonyBagnall]

the effect vanishes around 75

[TonyBagnall]

two possible causes:

1. off by one evaluation when no seasonality in ets_fit
2. poor initial estimates more influence when short series