Developing a Stone Soup pull request

[narykov]

What we provide:

IPLF filter:
IPLFKalmanUpdater, iterated posterior linearisation filter
slr_definition, statistical linear regression function
GeneralLinearGaussian measurement model that consumes explicitly specified transition matrix and bias

IPLS smoother:
IPLSKalmanSmoother, iterated posterior linearisation smoother
AugmentedGaussianStatePrediction, the state prediction class that can host cross-covariance
LinearTransitionModel, a transition model that consumes explicitly specified transition matrix and bias
AugmentedKalmanPredictor, predictor that uses the bias vector in LinearTransitionModel
AugmentedUnscentedKalmanPredictor that generates the cross-covariance and is able to suppress prediction noise

This is just a first iteration, apparently we need to split the branch into two separate pull requests

[A-acuto]

On the IPLF: I need to write the tests for the TMA deliverable so I'll be adding those (e.g. sending to someone, @mrcfon)

[A-acuto]

In the last commit I've done for the TMA project I added the IPLF functions and tests.
In case you want to take a look @mrcfon

[mrcfon]

IPLF PR: dstl#1016

[mrcfon]

IPLS failes the tests "test_kalman_smoother" in https://github.com/mrcfon/Stone-Soup/blob/an-ipls/stonesoup/smoother/tests/test_kalman.py", both with the SingleHypothesis and the MultipleHypothesis. I'm not sure about the latter, but the former is supposed to work.
It seems that we get an error when we compute the SLR on the measurement model, as we get "No measurement model specified". I suspect we're not using one of the Augmented states, even if the input type of the state into the SLR function is the same as in the example (GaussianStateUpdate).
@narykov can you spot the problem? Can we fix it in any ways? If not, we may just create an ad-hoc test for IPLS.

[narykov]

I downloaded the branch (and I can run the tutorials), but when I run 'test_kalman.py', the test suite is not detecting any tests to run:

Launching unittests with arguments python -m unittest /Users/alexeynarykov/Workspace/GitHub/Stone-Soup-an-ipls/stonesoup/smoother/tests/test_kalman.py in /Users/alexeynarykov/Workspace/GitHub/Stone-Soup-an-ipls/stonesoup/smoother/tests

Ran 0 tests in 0.000s
OK
Process finished with exit code 0
Empty suite
Empty suite

WAIDW?

[mrcfon]

Weird. Try to run this https://gist.github.com/mrcfon/6cc4e1af5c48c25a9e41f9ef504c4e2c
It tests IPLS with single and multiple hypotheses as test_kalman.py, without using pytest.

[A-acuto]

A couple of comments:

1. stone soup tests are run using pytest; indeed if you do python -m unittest .\stonesoup\smoother\tests\test_kalman.py runs 0 tests, while with python -m pytest .\stonesoup\smoother\tests\test_kalman.py it properly runs.
2. I don't understand the gist @mrcfon posted, I would copy as much as possible the existing code present for the other smoother test, and amend where the two components differ otherwise I think it will be much more tricky to spot why it is failing.

[mrcfon]

A couple of comments:

1. stone soup tests are run using `pytest`; indeed if you do `python -m unittest .\stonesoup\smoother\tests\test_kalman.py` runs 0 tests, while with `python -m pytest .\stonesoup\smoother\tests\test_kalman.py` it properly runs.

2. I don't understand the gist @mrcfon posted, I would copy as much as possible the existing code present for the other smoother test, and amend where the two components differ otherwise I think it will be much more tricky to spot why it is failing.

My replies:

1. I could run test_kalman.py with pytest from the terminal (with the env activated) or in pycharm (just pressing the run button as for a standard script). I don't know why Alexey is experiencing that problem.
2. Assuming the problem relates to pytest, I just copy-pasted the test in a normal script (just for IPLS). Once we found the way to make it work, I'll change the related test in test_kalman.py for the PR.

[narykov]

I fixed the measurement model issue (this already may require writing a separate test since now IPLSKalmanSmoother requires measurement_model in its initialisation since Detections are not equipped with their measurement models). However, now the test wants to work with Prediction instances over 'smoothed_track2' as if measurements were skipped. @mrcfon, could you point me at the workaround you developed working with Angel?

[mrcfon]

If I understood correctly, you're referring to the estimations within the measurement gaps, that were defined using GaussianStatePrediction in the first version of my code, and I had to turn them into GaussianStateUpdate to make them work properly, see line 38 in https://github.com/mrcfon/Stone-Soup-ESA/blob/main/stonesoup/space_manoeuvres/utils/general.py

[narykov]

I think I fixed everything, but this line in the test messes things up:

track[1] = GaussianStatePrediction(pred.state_vector, pred.covar, timestamp=pred.timestamp)

It forces the latest prediction (from the loop above) to be at position 1 in the track, which I don't understand why. Basically, this results in a track with timestamps like this:

[datetime.datetime(2024, 5, 16, 15, 23, 33, 485365),
datetime.datetime(2024, 5, 16, 15, 23, 37, 485365),
datetime.datetime(2024, 5, 16, 15, 23, 35, 485365),
datetime.datetime(2024, 5, 16, 15, 23, 36, 485365),
datetime.datetime(2024, 5, 16, 15, 23, 37, 485365)]

and the whole thing breaks down in 'smoothed_track2' on the first true IPLS iteration.

This is what the 'smoother/kalman.py' looks like after adjustments, feel free to test it.

[mrcfon]

I replaced 'smoother/kalman.py' with this, but it keeps giving me the same error, see below. @narykov did you change anything else?

[narykov]

this already may require writing a separate test since now IPLSKalmanSmoother requires measurement_model in its initialisation

Something like this:

smoother = IPLSKalmanSmoother(measurement_model=meas_model, transition_model=trans_model)

Sorry I didn't make it more clear. The measurements in the test are generated without a measurement model (which is quite unusual):

detections = [Detection(m, timestamp=timest) for m, timest in zip(measurements, times)]

and thus if the smoother is initialised without an explicitly specified measurement model, there is indeed no measurement model to be used.

[mrcfon]

IPLSKalmanSmoother(measurement_model=meas_model, transition_model=trans_model)

Thanks. IPLS now passes the test using SingleHypothesis. I doesn't pass the test with MultipleHypothesis because 'MultipleHypothesis' object has no attribute 'prediction'.
That's probably out of scope, I'd suggest pushing it as it is. Do you agree @narykov, @A-acuto?

[mrcfon]

IPLF PR: dstl#1016

@narykov, are there any concerns about the comments? Can I accept them all?