Flaky tests

[MartinHjelmare]

Issue to track known flaky tests. Let's fix them!

Currently impacting a lot of CI runs:

• TestStatisticsSensor.test_initialize_from_database_with_maxage

Others:

• Geofency.test_gps_enter_and_exit_home
• TestRecorderPurge.test_purge_method
• TestTrendBinarySensor.test_up_using_trendline
• TestLiteJetTrigger.test_held_in_range_just_right
• TestCheckConfig.test_component_platform_not_found
• TestCheckConfig.test_config_component_platform_fail_validation
• TestVultrBinarySensorSetup.test_failed_hub
• TestCanarySensorSetup.test_setup_sensors
• TestComponentHistory.test_get_significant_states
• TestComponentHistory.test_get_significant_states_include_exclude_entity
• components.test_updater.test_disable_reporting
• TestMicrosoftFaceSetup.test_service_status_400
• tests.components.weather.test_darksky.TestDarkSky
• TestFolderWatcher.test_event

[ChristianKuehnel]

I have one more: TestCanarySensorSetup.test_setup_sensors

[MartinHjelmare]

The vultr test should be fixed by:
#11477

[MartinHjelmare]

CC @snjoetw about the canary test that is flaky.

[MartinHjelmare]

Canary test should be fixed by:
#11519

[ReneNulschDE]

I have one more: tests.components.weather.test_darksky.TestDarkSky

[ChristianKuehnel]

And another one: TestComponentHistory.test_get_significant_states_include_exclude_entity

[balloobbot]

There hasn't been any activity on this issue recently. Due to the high number of incoming GitHub notifications, we have to clean some of the old issues, as many of them have already been resolved with the latest updates.

Please make sure to update to the latest Home Assistant version and check if that solves the issue. Let us know if that works for you by adding a comment 👍

[MartinHjelmare]

All of these are not fixed yet I think.

[MartinHjelmare]

@mxworm looks like homematicip cloud is doing I/O in the first config flow test. It's not patched in the correct place:
https://travis-ci.org/home-assistant/home-assistant/jobs/440417606#L1102