PR: Add flaky

[impact27]

Sometimes, with setup_kernel(cmd) as client: fails. This repeats the failing tests at most 3 times.

[ccordoba12]

@impact27, could you mark test_simple_print as flaky to see if it passes with multiple repetitions?

[impact27]

It works now, flaky was failing because subprocess are acting up sometimes and were not properly closed.

[blink1073]

Perhaps instead of using flaky we should increase the timeout based on an environment variable that we can set on Travis. That would distinguish between slow starts and actual failures.

[impact27]

The setup timeout is 60 seconds so I am not sure what a reasonable timeout would be.

[impact27]

Wait actually it is not quite True:

while not os.path.exists(connection_file) \
         and kernel.poll() is None \
         and time.time() < tic + SETUP_TIMEOUT:
         time.sleep(0.1)

This waits for the connection file to exist but not for it to be written. Maybe the test should be: can we read the json file?

[blink1073]

Yeah that sounds right.

[blink1073]

LGTM, thanks!