ascii encoding failure under Python 3 on EC2 / Elastic Beanstalk

[bitprophet]

See description in this comment on #350 by @aleksihakli: #350 (comment)

---

Quick note: hide=True just skips the automatic mirroring to stdout/err; presumably if one manually were to try printing or decoding run(...).stdout it'd explode there too.

---

I'm wondering why this is breaking for @aleksihakli; is their environment a normal-ish EC2 instance with a human user at a terminal? Or is Elastic Beanstalk executing it in a more limited environment?

They're on Python 3.4, so Invoke isn't manually encoding the string, it's assuming Python 3's io writer classes are doing the correct thing. My best guess is that in this case something is triggering use of a writer class that does not do the correct thing; or that the outer environment has properties we haven't encountered yet.

Hoping Aleksi can shed some light here :)

[stevenhair]

I see the same issue with Invoke 0.13.0 on Python 3.5 running on CentOS 7 when running mocha tests for our Node builds.

Calling run('mocha specs') fails with UnicodeEncodeError: 'ascii' codec can't encode characters in position 58: ordinal not in range(128).

You're right that calling run with hide=True does not fail, but subsequently printing result.stdout will fail with the same error.

[bitprophet]

@stevenhair are you running those interactively or e.g. via CI of some kind?

The issue here is that in our testing, Python 3's default sys.stdout/err handle arbitrary unicode writes pretty well, so you've found some edge case. If you can provide more details (esp if there's a way for me to reproduce your test run locally) that'd be helpful - thanks!

[stevenhair]

It's part of our Jenkins CI build. I can confirm that running locally works without a problem on both OS X 10.11 and CentOS 7.1. However, when you run it on CentOS 7.2 or Ubuntu 14.04 in Docker (either locally or in CI), then it throws the error.

I made a proof of concept project here, and here's the build log illustrating the issue.

[stevenhair]

Actually, it may just be a problem running in Docker in general. If you build that docker machine and then manually run inv test, it errors as well.

[bitprophet]

OK, so another mark under "probably something specific to noninteractive use". Thanks for setting up the repro case, I'll try to use that when I look next!

[jefflage]

@bitprophet It seems the 0.12.3 milestone was skipped and 0.13.0 has been released. Is this bug still open?

[bitprophet]

@jefflage It is, milestones are a bit wonky pre-1.0. Github ticket status is a clearer indicator than milestone version numbers, fwiw. If I consider something closed/done/wontfix, I always mash 'close'.

I've been focusing on feature work for the last few months but intend to put a bunch of love into bugfixes (on all my projects) very soon now.

[stevenhair]

@bitprophet another clue: https://travis-ci.org/stevenhair/vigilant-potato/builds/187228305

It turns out that the default encoding for CentOS is ANSI. If you switch it to UTF-8, everything works fine.

[rolandgeider]

FYI, I had a very similar problem (docker + bower + invoke) and the solution was to change the locale in the docker container