Issue with folium rendering HTML in notebooks using papermill

[lwasser]

Please add a code sample or a nbviewer link, copy-pastable if possible

Hi Colleagues.
I'm having some issues with how folium produces html outputs in the JSON of a jupyter notebook.
i posted my issue here in the papermill repo first:

nteract/papermill#482

@MSeal suggested that i post here.

I've been using papermill for a while to execute jupyter notebooks in a CI framework.

I'm running into an odd (i think?) bug. When i run a notebook with an interactive element that will require an iframe - case in point creating a map using leaflet (folium for python), it seems to be adding some code that makes the output render incorrectly.

I just ran one of my notebooks. Prior to running papermill, the iframe embed looked like this:
The important piece here is that the src=data:text/html part is correctly populated in the json.

"<div style=\"width:100%;\"><div style=\"position:relative;width:100%;height:0;padding-bottom:60%;\"><iframe src=\"data:text/html;charset=utf-8;base64,

So folium renders plots correctly when run in a notebook. HOwever, When I run papermill and execute a notebook using papermill, I get the following below: notice that the src now begins with about:blank.

"<div style=\"width:100%;\"><div style=\"position:relative;width:100%;height:0;padding-bottom:60%;\"><iframe src=\"about:blank\" style=\"position:absolute;width:100%;height:100%;left:0;top:0;border:none !important;\" data-html=PCFET0NUWVBFIGh0bWw+C

@MSeal mentioned this could related to something associated with headless execution and browsers. this is a but out of my scope of knowledge but i wondered if you would be willing to help me understand what is happening and what a fix might be?

I am using folium 0.10.1. And am happy to provide any additional information below. I've tried this on mac and linux and get the same results.

Many thanks.

# Your code here

Problem description

nteract/papermill#482

[this should explain why the current behavior is a problem and what is the expected output.]

Expected Output

Output of folium.__version__

[Conengmo]

Hi @lwasser, this seems related to a change we made fairly recent: python-visualization/branca#66

In short what changed:

Before we would store the map data in iframe src field. This lead to issues when map data was large, which affected many users with Chrome.

After the change we store the map data in a data attribute, which are not limited in size. But now we need extra Javascript to load the data.

I don't know about Papermill, but it seems they don't run this part of the code.

A quick fix would be to not use Branca version 0.4.0, but fall back to 0.3.1. And then let's think about what a long term fix could be.

[lwasser]

wow @Conengmo THANK YOU for the quick reply. That simple downgrade fixed our build (we have a bunch of folium maps failing in various lessons). I think for the time being i'll update our docker container to pin to branca 0.3.1 . And i'll continue to watch this issue for a longer term fix to it.

i'm super appreciative of your help. I was hoping to get some lessons that use folium live by next Wednesday and this fix will do just that for us. I am happy to test any fixes in the future if that would be helpful as well.

[Conengmo]

Happy to help :) @MSeal mentioned in the other issue that he might provide some guidance on how we can resolve this, that would be welcome. I'm not that knowledgable on Jupyter Notebooks. I'm now thinking we either make rendering with the data attribute work, or we detect that we need to fall back to embedding the data in the src parameter.

[MSeal]

Ok, so I got a working example going and dug into the topic a bit more. So the content for rendering should all be there, it's just moved as @Conengmo pointed out. The key difference is that the data-html now requires the browser to execute code when rendering to rehydrate the interface with the data extract. If you load a notebook run by papermill with folium outputs and go to File -> Trust Notebook it will reload the page and execute the required extration to render. Before the data was embeded in the html so it didn't need a check to authorize execution.

This was done in Jupyter to prevent malicious javascript injection into your browser by 3rd parties, but it means that rendering a notebook executed in headlesss mode looks like a 3rd party to your browser when it renders the page initially. The same thing occurs for ipywidgets that have javascript components.

It may be worth adding a default html field which says "Click Trust Notebook to render this graph" to help guide people who hit this in the future if the javascript needed is blocked.

[Conengmo]

Thanks for looking into this, it's clear now what's going on.

That we now need trust is unfortunate. I don't think we should reverse the change to how we store the map data though. Before it would truncate the data, breaking things unexpectedly.

I do agree we have to inform our users they need to trust the notebooks to be able to see the maps.

• Add this info to our readme or documentation. Is it important enough to show on the readme?

• Add a field at the top of each example notebook. If a user opens one of the example notebooks they currently don't see the output if it's not trusted. Our nbviewer links still work fortunately though.

[Conengmo]

Or a better idea maybe: change the code where we unpack the data to show a warning if the unpacking failed.

[MSeal]

Or a better idea maybe: change the code where we unpack the data to show a warning if the unpacking failed.

I believe you can output two media types on your display object and make the plain/text version say something like "Please make this Notebook Trusted to view: File -> Trust Notebook" might be a nice work-around. Try it out, I think that will work to emit some guidance when the code is not trusted.