View Rendering hangs under load in grails 3.1.7+ due to spring framework 4.2.6 upgrade

[davydotcom]

Starting with grails 3.1.7 production view rendering periodically hangs for long periods of time (60ms views turn into 60second loads), This is directly related to the upgrade to spring framework 4.2.6 from 4.2.5.

Steps to Reproduce

1. Run Grails 3.1.7+
2. Subject application war (tomcat) to load
3. Note page response times drop (hard to reproduce but seems to be related to concurrency).

Expected Behaviour

View (GSP) rendering should not pause or hang like it does since moving from spring framework 4.2.5 to 4.2.6 and 4.2.7

Actual Behaviour

Pages periodically hang for long periods of times making site appear slow and non responsive.

Environment Information

• Operating System: OS X
• Grails Version: 3.1.7+ Spring Framework 4.2.6.RELEASE +
• JDK Version: 8u91
• Container Version (If Applicable): Tomcat WAR

It has been confirmed that if using latest grails 3.1.10 and forcing spring framework down to 4.2.5 the view hang stops.

Workaround

Adjust build.gradle file:

configurations.all {
  resolutionStrategy.eachDependency { DependencyResolveDetails details ->
    if(details.requested.group == 'org.springframework' && details.requested.name != 'springloaded' ) {
            details.useVersion("4.2.5.RELEASE")
        }
  }        
}

[ctoestreich]

👍 We are testing the work around in an attempt to get off 3.1.6.

[graemerocher]

Is there any more debug information that can be provided? Like an example that reproduces the issue? We can report the issue on the Spring side but they are going to ask for an example

[ctoestreich]

I am going to lean on @davydotcom for these details as he has experienced this more frequently than we have. From talking to him recently he has also dove deeper into the spring code.

[davydotcom]

its very hard to reproduce on a small app but i can give a few more details to help. When a request is made to a gsp page with a controller/action, the controller/action run almost immediate in the logs. Then you get a hang up sporadically for sometimes a minute plus before a response is even sent. It is as if the view rendering layer is hung. A 60ms view will sporadically jump to minutes even though all data is there and minimal use of taglibs.. This primarily happens in production only tomcat 8

[graemerocher]

Hmm without further information I'm not sure how to progress this, I'll ask some of the Spring team if they have seen anything similar

[davydotcom]

Ill try to get more reproducible information this week but its a super hard one to consistently reproduce on a sample project because of its size

On Aug 23, 2016, at 2:06 PM, graemerocher notifications@github.com wrote:

Hmm without further information I'm not sure how to progress this, I'll ask some of the Spring team if they have seen anything similar

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub #10102 (comment), or mute the thread https://github.com/notifications/unsubscribe-auth/AABaEnkan29NazKJVeDJWhEhGzh-AIDKks5qizaOgaJpZM4Jj-Pm.

[mgkimsal]

Can you just hit a URL with "ab"? I've seen something similar when testing with ab last week (3.1.10, run-war). Hit a URL 500 times with, say, 40 clients, and, probably 10% of the time it would just hang... (may not be related but maybe that's a way to replicate?)

[davydotcom]

Sounds about right I'll try it

Sent from my iPhone

On Aug 24, 2016, at 7:33 AM, Michael Kimsal notifications@github.com wrote:

Can you just hit a URL with "ab"? I've seen something similar when testing with ab last week (3.1.10, run-war). Hit a URL 500 times with, say, 40 clients, and, probably 10% of the time it would just hang... (may not be related but maybe that's a way to replicate?)

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

[graemerocher]

Reported https://jira.spring.io/browse/SPR-14622 to get feedback from the Spring team

[graemerocher]

I have been trying to reproduce this with a simple app and have not been able too. Initially I thought I had reproduced it but it turned out to be an issue with ab on OS X:

http://stackoverflow.com/questions/1216267/ab-program-freezes-after-lots-of-requests-why

And I spent way too much wasted time trying to debug that before figuring out the issue was with ab/OS X and not Grails. I've tested with both ab and siege and the only freezing that occurs is when the OS runs out of ports (see above).

So right now I have no way to progress this forward nor provide feedback to the Spring team.

[ctoestreich]

This is not during a load tests and when we are simply running grails 3.1.6 with bootRun via gradle.

[graemerocher]

Could you please create a separate issue these comments and reports are polluting this thread as they are unrelated issues. One is spring loaded another relates to production mode without spring loaded with newer versions of Spring