JMeter, Another lesson (cookie manager)
So, I haven't given up on load-testing Tomcat on my hardware (w1100z). It just moves quite slowly. If a small detour is allowed here... a-ah, what the heck! Installed Windows XP Pro 64 bit edition, with 64 bit JRE to match. Win keeps on loosing keyboard and mouse. My current workaround is to boot machine with mouse and keyboard unplugged, and only plug when login screen is seen. Another thing I learned today was that I unnecessarily been stressing Tomcat and JVM it's running on by not having Cookie Manager in my test plan. The result was that for every request to the server a new session would be created. These sessions would, ultimately, flood the memory and adversely and unrealistically decrease performance, especially when throughput is high. Sigh... good thing or two is learned every time, but no solid results produced yet.
posted by denka at 12:48 AM
4 Comments:
Rather than respond to your comment on Cameron's blog, I thought I'd respond here. JMeter definitely has limitations when it comes to simulating more than 250 threads. It can be done if there's sufficient think time between each request. Though honestly, I havne't had to simulate more than 50 concurrent threads without any think time. I've provide an example. I worked a very well known online yellow page site. The peak concurrent request across the entire cluster was 40. That kind of load generated around 10million+ page views per day back in 2002. The traffic is much higher now, but the peak and average concurrent load isn't that high.
Of the three active developers, sebb, mike and myself, none of us have had to test truly insance concurrency with jmeter. By insane I mean 2K+ concurrent requests. To simulate that kind of load, requires a lot of hardware. I know one of the largest financial firms in boston has 400+ client machines to simulate high concurrent load. Their setup is rather complex, but all the systems have atleast 2 ethernet ports. 1 port is used to simulate load against the cluster and the other port is used to send the result of the tests. All of this runs on gigabit ethernet.
the bottom line is that testing 500 concurrent threads w/o any think time is very hard to do accurately and effectively.
peter lin
Thanks for your cosiderate reply. In my test simulating 500 concurrent slow connections I did not try to max the bandwidth out, I wanted to see 500 connections open at the same time, but getting very little data each at any given moment. I understand some DOS attacks are made this way? Also, this is what a MMORG server would have to handle were it to work with clients running on mobiles... This test in particular is of interest in context of NIO-based HTTP servers, like Grizzly, contrasted to usual "1 thread per open socket per connection" of Tomcat.
I've toyed with the idea of an NIO implementation for JMeter, but honestly, it's way beyond my expertise and skill. I've looked at the various attempts at NIO for tomcat over the last few years and I'm pretty sure I don't know enough to write a NIO HTTP client. I'm hoping if I'm lazy enough, someone on commons HTTPClient will write one and I can simply use it.
Sebb checked in a change (last 2 weeks) to simulate slow connections, so you may be able to use the nightly build? If that works for you great. I've tried simulating "real world" load in the past and it is a royal pain. What I had to do was borrow connections from friends and then kick off the test from 6-8 remote locations. Even then, it was only a rough estimate. In my case, I was working on a wireless portal, so we wanted to get a "rough" estimate of what "real world" traffic would be like. Though honestly, it wasn't that real. One thing we couldn't simulate is a hundred WAP phones using our website. To do that would require Sprint's engineers and cooperation.
Trying to accurately measure real work loads is much harder that one would expect. JMeter currently does use a 1 thread per user model, so that is definitely a limitation. Though compared to Mercury load runner and other test frameworks, I think it's acceptable. To the best of my knowledge, loadRunner also uses 1 thread to simulate 1 user.
peter lin
Wireless... exactly! Meanwhile I managed to generate at least 600 concurrent slow connections from a single instance of JMeter, and I could to twice as much :) The secret for me was decreasing the stack size. On Windows, I set it to -Xss32k, but JDK 1.5_06 on Fedora refused to start up with less than 48k, another mystery...
Post a Comment
<< Home