[Tsung] Sustained Load Capability

[proteus guy]

   First off I want to say congratulations & thanx for the really nice
performance/capacity testing tool. Its a perfect use for Erlang which I love
but rarely get the opportunity to utilize in practice. That said, there's a
critical missing capability that I really need that surprises me that its
not present.

    I build SOA-style systems and, as you might imagine with disparate
services, load balancers, multiple databases (looking forward to applying
tsung directly on the postgres servers!), caching, and program design
options - determining the overall capacity impact of a single change in the
system can be quite a challenge. One of my techniques for getting the best
balance of these variables is to run the system with a constant load plan
and gradually scale it up until something starts to give. Then I address
that issue (add another horizontal server, adjust the DB access, alter a
cache plan, etc...) and go back and repeat until I've reached a capacity
model that meets my business requirements. Another test I perform is to test
system resiliency by putting a load on the system then taking a component
out like an application server or causing a DB failover to occur. Then I
measure what impact it had on my system capacity and how long it took for
any fail-over mechanisms to kick in and restore service levels and
resiliency. Without a consistent load I can't meaningfully measure how many
user transactions were impacted by a failure consistently and system
recovery times will be highly variable.

    I see in Appendix.5 of the documentation that this capability is
explicitly missing but I really need this ability to define a sustained load
model to accomplish these standard capacity testing operations. Since your
method of determining concurrent users is completely dependent upon the time
it takes for the system to execute a session or its transactions and since
these are by definition highly variable due to the inherent impact of the
capacity of the system (and perhaps even the definition of some types of
dynamic transactions), its really impossible to just generate a stable
graduated load profile.

    I'm sitting here trying to think of how one might specify a load limit.
The most apparent one that comes to me would be able to add a  max_users
option to the users or arrivalphase attributes. You can specify this for a
client already. Would it be difficult to add a new level of granularity for
this parameter? Appreciate any ideas or suggestions.

    Again, thanx much for the great tool. Its quickly becoming my favorite.
I'm considering doing some different automated graph generation of my own
for comparing different test runs or being able to specify a fixed scale for
particular graphs so consecutive runs will be easier to compare. If anyone's
interested in this kind of thing I'll see if I can't release it here as
well. I'm afraid my code's more likely to be in python rather than perl,
however.

    best regards,

    -- Ben Scherrey
        Proteus Technologies Ltd. Thailand
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.process-one.net/pipermail/tsung-users/attachments/20071220/dcc1d11b/attachment.html