use Benchmark.pm?

merlin's picture

They have: 410 posts

Joined: Oct 1999

to measure the execution time of a perl-script i use Benchmark

use Benchmark;
$t0 = new Benchmark;  
(your code here)  
$t1 = new Benchmark;
$td = timediff($t1, $t0);
print "the code took:",timestr($td),"\n";
'

and get four numbers out of it:
- wallclock seconds ('real' seconds passed)
- usr
- sys
- CPU (the sum of usr and sys)

now: what do 'usr' and 'sys' tell me? what's the difference between them? i guess they're things from the unix-world? and i'm not used to those things... and how are the wallclock-secs and usr/sys/CPU related?

is there a way to format the output, ahm no, there is a way to format the output. where?

thank you for any help!

They have: 601 posts

Joined: Nov 2001

Does the following document answer your questions?

http://search.cpan.org/doc/GSAR/perl-5.6.1-TRIAL3/lib/Benchmark.pm

merlin's picture

They have: 410 posts

Joined: Oct 1999

wow, that's what i call a fast answer! Smiling

no, i already looked through this document. it answers a lot of questions i didn't eaven know i would have. but not the one i have.

in one sentence:
what means usr, sys and wallclock-seconds?

They have: 601 posts

Joined: Nov 2001

usr and sys are two times returned from the system times command (unix varianets OSs). Basically it returns the user time and the system time. These aren't important, and on any decent server you will probably get a system time of 0 very frequently.

cpu time is the one you really should be interested in - the final number. This is the one you should use to make your comparisons.

- wil

merlin's picture

They have: 410 posts

Joined: Oct 1999

so... what's the difference between wallclock-secs and CPU? i thought i should use the wcs for comparisons, although they're almost always 0 or 1 (not very comparable).

They have: 601 posts

Joined: Nov 2001

from the bechmark.pm docs:

"CPU seconds is, in UNIX terms, the user time plus the system time of the process itself, as opposed to the real (wallclock) time and the time spent by the child processes."

It's the CPU time you really need to pay attention to. I know that, but I can't really explain to you in a more clear way. I'll try and think of an example for you.

And that's all I can also tell you is that wallclock times are completly innacurate when it comes to bechmarking and should be avoided.

- wil

merlin's picture

They have: 410 posts

Joined: Oct 1999

now that i know that (thank you!): is there an easy way to format the output? with using timestr($td) it puts out those 4 numbers with some words.
along your explanations i only need CPU and would like to add words on my own. how can i do that?

They have: 601 posts

Joined: Nov 2001

Um. There's no way to format the output by default, but a quick hack would do it. Can you give me an example of the output and I'll see if I can write a regex to extract only the CPU usage.

- wil

merlin's picture

They have: 410 posts

Joined: Oct 1999

my output is:
1 wallclock secs (0.26 usr + 0.00 sys = 0.26 CPU)'

thank you Wil for your help, i'm really glad 'bout it!!

They have: 601 posts

Joined: Nov 2001

Interesting. The way I would do this using another script in Perl to format the results would look something like this:

chomp($output);
($junk,$cpu_time) = split (/\=/, $output);

print $output;
'

- wil

They have: 601 posts

Joined: Nov 2001

If that doesn't work for you; I've thought of a rather long-winded way of supressing the output of benchmark.pm instead of using another script.

Let me know how it goes.

- wil

merlin's picture

They have: 410 posts

Joined: Oct 1999

i will check your suggestion on monday, i'm going to the mountains, snowboarding now. thanks!

They have: 601 posts

Joined: Nov 2001

Cool! I've just come back from Courcheval, south France. Where you heading? You going for the 'last snow', huh?! Have fun!

merlin's picture

They have: 410 posts

Joined: Oct 1999

yeah, last snow. the very very last snow. i went to arosa, switzerland, quite nice there.

back to the topic.
it worked fine, but you should use chop instead of chomp to delete the last sign. now i have what i was looking for but i'm thinking of going even further:
saving the data into a file to build a kind of statitics of the access-rate. but i will think of that and if i'm encountering any difficulties i feel free to post them here... Smiling

thank you!!

They have: 601 posts

Joined: Nov 2001

I'm glad you had a good time! While you've been away, I've been thinking...

Here is another method, a very generic method and a method which is not very efficient.

Short of modifying Benchmark.pm to produce usable results as a data return rather than a mixture of prints along with variable, hash, and array reference returns, depending onwhich functions are called, it is possible to redirect Standard Output to an alias filehandle then later restore Standard Output to a console.

This is not difficult to do and affords some options. However, as said, this is not all that efficient. Nonetheless, this would allow all functions in Benchmark to be employed while suppressing printing to a console; portability is had.

A trade-off is having to write code which will parse returns per Benchmark function called; printed results vary greatly.

My test script shows one method of a number of methods. This simply redirects Standard Output to a file, which could be a temporary file later unlinked. Results are printed to a file rather than a console. Parsing of this file would yield only those results of interest.

For my example I include a print command for the "CPU" element of an array reference returned by Benchmark. You will note this is actually duplicated; once by Benchmark's printing and again by my syntax. It would be just as easy to leave out my array reference print and parse general returns; this affords a wide selection of desired output to be printed and affords all Benchmark functions to be employed and printed.

It is clear this problem of module authors mixing prints to Standard Output with pure variable returns, is a nuisance. Logic dictates a good module writer would provide options as to how returns are presented; console versus variables.

Personally, I would write a custom benchmark module which produces returns as I need, as a variable or an array, both of which could be easily accessed, should this become important.

#!perl

print "Content-type: text/plain\n\n";

use Benchmark;

open(REDIRECT, ">&STDOUT");
open(STDOUT, ">test.txt") or die "Standard Output Redirect Failed: $!";

$results = timethese (1000000,
  {
   'Test_One' =>
'$variable_one = "variable one";',
  } );

foreach $key ( keys %$results )
{ print "CPU $results->{$key}[1]"; }

close(STDOUT) or die "Standard Out Closure Failed: $!";

open(STDOUT, ">&REDIRECT") or die "Standard Output Restoration Failed: $!";

close (REDIRECT) or die "Redirect Closure Failed: $!";

open (RESULTS, "test.txt") or die "Open Of Data Results Failed: $!";

while (<RESULTS>)
{
  if (index ($_, "CPU") == 0)
   { print $_; }
}

close (RESULTS) or die "Results Closure Failed: $!";

exit;



TEST.TXT PRINTED RESULTS:
_________________________

Benchmark: timing 1000000 iterations of Test_One...
  Test_One:  1 wallclock secs ( 0.61 usr +  0.00 sys =  0.61 CPU) @ 1639344.26/s (n=1000000)
CPU 0.61


CONSOLE PRINTED RESULTS:
________________

CPU 0.61
'

I hope this helps!

- wil

merlin's picture

They have: 410 posts

Joined: Oct 1999

do i understand you right:
you don't really understand, why the module-author is using a defined output like 1 wallclock-secs (..usr + ..sys).......? i was quite surprised too. wouldn't a normal thing just put out some numbers? and that is what you are doing in your code sample?
i'm a little bit puzzled at this point. ???

They have: 601 posts

Joined: Nov 2001

Yes. IMO, the module should output an array or a hashref of numbers so you can pick and choose how to format the output. Hardcoding the output in a print statement is very weak. It would be far more flexible to output into a hashref that could be incorporated into a database or another program.

- wil

Want to join the discussion? Create an account or log in if you already have one. Joining is fast, free and painless! We’ll even whisk you back here when you’ve finished.