Commit d0501094 authored by Florent D'Halluin's avatar Florent D'Halluin Committed by Alexandre Duret-Lutz
Browse files

Update cbs.

CBS version: e97a404a44e5b5d19d4a869aa1f43524386c5b09.
See cbs/Changelog.
parent a9f7f2e1
2009-04-11 d-halluin <d-halluin@lrde.epita.fr>
Update cbs.
CBS version: e97a404a44e5b5d19d4a869aa1f43524386c5b09.
See cbs/Changelog.
2009-04-10 d-halluin <d-halluin@lrde.epita.fr>
Update cbs.
CBS version: d60af6daaf8fa20e8a9fe92017fb69f4db30f9ba.
See cbs/Changelog.
2009-04-09 d-halluin <d-halluin@lrde.epita.fr>
Update cbs.
CBS version: bbef600f610af475eda1ace19a8ca60a30c1faca.
See cbs/Changelog.
2009-04-06 d-halluin <d-halluin@lrde.epita.fr>
 
Add cbs (libbench).
......
Makefile.in
COPYING
INSTALL
aclocal.m4
autom4te.cache/
build-aux/config.guess
build-aux/config.sub
build-aux/depcomp
build-aux/install-sh
build-aux/ltmain.sh
build-aux/missing
configure
demo/Makefile.in
demo/bench/Makefile.in
libbench.tar
m4/libtool.m4
m4/ltoptions.m4
m4/ltsugar.m4
m4/ltversion.m4
m4/lt~obsolete.m4
2009-04-12 d-halluin <d-halluin@lrde.epita.fr>
Add convenient overloads for result/parameter.
* lib/bench.cc,
* include/cbs/bench/bench.hh: Do that.
2009-04-11 d-halluin <d-halluin@lrde.epita.fr>
Add timer benchmark demo.
Use it to bench CBS itself.
* README_LIBBENCH: Add some bench figures.
* configure.ac,
* demo/Makefile.am: Add new files.
* demo/timer-bench/Makefile.am: New.
* demo/timer-bench/main.cc: New.
2009-04-11 d-halluin <d-halluin@lrde.epita.fr>
Add Perl script for parameter extraction.
* bin/plot.pl: .
2009-04-10 d-halluin <d-halluin@lrde.epita.fr>
Document.
* README_LIBBENCH: Add typical usage section.
* include/cbs/bench/bench.hh (bench): Add comments.
* include/cbs/bench/timer.hh: Fix comments.
2009-04-10 d-halluin <d-halluin@lrde.epita.fr>
Allow disabling summary in export functions.
* include/cbs/bench/bench.hh (bench),
* lib/bench.cc (bench): Add option to enum, use it.
2009-04-10 d-halluin <d-halluin@lrde.epita.fr>
Improve option handling in export functions.
* include/cbs/bench/bench.hh (bench),
* lib/bench.cc (bench): Add get_verbosity().
2009-04-10 d-halluin <d-halluin@lrde.epita.fr>
Allow access to the internal timer.
* include/Makefile.am (nobase_include_HEADERS): Add new header.
* include/cbs/bench/bench_internals.hh: New.
Allow access to the internal timer.
* lib/bench.cc: Implement new functions.
* include/cbs/bench/bench_macros.hh: Fixes.
2009-04-09 d-halluin <d-halluin@lrde.epita.fr>
Tweak Makefile.am
* lib/Makefile.am: Simplify.
2009-04-09 d-halluin <d-halluin@lrde.epita.fr>
Add memplot interface.
* include/cbs/bench/bench.hh: Rename mem_plot() into memplot().
* include/cbs/bench/memplot.hh: Add interface.
* include/cbs/bench/timer.hh: Add needed getter.
* lib/bench.cc: Rename mem_plot() into memplot().
* lib/timer.cc: Add needed getter.
2009-04-09 d-halluin <d-halluin@lrde.epita.fr>
Implement wall time.
Add missing const.
* lib/timer_graph.cc: .
* include/cbs/bench/internal/timer_gathering.hh: .
* include/cbs/bench/internal/timer_gathering.hxx: .
* include/cbs/bench/internal/timer_graph.hh: .
* include/cbs/bench/timer.hh: .
* include/cbs/bench/timer.hxx: .
* lib/bench.cc: .
* lib/timer.cc: .
* lib/timer_gathering.cc: .
* lib/timer_graph.cc: .
2009-04-09 d-halluin <d-halluin@lrde.epita.fr>
Add .gitignore.
* .gitignore: Add generated files.
2009-04-06 d-halluin <d-halluin@lrde.epita.fr>
Add memory consumption to bench demo.
* demo/bench/main.cc (main): Add memory consumption.
2009-04-06 d-halluin <d-halluin@lrde.epita.fr>
Cosmetic fix.
* lib/bench.cc (bench): Change time parameter names.
2009-04-06 d-halluin <d-halluin@lrde.epita.fr>
Add TODO.
* TODO: New.
2009-04-06 d-halluin <d-halluin@lrde.epita.fr>
Update libbench.
Add:
- Date info.
- User time result.
- System time result.
* include/cbs/bench/internal/timer_gathering.hh: .
* include/cbs/bench/internal/timer_gathering.hxx: .
* include/cbs/bench/timer.hh: .
* include/cbs/bench/timer.hxx: .
* lib/bench.cc: .
* lib/timer.cc: .
2009-04-06 d-halluin <d-halluin@lrde.epita.fr>
Add .gitignore files in empty directories.
......
Libbench: measure performance of equiped C++ code.
Influence on equiped code (in time taken per 1 million measure):
2Ghz Intel Centrino, 1GB RAM:
2.5s for 1 task (1.25s in parent, 1.25s in child).
+3.5s for 100k different tasks.
i.e. using CBS to measure 100 different tasks for 100000 measures takes:
100 000 * 2.5s / 1 000 000 (= 250 ms)
+ 100 000 * 3.5s / 1 000 000 * 100 / 100 000 (= 0.350ms)
total: ~250ms.
Use the timer-bench demo to compute those figures on another system.
Typical usage:
(see demo/bench/main.cc)
________________________________________________________
#include <cbs/bench/bench_macros.hh>
int main()
{
BENCH_START("My benchmark", "My description");
BENCH_PARAMETER("My parameter", "My value");
do_some_tests();
BENCH_RESULT("My parameter", "My result");
BENCH_PRINT(bench::Options(bench::Options::VE_NORMAL,
bench::Options::FO_TEXT,
0));
BENCH_SAVE("benchmark.xml", bench::Options());
return 0;
}
void do_some_tests()
{
// Only one scoped task per scope.
BENCH_TASK_SCOPED("tests");
BENCH_TASK_START("tests_main");
{
BENCH_TASK_SCOPED("tests_inner_scope");
work();
// tests_inner_scope ends here.
}
// All tasks started with BENCH_TASK_START
// have to be stopped.
BENCH_TASK_STOP(); // tests_main
// tests ends here.
}
________________________________________________________
Note:
Parameters and results are std::string.
Use std::stringstream to convert other values:
#include <stringstream>
int n = 42;
std::stringstream n_str;
n_str << n;
BENCH_PARAMETER("n", n_str.str());
To do:
- Implement memplot
- Add more system info (e.g. hostname)
- Allow bench loops
vcsn:
- Equip bench
- Gather xml files
- Generate html
#!/usr/bin/env perl
use strict;
use warnings;
sub usage()
{
print "$0: Extract parameters and results from CBS XML files.\n";
print "\n";
print "Usage:\n";
print " $0 [options] [files...] \n";
print "\n";
print "Produce a gnuplot compatible output. Values for one file\n";
print "are printed on a single line and separated by whitespace.\n";
print "Parameters come before results, and both are sorted in lexical\n";
print "order. See output for details.\n";
print "\n";
print "Note: Non-existant or incorrect (containing whitespace) values\n";
print " default to 0.\n";
print "\n";
print "Options:\n";
print "\n";
print " -h: Display this help message.\n";
print "\n";
print " -p <parameter>:\n";
print " Extract <parameter> from the parameters.\n";
print " Can be used multiple times.\n";
print "\n";
print " -r <result>:\n";
print " Extract <result> from the result.\n";
print " Can be used multiple times.\n";
print "\n";
print " -d <directory>:\n";
print " Include all .xml files in <directory>.\n";
print " Subdirectories are not included.\n";
print " Can be used multiple times.\n";
print "\n";
print " -l <file>:\n";
print " Include all parameters and results present in <file>.\n";
print " The values in <file> are _not_ included in the output.\n";
print " Can be used multiple times.\n";
print "\n";
print " -a:\n";
print " Extract all parameters and results from all files.\n";
print "\n";
print "Example:\n";
print "\n";
print " $0 -p \"test depth\" -r time -d ../bench/xml -l ../dummy.xml\n";
print "\n";
}
my %parameters;
my %results;
my @files;
my @dirs;
my @ref_files;
my $extract_all;
if (@ARGV == 0)
{
usage();
exit;
}
# Parse arguments
foreach (my $i = 0; $i < @ARGV; $i++)
{
if ($ARGV[$i] eq "-h")
{
usage();
exit;
}
if ($ARGV[$i] eq "-l")
{
$i += 1;
$ref_files[$#ref_files + 1] = $ARGV[$i];
next;
}
if ($ARGV[$i] eq "-a")
{
$extract_all = 1;
next;
}
if ($ARGV[$i] eq "-p")
{
$i += 1;
$parameters{$ARGV[$i]} = 0;
next;
}
if ($ARGV[$i] eq "-r")
{
$i += 1;
$results{$ARGV[$i]} = 0;
next;
}
if ($ARGV[$i] eq "-d")
{
$i += 1;
$dirs[$#dirs + 1] = $ARGV[$i];
next;
}
$files[$#files + 1] = $ARGV[$i];
}
# Read directories
foreach (@dirs)
{
if (opendir(DIR, $_))
{
my $dir = $_;
while (my $file = readdir(DIR))
{
if ($file =~ /\.xml$/)
{
$files[$#files + 1] = "$dir/$file";
}
}
closedir DIR;
}
else
{
warn "Cannot open directory opendir $_: $!";
}
}
# Handle -a
if ($extract_all)
{
$ref_files[$#ref_files + 1] = $_ foreach (@files);
}
# Process reference files
foreach (@ref_files)
{
if (open(my $in, "<", $_))
{
while (<$in>)
{
last if (/<\/bench>/);
next unless (/<parameter / or /<result /);
if (/<parameter name="([^"]*)" value="([^"]*)" \/>/)
{
$parameters{$1} = 0;
}
if (/<result name="([^"]*)" value="([^"]*)" \/>/)
{
$results{$1} = 0;
}
}
}
else
{
warn "Can't open reference file $_: $!";
}
}
# Print header
my @parameters_keys = keys %parameters;
my @results_keys = keys %results;
@parameters_keys = sort @parameters_keys;
@results_keys = sort @results_keys;
print "# Benchmark results\n";
print "#\n";
print "# Parameters:\n";
print "# * $_\n" foreach (@parameters_keys);
print "# Results:\n";
print "# * $_\n" foreach (@results_keys);
print "# Files:\n";
print "# * $_\n" foreach (@files);
print "#\n";
# Process files
foreach (@files)
{
if (open(my $in, "<", $_))
{
$parameters{$_} = 0 foreach (@parameters_keys);
$results{$_} = 0 foreach (@results_keys);
while (<$in>)
{
last if (/<\/bench>/);
next unless (/<parameter / or /<result /);
if (/<parameter name="([^"]*)" value="([^\s"]*)" \/>/)
{
$parameters{$1} = $2 if defined $parameters{$1};
}
if (/<result name="([^"]*)" value="([^\s"]*)" \/>/)
{
$results{$1} = $2 if defined $results{$1};
}
}
print $parameters{$parameters_keys[0]} if $parameters_keys[0];
foreach (1 .. $#parameters_keys)
{
print " $parameters{$parameters_keys[$_]}";
}
print " $results{$_}" foreach (@results_keys);
print "\n";
}
else
{
warn "Can't open $_: $!";
}
}
......@@ -45,6 +45,7 @@ AC_CONFIG_FILES([
include/Makefile
demo/Makefile
demo/timer/Makefile
demo/timer-bench/Makefile
demo/bench/Makefile])
AC_OUTPUT
......@@ -14,4 +14,4 @@
include $(top_srcdir)/cbs.mk
SUBDIRS = timer bench
SUBDIRS = timer bench timer-bench
......@@ -15,6 +15,7 @@
#include <iostream>
#include <sstream>
#include <list>
#include <cbs/bench/bench_macros.hh>
#define WAIT(Duration) \
......@@ -25,6 +26,17 @@
delete i; \
}
#define CONSUME(Memory) \
{ \
std::list<int> list; \
for (int j = 0; j < Memory; ++j) \
for (int k = 0; k < 250000; ++k) \
{ \
list.push_back(k); \
} \
list.clear(); \
}
void a();
void b(int);
void c(int);
......@@ -45,6 +57,8 @@ int main()
for (int i = 0; i < n; ++i)
e();
CONSUME(130 + iter * 20)
BENCH_STOP();
{
......@@ -62,6 +76,10 @@ int main()
std::string name = "benchmark_" + parameter.str();
BENCH_PRINT(bench::Options(bench::Options::VE_NORMAL,
bench::Options::FO_TEXT,
0));
BENCH_SAVE(name + ".out", bench::Options(bench::Options::VE_NORMAL,
bench::Options::FO_TEXT,
0));
......@@ -114,7 +132,7 @@ void e()
{
BENCH_TASK_SCOPED("Task E");
WAIT(3)
WAIT(1)
d(0);
}
## CBS, a C++ benchmarking suite.
##
## Copyright (C) 2007, 2009 The CBS Authors.
##
## This program is free software; you can redistribute it and/or
## modify it under the terms of the GNU General Public License
## as published by the Free Software Foundation; either version 2
## of the License, or (at your option) any later version.
##
## The complete GNU General Public Licence Notice can be found as the
## `COPYING' file in the root directory.
##
## The CBS Authors consists of people listed in the `AUTHORS' file.
include $(top_srcdir)/cbs.mk
noinst_PROGRAMS = demo-timer-bench
demo_timer_bench_SOURCES = main.cc
demo_timer_bench_LDADD = $(top_builddir)/lib/libbench.la
demo_timer_bench_CPPFLAGS = $(AM_CPPFLAGS)
// main.cc: this file is part of the CBS project.
//
// CBS, a C++ benchmarking suite.
//
// Copyright (C) 2007, 2009 Florent D'Halluin.
//
// This program is free software; you can redistribute it and/or
// modify it under the terms of the GNU General Public License
// as published by the Free Software Foundation; either version 2
// of the License, or (at your option) any later version.
//
// The complete GNU General Public Licence Notice can be found as the
// `COPYING' file in the root directory.
//
#include <iostream>
#include <cbs/bench/timer.hh>
#include <cbs/bench/bench_macros.hh>
using namespace timer;
Timer global_timer;
void recursion(int n)
{
BENCH_TASK_START("recursion");
if (n > 0)
recursion(--n);
BENCH_TASK_STOP();
}
int main()
{
int n;
for (n = 100000; n < 5000000; n *= 2)
{
BENCH_START("timer-scoped", "Empty scoped tasks.");
for (int i = 0; i < n; ++i)
{
BENCH_TASK_SCOPED("Scoped");
}
{
BENCH_STOP();
std::stringstream parameter;
parameter << n;
BENCH_PARAMETER("empty scopes", parameter.str());
BENCH_PARAMETER("tasks", "1");
std::string name = "timer-scoped_" + parameter.str();
BENCH_PRINT(bench::Options(bench::Options::VE_NORMAL,
bench::Options::FO_TEXT,
0));
BENCH_SAVE(name + ".out", bench::Options(bench::Options::VE_NORMAL,
bench::Options::FO_TEXT,
0));
BENCH_SAVE(name + ".xml", bench::Options());
BENCH_SAVE(name + ".dot", bench::Options(bench::Options::VE_MINIMAL,
bench::Options::FO_DOT));
}
}
for (n = 100000; n < 5000000; n *= 2)
{
BENCH_START("timer-start-stop", "Empty tasks started and stopped.");
for (int i = 0; i < n; ++i)