Update cbs.

CBS version: aef3055d95b926d6242ebbce64a128df044cbba2.
See cbs/Changelog.

ChangeLog

+2009-04-28  d-halluin  <d-halluin@lrde.epita.fr>
+
+	Update cbs.
+
+	CBS version: aef3055d95b926d6242ebbce64a128df044cbba2.
+	See cbs/Changelog.
+
 2009-04-16  d-halluin  <d-halluin@lrde.epita.fr>
 	Fix make check.

cbs/ChangeLog

+2009-04-26  d-halluin  <d-halluin@lrde.epita.fr>
+
+	Update README_LIBBENCH figures.
+
+	* README_LIBBENCH: Update figures.
+
+2009-04-26  d-halluin  <d-halluin@lrde.epita.fr>
+
+	Add data from vcsn tests.
+
+	* demo/vcsn/vcsn-quotient-fix-1.data,
+	* demo/vcsn/vcsn-quotient-fix.data,
+	* demo/vcsn/vcsn-quotient-orig.data,
+	* demo/vcsn/vcsn-quotient.plot: New.
+
+2009-04-26  d-halluin  <d-halluin@lrde.epita.fr>
+
+	Add plotting helpers to timer-bench.
+
+	* demo/timer-bench/get-results.sh,
+	* demo/timer-bench/timer-many-tasks-memory.plot,
+	* demo/timer-bench/timer-many-tasks.plot,
+	* demo/timer-bench/timer-recursion-memory.plot,
+	* demo/timer-bench/timer-recursion-time.plot,
+	* demo/timer-bench/timer-scoped-memory.plot,
+	* demo/timer-bench/timer-scoped.plot: New.
+
+2009-04-26  d-halluin  <d-halluin@lrde.epita.fr>
+
+	Update comparaison demo.
+
+	* demo/comparison/main.cc: Update.
+
+2009-04-26  d-halluin  <d-halluin@lrde.epita.fr>
+
+	Add memplot bench demo.
+
+	* configure.ac: Add Makefile.
+	* demo/Makefile.am: Add subdir.
+	* demo/memplot-bench/Makefile.am: New.
+	* demo/memplot-bench/main.cc: New.
+	* demo/memplot-bench/get-results.sh,
+	* demo/memplot-bench/results-memory.plot,
+	* demo/memplot-bench/results-time.plot: New.  Plotting helpers.
+
+2009-04-26  d-halluin  <d-halluin@lrde.epita.fr>
+
+	Add comparison demo for several profilers.
+
+	Works with CBS, gprof, kcachegrind.
+
+	* configure.ac: Add Makefile.
+	* demo/Makefile.am: Add subdir.
+	* demo/comparison/Makefile.am,
+	* demo/comparison/README,
+	* demo/comparison/main.cc: Add demo.
+
+2009-04-26  d-halluin  <d-halluin@lrde.epita.fr>
+
+	Remove obsolete demo scripts.
+
+	* demo/bench/demo.sh: Deleted.
+	* demo/bench/display.sh: Deleted.
+	* demo/bench/plot.sh: Deleted.
+	* demo/timer/demo.sh: Deleted.
+	* demo/timer/display.sh: Deleted.
+	* demo/timer/timer-demo.sh: Deleted.
+
+2009-04-26  d-halluin  <d-halluin@lrde.epita.fr>
+
+	(Memplot) Fix measure.
+
+	* lib/memplot.cc: Remove loop.
+
+2009-04-26  d-halluin  <d-halluin@lrde.epita.fr>
+
+	(Memplot) Remove brk measure.
+
+	* include/cbs/bench/memplot.hh,
+	* lib/memplot.cc: Remove memory_brk.
+
+2009-04-26  d-halluin  <d-halluin@lrde.epita.fr>
+
+	Update demos.
+
+	* demo/Makefile.am: Add new dir.
+	* demo/bench/main.cc: Add memplot, tweak.
+	* demo/memplot/Makefile.am,
+	* demo/memplot/main.cc: Add memplot demo.
+
+2009-04-26  d-halluin  <d-halluin@lrde.epita.fr>
+
+	Tweak timer main task name.
+
+	* lib/timer.cc: Add macro for name, change name.
+
+2009-04-26  d-halluin  <d-halluin@lrde.epita.fr>
+
+	Update TODO.
+
+	* TODO: Update.
+
+2009-04-26  d-halluin  <d-halluin@lrde.epita.fr>
+
+	Add memplot implementation.
+
+	* configure.ac: Add new Makefile.
+	* include/cbs/bench/memplot.hh: Update interface.
+	* lib/Makefile.am: Add new file.
+	* lib/bench.cc: Add implementation.
+	* lib/memplot.cc: Add implementation.
+
 2009-04-12  d-halluin  <d-halluin@lrde.epita.fr>
 
 	Add convenient overloads for result/parameter.

cbs/README_LIBBENCH

 Libbench: measure performance of equiped C++ code.
 
-Influence on equiped code (in time taken per 1 million measure):
+## Influence on instrumented code performance ##
 
-2Ghz Intel Centrino, 1GB RAM:
+Measures made on a 2GHz Intel Centrino, 1GB RAM.
+
+-- CPU Time measures (BENCH_TASK) --
+
+For 1 million measures:
 
  2.5s for 1 task (1.25s in parent, 1.25s in child).
 +3.5s for 100k different tasks.
@@ -12,9 +16,23 @@ i.e. using CBS to measure 100 different tasks for 100000 measures takes:
 + 100 000 * 3.5s / 1 000 000 * 100 / 100 000 (= 0.350ms)
 total: ~250ms.
 
-Use the timer-bench demo to compute those figures on another system.
+-- Memory measures (BENCH_MEMPLOT) --
+
+Time:
+ 2s per 100 000 measures.
+
+Memory:
+ 10MB per 100 000 measures.
+
 
-Typical usage:
+-- Benchmarking on another machine --
+
+Use the timer-bench and memplot-bench demos to compute those figures
+on your system.
+
+
+
+## Typical usage ##
 
 (see demo/bench/main.cc)
 
@@ -63,13 +81,13 @@ void do_some_tests()
 ________________________________________________________
 
 Note:
-Parameters and results are std::string.
-Use std::stringstream to convert other values:
+Parameters and results are std::string, long or double.
+Use explicit cast or std::stringstream to convert other values:
 
   #include <stringstream>
 
   int n = 42;
   std::stringstream n_str;
   n_str << n;
-  BENCH_PARAMETER("n", n_str.str());
-
+  BENCH_PARAMETER("n (string)", n_str.str());
+  BENCH_PARAMETER("n (long)", (long) n));

cbs/TODO

 To do:
-  - Implement memplot
+
   - Add more system info (e.g. hostname)
-  - Allow bench loops
+
+  ~ Allow bench loops:
+      CBS works with vcsn's global bencher.
+
+  ~ Support Windows & Mac:
+      Write windows & mac versions of the timing primitives.
 
 vcsn:
-  - Equip bench
-  - Gather xml files
-  - Generate html
+  - Equip bench:
+      Done, run make bench on a powerful machine
+
+  - Gather xml files:
+      Keep the directories as they are in make bench?
+
+  - Generate html:
+      Probably not necessary.
+      Extraction tool in bin/plot.pl

cbs/configure.ac

@@ -44,8 +44,11 @@ AC_CONFIG_FILES([
 	lib/Makefile
 	include/Makefile
 	demo/Makefile
+	  demo/comparison/Makefile
 	  demo/timer/Makefile
 	  demo/timer-bench/Makefile
+	  demo/memplot-bench/Makefile
+	  demo/memplot/Makefile
 	  demo/bench/Makefile])
 
 AC_OUTPUT

cbs/demo/Makefile.am

@@ -14,4 +14,4 @@
 
 include $(top_srcdir)/cbs.mk
 
-SUBDIRS = timer bench timer-bench
+SUBDIRS = timer bench timer-bench memplot comparison memplot-bench

cbs/demo/bench/demo.sh

-#!/bin/sh
-##
-
-make && ./timer-demo && ./display.sh&

cbs/demo/bench/display.sh

-#!/bin/sh
-##
-
-dot -Tpng benchmark_1.dot -o benchmark_1.png
-dot -Tpng benchmark_10.dot -o benchmark_10.png
-dot -Tpng benchmark_100.dot -o benchmark_100.png
-dot -Tpng benchmark_1000.dot -o benchmark_1000.png
-dot -Tpng benchmark_10000.dot -o benchmark_10000.png
-display benchmark_1.png &
-display benchmark_10.png &
-display benchmark_100.png &
-display benchmark_1000.png &
-display benchmark_10000.png &
-

cbs/demo/bench/main.cc

@@ -34,6 +34,7 @@
       {						\
         list.push_back(k);			\
       }						\
+    BENCH_MEMPLOT("Memory consumption");	\
     list.clear();				\
   }
 
@@ -57,36 +58,28 @@ int main()
     for (int i = 0; i < n; ++i)
       e();
 
-    CONSUME(130 + iter * 20)
+    CONSUME(10 + iter * 10)
 
     BENCH_STOP();
 
-    {
-      std::stringstream parameter;
-      parameter << iter;
+    BENCH_PARAMETER("iteration", (long) iter);
 
-      BENCH_PARAMETER("iteration", parameter.str());
-    }
-
-    {
-      std::stringstream parameter;
-      parameter << n;
+    BENCH_PARAMETER("n", (long) n);
 
-      BENCH_PARAMETER("n", parameter.str());
+    std::stringstream name_str;
+    name_str << "benchmark_" << n;
+    std::string name = name_str.str();
 
-      std::string name = "benchmark_" + parameter.str();
+    BENCH_PRINT(bench::Options(bench::Options::VE_NORMAL,
+			       bench::Options::FO_TEXT,
+			       0));
 
-      BENCH_PRINT(bench::Options(bench::Options::VE_NORMAL,
-				 bench::Options::FO_TEXT,
-				 0));
-
-      BENCH_SAVE(name + ".out", bench::Options(bench::Options::VE_NORMAL,
-					       bench::Options::FO_TEXT,
-					       0));
-      BENCH_SAVE(name + ".xml", bench::Options());
-      BENCH_SAVE(name + ".dot", bench::Options(bench::Options::VE_MINIMAL,
-					       bench::Options::FO_DOT));
-    }
+    BENCH_SAVE(name + ".out", bench::Options(bench::Options::VE_NORMAL,
+					     bench::Options::FO_TEXT,
+					     0));
+    BENCH_SAVE(name + ".xml", bench::Options());
+    BENCH_SAVE(name + ".dot", bench::Options(bench::Options::VE_MINIMAL,
+					     bench::Options::FO_DOT));
   }
 }
 
@@ -132,6 +125,7 @@ void e()
 {
   BENCH_TASK_SCOPED("Task E");
 
+  CONSUME(50)
   WAIT(1)
 
   d(0);

cbs/demo/bench/plot.sh

-#!/bin/sh
-cat *.xml |\
-sed -n '
-s/    <parameter name="[^"]*" value="\([0-9]*\)" \/>/\1/p
-s/    <result name="[^"]*" value="\([0-9]*\)" \/>/\1/p
-s/<\/bench>/NL/p
-' |\
-tr '\012' ' ' |\
-sed '
-{
-s/ NL /\
-/g
-}' |\
-sort -g > plot.plot

cbs/demo/comparison/Makefile.am

+## CBS, a C++ benchmarking suite.
+##
+## Copyright (C) 2007, 2009 The CBS Authors.
+##
+## This program is free software; you can redistribute it and/or
+## modify it under the terms of the GNU General Public License
+## as published by the Free Software Foundation; either version 2
+## of the License, or (at your option) any later version.
+##
+## The complete GNU General Public Licence Notice can be found as the
+## `COPYING' file in the root directory.
+##
+## The CBS Authors consists of people listed in the `AUTHORS' file.
+
+include $(top_srcdir)/cbs.mk
+
+noinst_PROGRAMS = demo-cbs demo-gprof demo-generic
+
+demo_cbs_SOURCES = main.cc
+demo_cbs_LDADD = $(top_builddir)/lib/libbench.la
+demo_cbs_CPPFLAGS = $(AM_CPPFLAGS)
+
+demo_gprof_SOURCES = main.cc
+demo_gprof_LDADD =
+demo_gprof_CPPFLAGS = $(AM_CPPFLAGS) -DBENCH_DISABLED -pg
+demo_gprof_LDFLAGS = -pg
+
+demo_generic_SOURCES = main.cc
+demo_generic_LDADD =
+demo_generic_CPPFLAGS = $(AM_CPPFLAGS) -DBENCH_DISABLED

cbs/demo/comparison/README

+-- Using CBS --
+
+In build dir:
+
+   time ./demo-cbs <n>
+
+
+-- Using gprof --
+
+In build dir:
+
+   time ./demo-gprof <n>
+   gprof ./demo-gprof
+
+Note: results seem wrong.
+
+
+-- Using callgrind --
+
+In build dir:
+
+   time valgrind --tool=callgrind --dump-every-bb=10000000 ./demo-generic <n>
+   kcachegrind
+   rm callgrind.*
+
+Note: callgrind files would stack between multiple executions.

cbs/demo/comparison/main.cc

+// main.cc: this file is part of the CBS project.
+//
+// CBS, a C++ benchmarking suite.
+//
+// Copyright (C) 2007, 2009 Florent D'Halluin.
+//
+// This program is free software; you can redistribute it and/or
+// modify it under the terms of the GNU General Public License
+// as published by the Free Software Foundation; either version 2
+// of the License, or (at your option) any later version.
+//
+// The complete GNU General Public Licence Notice can be found as the
+// `COPYING' file in the root directory.
+//
+
+#include <iostream>
+#include <sstream>
+#include <fstream>
+#include <list>
+#include <cbs/bench/bench_macros.hh>
+
+void parent(int);
+void new_integer(int);
+void list_push_back(int, std::list<int>& list);
+void file_io(int);
+
+int main(int argc, char** argv)
+{
+  BENCH_START("Profiler comparison",
+	      "A simple program that consumes time and memory\n"
+	      "The parameter n defines the program complexity,\n"
+	      "i.e. the time and memory taken.");
+
+  int n;
+
+  if (argc < 2)
+  {
+    std::cerr << "Usage: " << argv[0] << " <int benchmark_depth>" << std::endl;
+    return -1;
+  }
+
+  std::stringstream n_str(argv[1]);
+  n_str >> n;
+
+  if (n < 0 || n > 100)
+  {
+    std::cerr << "<benchmark_depth> must be between 0 and 100." << std::endl;
+    return -1;
+  }
+
+  std::cout << "Bench for n=" << n << "..." << std::endl;
+
+  parent(n);
+
+  std::cout << "Completed." << std::endl;
+
+  std::string name = "bench_" + std::string(argv[1]);
+
+  BENCH_PARAMETER("n", (long) n);
+
+  BENCH_STOP();
+
+  BENCH_PRINT(bench::Options(bench::Options::VE_NORMAL,
+			     bench::Options::FO_TEXT,
+			     0));
+
+  BENCH_SAVE(name + ".out", bench::Options(bench::Options::VE_FULL,
+					   bench::Options::FO_TEXT));
+  BENCH_SAVE(name + ".xml", bench::Options());
+  BENCH_SAVE(name + ".dot", bench::Options(bench::Options::VE_FULL,
+					   bench::Options::FO_DOT));
+}
+
+void parent(int n)
+{
+  BENCH_TASK_SCOPED("parent()");
+
+  std::list<int> list;
+  for (int i = 0; i < n; ++i)
+  {
+    new_integer(n);
+    list_push_back(n, list);
+    list.clear();
+
+    std::stringstream i_str;
+    i_str << i;
+
+    BENCH_MEMPLOT("parent(" + i_str.str() + ")");
+  }
+}
+
+void new_integer(int n)
+{
+  BENCH_TASK_SCOPED("new_integer()");
+
+  for (int i = 0; i < n * 100000; ++i)
+  {
+    int* a = new int(4);
+    ++(*a);
+    delete a;
+  }
+
+  file_io(n);
+}
+
+void list_push_back(int n, std::list<int>& list)
+{
+  BENCH_TASK_SCOPED("list_push_back()");
+
+  for (int i = 0; i < n * 100000; ++i)
+  {
+    list.push_back(i);
+  }
+
+  if (n > 0)
+    list_push_back(n - 1, list);
+}
+
+void file_io(int n)
+{
+  BENCH_TASK_SCOPED("file_io()");
+
+  std::ofstream file;
+
+  for (int i = 0; i < n * 1000; ++i)
+  {
+    file.open("_dummy");
+    file.close();
+  }
+
+  if (n > 0)
+    new_integer(n - 1);
+}

cbs/demo/memplot-bench/Makefile.am

+## CBS, a C++ benchmarking suite.
+##
+## Copyright (C) 2007, 2009 The CBS Authors.
+##
+## This program is free software; you can redistribute it and/or
+## modify it under the terms of the GNU General Public License
+## as published by the Free Software Foundation; either version 2
+## of the License, or (at your option) any later version.
+##
+## The complete GNU General Public Licence Notice can be found as the
+## `COPYING' file in the root directory.
+##
+## The CBS Authors consists of people listed in the `AUTHORS' file.
+
+include $(top_srcdir)/cbs.mk
+
+noinst_PROGRAMS = demo-memplot-bench
+demo_memplot_bench_SOURCES = main.cc
+demo_memplot_bench_LDADD = $(top_builddir)/lib/libbench.la
+demo_memplot_bench_CPPFLAGS = $(AM_CPPFLAGS)

cbs/demo/memplot-bench/get-results.sh

+#!/bin/sh
+../../../bin/plot.pl -a memplot_*.xml  > memplot-bench.data

cbs/demo/memplot-bench/main.cc

+// main.cc: this file is part of the CBS project.
+//
+// CBS, a C++ benchmarking suite.
+//
+// Copyright (C) 2007, 2009 Florent D'Halluin.
+//
+// This program is free software; you can redistribute it and/or
+// modify it under the terms of the GNU General Public License
+// as published by the Free Software Foundation; either version 2
+// of the License, or (at your option) any later version.
+//
+// The complete GNU General Public Licence Notice can be found as the
+// `COPYING' file in the root directory.
+//
+
+#include <iostream>
+#include <cbs/bench/timer.hh>
+#include <cbs/bench/bench_macros.hh>
+
+using namespace timer;
+
+int main()
+{
+  int n;
+
+  for (n = 10000; n < 500000; n *= 2)
+  {
+    BENCH_START("memplot", "Empty tasks.");
+
+    for (int i = 0; i < n; ++i)
+    {
+      BENCH_MEMPLOT("Empty task");
+    }
+
+    {
+      BENCH_STOP();
+
+      std::stringstream parameter;
+      parameter << n;
+
+      BENCH_PARAMETER("empty tasks", parameter.str());
+
+      std::string name = "memplot_" + parameter.str();
+
+
+      BENCH_PRINT(bench::Options(bench::Options::VE_NORMAL,
+				 bench::Options::FO_TEXT,
+				 0));
+
+      BENCH_SAVE(name + ".out", bench::Options(bench::Options::VE_NORMAL,
+					       bench::Options::FO_TEXT,
+					       0));
+      BENCH_SAVE(name + ".xml", bench::Options());
+      BENCH_SAVE(name + ".dot", bench::Options(bench::Options::VE_MINIMAL,
+					       bench::Options::FO_DOT));
+    }
+  }
+}
+

cbs/demo/memplot-bench/results-memory.plot

+set title "Memory consumption of CBS Memplot (2GHz Intel Centrino, 1GB RAM)"
+set xlabel "Number of measures"
+set ylabel "Memory usage (MB)"
+set key top left
+plot 'memplot-bench.data' using ($1):($2/1024/1024) t "Memory consumption" smooth unique w linespoints pt 5 lc 1 lw 2

cbs/demo/memplot-bench/results-time.plot

+set title "Time consumption of CBS Memplot (2GHz Intel Centrino, 1GB RAM)"
+set xlabel "Number of measures"
+set ylabel "Time taken (ms)"
+set key top left
+plot 'memplot-bench.data' using ($1):($3) t "CPU time" smooth unique w linespoints pt 5 lc 1 lw 2, \
+     'memplot-bench.data' using ($1):($4) t "user" smooth unique w linespoints pt 6 lc 1, \
+     'memplot-bench.data' using ($1):($5) t "system" smooth unique w linespoints pt 3 lc 1

cbs/demo/memplot/Makefile.am

+## CBS, a C++ benchmarking suite.
+##
+## Copyright (C) 2007, 2009 The CBS Authors.
+##
+## This program is free software; you can redistribute it and/or
+## modify it under the terms of the GNU General Public License