Commit 16a8c031 authored by Alexandre Duret-Lutz's avatar Alexandre Duret-Lutz

ltldo: new binary

* src/bin/common_trans.cc, src/bin/common_trans.hh: New files,
extracted from...
* src/bin/ltlcross.cc: ... here, so that ltldo can use them.
* src/bin/ltldo.cc: New file.
* src/bin/Makefile.am: Adjust.
* src/bin/common_aoutput.cc, src/bin/common_aoutput.hh: Make
it possible to add new statistics.
* doc/org/ltldo.org: New file.
* doc/Makefile.am, doc/org/tools.org: Adjust.
* src/bin/man/ltldo.x: New file.
* src/bin/man/Makefile.am: Adjust.
* src/bin/man/ltlcross.x, src/bin/man/ltlfilt.x: Mention ltldo(1).
* src/tgbatest/ltldo.test, src/tgbatest/ltldo2.test: New files.
* src/tgbatest/Makefile.am: Add them.
* NEWS: Mention ltldo.
parent e5294aac
......@@ -20,6 +20,10 @@ New in spot 1.99a (not yet released)
- autfilt is a new tool that converts/filters/transforms a
stream of automata.
- ltldo is a new tool that run LTL/PSL formulas through other
tools, but uses Spot's command-line interfaces for specifying
input and output.
- "ltlfilt --stutter-invariant" will now work with PSL formula.
The implementation is actually much more efficient
than the previous implementation that was only for LTL.
......
......@@ -69,6 +69,7 @@ ORG_FILES = \
org/ltl2tgba.org \
org/ltl2tgta.org \
org/ltlcross.org \
org/ltldo.org \
org/ltlfilt.org \
org/ltlgrind.org \
org/oaut.org \
......
......@@ -11,3 +11,4 @@ gfagfb
bogus
bogus-grind
example.hoa
sample.ltl
# -*- coding: utf-8 -*-
#+TITLE: =ltldo=
#+SETUPFILE: setup.org
#+HTML_LINK_UP: tools.html
This tool is a wrapper for tools that read LTL/PSL formulas and
(optionally) output automata.
It reads formulas specified using the [[file:ioltl.org][common options for specifying
input]] and passes each formula to a tool (or a list of tools) specified
using options similar to those of [[file:ltlcross.org][=ltlcross=]]. In case that tool
returns an automaton, the resulting automaton is read back by =ltldo=
and is finally output as specified using the [[file:oaut.org][common options for
outputing automata]].
In effect, =ltldo= wraps the I/O interface of the Spot tools on top of
any other tool.
* Example: computing statistics for =ltl3ba=
As a motivating example, consider a scenario where we want to run
[[http://sourceforge.net/projects/ltl3ba/][=ltl3ba=]] on a set of 10 formulas stored in a file. For each formula
we would like to compute compute the number of states and edges in the
Büchi automaton produced by =ltl3ba=.
Here is the input file:
#+BEGIN_SRC sh :results verbatim :exports code
cat >sample.ltl <<EOF
1
1 U a
!(!((a U Gb) U b) U GFa)
(b <-> Xc) xor Fb
FXb R (a R (1 U b))
Ga
G(!(c | (a & (a W Gb))) M Xa)
GF((b R !a) U (Xc M 1))
G(Xb | Gc)
XG!F(a xor Gb)
EOF
#+END_SRC
#+RESULTS:
We will first implement this scenario without =ltldo=.
A first problem that the input is not in the correct syntax: although
=ltl3ba= understands =G= and =F=, it does not support =xor= or =M=,
and requires the Boolean operators =||= and =&&=. This syntax
issue can be fixed by processing the input with [[file:ltlfilt.org][=ltlfilt -s=]].
A second problem is that =ltl3ba= (at least version 1.1.1) can only
process one formula at a time. So we'll need to call =ltl3ba= in a
loop.
Finally, one way to compute the size of the resulting Büchi automaton
is to pipe the output of =ltl3ba= through [[file:autfilt.org][=autfilt=]].
Here is how the shell command could look like:
#+BEGIN_SRC sh :results verbatim :exports both
ltlfilt -F sample.ltl -s |
while read f; do
ltl3ba -f "$f" | autfilt --stats="$f,%s,%t"
done
#+END_SRC
#+RESULTS:
#+begin_example
true,1,1
true U a,2,4
!(!((a U []b) U b) U []<>a),2,4
(((!b && !Xc) || (b && Xc)) && !<>b) || (<>b && !((!b && !Xc) || (b && Xc))),7,21
<>Xb V (a V (true U b)),6,28
[]a,1,1
[](Xa U (Xa && !(c || (a && ([]b V (a || []b)))))),1,0
[]<>((b V !a) U (true U Xc)),2,4
[](Xb || []c),3,11
X[]!<>((a && ![]b) || (!a && []b)),4,10
#+end_example
Using =ltldo= the above command can be reduced to this:
#+BEGIN_SRC sh :results verbatim :exports both
ltldo -F sample.ltl 'ltl3ba -f %s>%N' --stats='%f,%s,%t'
#+END_SRC
#+RESULTS:
#+begin_example
1,1,1
1 U a,2,4
!(!((a U Gb) U b) U GFa),2,4
(b <-> Xc) xor Fb,7,21
FXb R (a R (1 U b)),6,28
Ga,1,1
G(!(c | (a & (a W Gb))) M Xa),1,0
GF((b R !a) U (Xc M 1)),2,4
G(Xb | Gc),3,11
XG!F(a xor Gb),4,10
#+end_example
Note that the formulas look different in both cases, because in the
=while= loop the formula printed has already been processed with
=ltlfilt=, while =ltldo= emits the input string untouched.
* Example: running =spin= and producing HOA
Here is another example, where we use Spin to produce two automata in
the [[http://adl.github.io/hoaf/][HOA format]]. Spin has no support for HOA, but =ltldo= simply
converts the never claim produced by =spin= into this format.
#+BEGIN_SRC sh :results verbatim :exports both
ltldo -f a -f GFa 'spin -f %s>%N' -H
#+END_SRC
#+RESULTS:
#+begin_example
HOA: v1
States: 2
Start: 0
AP: 1 "a"
acc-name: Buchi
Acceptance: 1 Inf(0)
properties: trans-labels explicit-labels state-acc deterministic
--BODY--
State: 0 {0}
[0] 1
State: 1 {0}
[t] 1
--END--
HOA: v1
States: 2
Start: 0
AP: 1 "a"
acc-name: Buchi
Acceptance: 1 Inf(0)
properties: trans-labels explicit-labels state-acc complete
--BODY--
State: 0
[0] 1
[t] 0
State: 1 {0}
[t] 0
--END--
#+end_example
* Syntax for specifying tools to call
The syntax for specifying how a tool should be called is the same as
in [[file:ltlcross.org][=ltlcross=]]. Namely, the following sequences are available.
#+BEGIN_SRC sh :results verbatim :exports results
ltldo --help | sed -n '/character sequences:/,/^$/p' | sed '1d;$d'
#+END_SRC
#+RESULTS:
: %f,%s,%l,%w the formula as a (quoted) string in Spot, Spin,
: LBT, or Wring's syntax
: %F,%S,%L,%W the formula as a file in Spot, Spin, LBT, or
: Wring's syntax
: %N,%T,%D,%H the automaton is output as a Never claim, or in
: LBTT's, in LTL2DSTAR's, or in the HOA format
Contrarily to =ltlcross=, it this not mandatory to specify an output filename
using one of the sequence for that later lines. For instance we could
simply run a formula though =echo= to compare different output syntaxes:
#+BEGIN_SRC sh :results verbatim :exports both
ltldo -f 'p0 U p1' -f 'GFp0' 'echo %f, %s, %l, %w'
#+END_SRC
#+RESULTS:
: (p0) U (p1), (p0) U (p1), U p0 p1, (p0=1) U (p1=1)
: (G(F(p0))), ([](<>(p0))), G F p0, (G(F(p0=1)))
In this case (i.e., when the command does not specify any output
filename), =ltldo= will not output anything.
As will =ltlcross=, multiple commands can be given, and they will be
executed on each formula in the same order.
A typical use-case is to compare statistics of different tools:
#+BEGIN_SRC sh :results verbatim :exports both
ltldo -F sample.ltl 'spin -f %s>%N' 'ltl3ba -f %s>%N' --stats=%T,%f,%s,%e
#+END_SRC
#+RESULTS:
#+begin_example
spin -f %s>%N,1,2,2
ltl3ba -f %s>%N,1,1,1
spin -f %s>%N,1 U a,2,3
ltl3ba -f %s>%N,1 U a,2,3
spin -f %s>%N,!(!((a U Gb) U b) U GFa),23,86
ltl3ba -f %s>%N,!(!((a U Gb) U b) U GFa),2,3
spin -f %s>%N,(b <-> Xc) xor Fb,12,23
ltl3ba -f %s>%N,(b <-> Xc) xor Fb,7,11
spin -f %s>%N,FXb R (a R (1 U b)),28,176
ltl3ba -f %s>%N,FXb R (a R (1 U b)),6,20
spin -f %s>%N,Ga,1,1
ltl3ba -f %s>%N,Ga,1,1
spin -f %s>%N,G(!(c | (a & (a W Gb))) M Xa),15,51
ltl3ba -f %s>%N,G(!(c | (a & (a W Gb))) M Xa),1,0
spin -f %s>%N,GF((b R !a) U (Xc M 1)),12,60
ltl3ba -f %s>%N,GF((b R !a) U (Xc M 1)),2,4
spin -f %s>%N,G(Xb | Gc),4,8
ltl3ba -f %s>%N,G(Xb | Gc),3,5
spin -f %s>%N,XG!F(a xor Gb),8,21
ltl3ba -f %s>%N,XG!F(a xor Gb),4,7
#+end_example
Here we used =%T= to output the name of the tool used to translate the
formula =%f= as an automaton with =%s= states and =%e= edges.
If you feel that =%T= has too much clutter, you can give each tool
a shorter name by prefixing its command with ={name}=.
In the following example, we moved the formula used on its own line
using the trick that the command =echo %f= will not be subject to
=--stats= (since it does not declare any output automaton).
#+BEGIN_SRC sh :results verbatim :exports both
ltldo -F sample.ltl --stats=%T,%s,%e \
'echo "#" %f' '{spin}spin -f %s>%N' '{ltl3ba}ltl3ba -f %s>%N'
#+END_SRC
#+RESULTS:
#+begin_example
# (1)
spin,2,2
ltl3ba,1,1
# (1) U (a)
spin,2,3
ltl3ba,2,3
# (!((!(((a) U (G(b))) U (b))) U (G(F(a)))))
spin,23,86
ltl3ba,2,3
# ((b) <-> (X(c))) xor (F(b))
spin,12,23
ltl3ba,7,11
# (F(X(b))) R ((a) R ((1) U (b)))
spin,28,176
ltl3ba,6,20
# (G(a))
spin,1,1
ltl3ba,1,1
# (G((!((c) | ((a) & ((a) W (G(b)))))) M (X(a))))
spin,15,51
ltl3ba,1,0
# (G(F(((b) R (!(a))) U ((X(c)) M (1)))))
spin,12,60
ltl3ba,2,4
# (G((X(b)) | (G(c))))
spin,4,8
ltl3ba,3,5
# (X(G(!(F((a) xor (G(b)))))))
spin,8,21
ltl3ba,4,7
#+end_example
Much more readable!
* Controlling and measuring time
The run time of each command can be restricted with the =-T NUM=
option. The argument is the maximum number of seconds that each
command is allowed to run.
When a timeout occurs a warning is printed on stderr, and no automaton
(or statistic) is output by =ltdo= for this specific pair of
command/formula. The processing then continue with other formulas and
tools. Timeouts are not considered as errors, so they have no effect
on the exit status of =ltldo=.
For each command (that does not terminate with a timeout) the runtime
can be printed using the =%r= escape sequence. This makes =ltldo= an
alternative to [[file:ltlcross.org][=ltlcross=]] for running benchmarks without any
verification.
......@@ -48,6 +48,8 @@ corresponding commands are hidden.
Büchi automata.
- [[file:randaut.org][=randaut=]] Generate random automata.
- [[file:autfilt.org][=autfilt=]] Filter and convert automata.
- [[file:ltldo.org][=ltldo=]] Run LTL/PSL formulas through other tools using common [[file:ioltl.org][input]]
and [[file:oaut.org][output]] interfaces.
* Advanced use-cases
......
......@@ -4,6 +4,7 @@ genltl
ltl2tgba
ltl2tgta
ltlcross
ltldo
ltlfilt
ltlgrind
randaut
......
......@@ -44,10 +44,22 @@ libcommon_a_SOURCES = \
common_r.hh \
common_setup.cc \
common_setup.hh \
common_sys.hh
common_sys.hh \
common_trans.cc \
common_trans.hh
bin_PROGRAMS = autfilt ltlfilt genltl randaut randltl ltl2tgba \
ltl2tgta ltlcross dstar2tgba ltlgrind
bin_PROGRAMS = \
autfilt \
dstar2tgba \
genltl \
ltl2tgba \
ltl2tgta \
ltlcross \
ltldo \
ltlfilt \
ltlgrind \
randaut \
randltl
# Dummy program used just to generate man/spot-x.7 in a way that is
# consistent with the other man pages (e.g., with a version number that
......@@ -63,6 +75,7 @@ ltl2tgba_SOURCES = ltl2tgba.cc
ltl2tgta_SOURCES = ltl2tgta.cc
ltlcross_SOURCES = ltlcross.cc
ltlgrind_SOURCES = ltlgrind.cc
ltldo_SOURCES = ltldo.cc
dstar2tgba_SOURCES = dstar2tgba.cc
spot_x_SOURCES = spot-x.cc
ltlcross_LDADD = $(LDADD) $(LIB_GETHRXTIME)
......
......@@ -23,6 +23,7 @@
#include "common_aoutput.hh"
#include "common_post.hh"
#include "common_cout.hh"
#include "common_post.hh"
#include "tgba/bddprint.hh"
......@@ -259,3 +260,9 @@ automaton_printer::print(const spot::tgba_digraph_ptr& aut,
}
flush_cout();
}
void automaton_printer::add_stat(char c, const spot::printable* p)
{
namer.declare(c, p);
statistics.declare(c, p);
}
......@@ -97,6 +97,8 @@ public:
declare('w', &aut_word_);
}
using spot::formater::declare;
/// \brief print the configured statistics.
///
/// The \a f argument is not needed if the Formula does not need
......@@ -218,6 +220,8 @@ public:
// Time and input automaton for statistics
double time = 0.0,
const spot::const_hoa_aut_ptr& haut = nullptr);
void add_stat(char c, const spot::printable* p);
};
......
This diff is collapsed.
// -*- coding: utf-8 -*-
// Copyright (C) 2015 Laboratoire de Recherche et Développement de
// l'Epita (LRDE).
//
// This file is part of Spot, a model checking library.
//
// Spot is free software; you can redistribute it and/or modify it
// under the terms of the GNU General Public License as published by
// the Free Software Foundation; either version 3 of the License, or
// (at your option) any later version.
//
// Spot is distributed in the hope that it will be useful, but WITHOUT
// ANY WARRANTY; without even the implied warranty of MERCHANTABILITY
// or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public
// License for more details.
//
// You should have received a copy of the GNU General Public License
// along with this program. If not, see <http://www.gnu.org/licenses/>.
#ifndef SPOT_BIN_COMMON_TRANS_HH
#define SPOT_BIN_COMMON_TRANS_HH
#include "common_sys.hh"
#include <vector>
#include <argp.h>
#include "misc/formater.hh"
#include "misc/tmpfile.hh"
#include "tgba/tgbagraph.hh"
extern const struct argp trans_argp;
struct translator_spec
{
// The translator command, as specified on the command-line.
// If this has the form of
// {name}cmd
// then it is split in two components.
// Otherwise, spec=cmd=name.
const char* spec;
// actual shell command (or spec)
const char* cmd;
// name of the translator (or spec)
const char* name;
translator_spec(const char* spec);
translator_spec(const translator_spec& other);
~translator_spec();
};
extern std::vector<translator_spec> translators;
struct quoted_string final: public spot::printable_value<std::string>
{
using spot::printable_value<std::string>::operator=;
void print(std::ostream& os, const char* pos) const override;
};
struct printable_result_filename final:
public spot::printable_value<spot::temporary_file*>
{
unsigned translator_num;
enum output_format { None, Lbtt, Dstar, Hoa };
mutable output_format format;
printable_result_filename();
~printable_result_filename();
void reset(unsigned n);
void cleanup();
void print(std::ostream& os, const char* pos) const override;
};
class translator_runner: protected spot::formater
{
protected:
spot::bdd_dict_ptr dict;
// Round-specific variables
quoted_string string_ltl_spot;
quoted_string string_ltl_spin;
quoted_string string_ltl_lbt;
quoted_string string_ltl_wring;
quoted_string filename_ltl_spot;
quoted_string filename_ltl_spin;
quoted_string filename_ltl_lbt;
quoted_string filename_ltl_wring;
// Run-specific variables
printable_result_filename output;
public:
using spot::formater::has;
translator_runner(spot::bdd_dict_ptr dict,
// whether we accept the absence of output
// specifier
bool no_output_allowed = false);
void string_to_tmp(std::string& str, unsigned n, std::string& tmpname);
const std::string& formula() const;
void round_formula(const spot::ltl::formula* f, unsigned serial);
};
// Disable handling of timeout on systems that miss kill() or alarm().
// For instance MinGW.
#if HAVE_KILL && HAVE_ALARM
# define ENABLE_TIMEOUT 1
#else
# define ENABLE_TIMEOUT 0
#endif
extern volatile bool timed_out;
extern unsigned timeout_count;
#if ENABLE_TIMEOUT
void setup_sig_handler();
int exec_with_timeout(const char* cmd);
#else // !ENABLE_TIMEOUT
#define exec_with_timeout(cmd) system(cmd)
#define setup_sig_handler() while (0);
#endif // !ENABLE_TIMEOUT
#endif // SPOT_BIN_COMMON_TRANS_HH
This diff is collapsed.
// -*- coding: utf-8 -*-
// Copyright (C) 2015 Laboratoire de Recherche et Développement de
// l'Epita (LRDE).
//
// This file is part of Spot, a model checking library.
//
// Spot is free software; you can redistribute it and/or modify it
// under the terms of the GNU General Public License as published by
// the Free Software Foundation; either version 3 of the License, or
// (at your option) any later version.
//
// Spot is distributed in the hope that it will be useful, but WITHOUT
// ANY WARRANTY; without even the implied warranty of MERCHANTABILITY
// or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public
// License for more details.
//
// You should have received a copy of the GNU General Public License
// along with this program. If not, see <http://www.gnu.org/licenses/>.
#include "common_sys.hh"
#include <string>
#include <iostream>
#include <sstream>
#include <fstream>
#include "error.h"
#include "common_setup.hh"
#include "common_cout.hh"
#include "common_conv.hh"
#include "common_finput.hh"
#include "common_aoutput.hh"
#include "common_post.hh"
#include "common_trans.hh"
#include "ltlvisit/tostring.hh"
#include "misc/timer.hh"
#include "tgbaalgos/lbtt.hh"
#include "hoaparse/public.hh"
#include "dstarparse/public.hh"
const char argp_program_doc[] ="\
Run LTL/PSL formulas through another program, performing conversion\n\
of input and output as required.";
static const argp_option options[] =
{
{ 0, 0, 0, 0, "Miscellaneous options:", -1 },
{ 0, 0, 0, 0, 0, 0 }
};
static const argp_option more_o_format[] =
{
{ "%R", 0, 0, OPTION_DOC | OPTION_NO_USAGE,
"serial number of the formula translated", 0 },
{ "%T", 0, 0, OPTION_DOC | OPTION_NO_USAGE,
"tool used for translation", 0 },
{ "%f", 0, 0, OPTION_DOC | OPTION_NO_USAGE,
"formula translated", 0 },
{ 0, 0, 0, 0, 0, 0 }
};
// This is not very elegant, but we need to add the above %-escape
// sequences to those of aoutput_o_fromat_argp for the --help output.
// So far I've failed to instruct argp to merge those two lists into a
// single block.
static const struct argp*
build_percent_list()
{
const argp_option* iter = aoutput_o_format_argp.options;
unsigned count = 0;
while (iter->name || iter->doc)
{
++count;
++iter;
}
unsigned s = count * sizeof(argp_option);
argp_option* d =
static_cast<argp_option*>(malloc(sizeof(more_o_format) + s));
memcpy(d, aoutput_o_format_argp.options, s);
memcpy(d + count, more_o_format, sizeof(more_o_format));
static const struct argp more_o_format_argp =
{ d, 0, 0, 0, 0, 0, 0 };
return &more_o_format_argp;
}
const struct argp_child children[] =
{
{ &finput_argp, 0, 0, 1 },
{ &trans_argp, 0, 0, 3 },
{ &aoutput_argp, 0, 0, 4 },
{ build_percent_list(), 0, 0, 5 },
{ &misc_argp, 0, 0, -1 },
{ 0, 0, 0, 0 }
};
static int
parse_opt(int key, char* arg, struct argp_state*)
{
switch (key)
{
case ARGP_KEY_ARG:
translators.push_back(arg);
break;
default:
return ARGP_ERR_UNKNOWN;
}
return 0;
}
namespace
{
class xtranslator_runner: public translator_runner
{
public:
xtranslator_runner(spot::bdd_dict_ptr dict)
: translator_runner(dict, true)
{
}
spot::tgba_digraph_ptr
translate(unsigned int translator_num, bool& problem, double& duration)
{
output.reset(translator_num);
std::ostringstream command;
format(command, translators[translator_num].cmd);
//assert(output.format != printable_result_filename::None);
std::string cmd = command.str();
//std::cerr << "Running [" << l << translator_num << "]: "
// << cmd << std::endl;
spot::stopwatch sw;
sw.start();
int es = exec_with_timeout(cmd.c_str());
duration = sw.stop();
spot::tgba_digraph_ptr res = nullptr;
if (timed_out)
{
problem = false; // A timeout is considered benign
std::cerr << "warning: timeout during execution of command \""
<< cmd << "\"\n";
++timeout_count;
}
else if (WIFSIGNALED(es))
{
problem = true;