Statestep

FAQ

[Serbo-Croatian translation]

What can I use Statestep for?

It is difficult to anticipate all the ways in which it could be useful. In the area of software engineering, it is designed to support a particular method of specification which has proven valuable in the context of embedded interactive systems (though it will not suit all such systems). It could also potentially be used, for example, to produce a test oracle to avoid having to validate test outputs manually.

More generally, it has potential application in any area where you might wish to consider many combinations of circumstances - especially where, because of the very large number of combinations, you might not previously have imagined such a thorough approach was even possible. Perhaps the best way to get an idea of whether and how it may be of use to you is to look through the first two or three pages of the tutorial.

What's the difference between Statestep and decision table tools?

You can use Statestep merely as a decision table tool. Used in this way, Statestep has a unique interface that allows you to easily create new rules for uncovered cases even if you're trying to manage millions or billions of combinations - whereas a regular decision table tool may start to become unwieldy with only hundreds of combinations. Also, Statestep allows for impossible cases to be dealt with much more efficiently than the dummy rule strategy of decision table tools. Further, you're not even limited to a tabular representation of rule conditions: you can use formulas wherever it would be inefficient or awkward to use only a table.

However, Statestep can also be used to create a finite state machine model as a way of describing system behaviour; indeed this is what it's designed for. And it's also possible to use the constraint table as a standalone element to explore logical relationships, find contradictions, etc.

Do we have time to use Statestep?

If it is suited to specifying your system then it should save you time. In fact, the approach Statestep supports originated in industry from a commercial need to develop software for an embedded system within a very tight schedule. Because specification errors cost far more time to fix when discovered later in a project, a little extra time spent creating a better specification means a big saving in overall development time.

Further, in contrast to techniques which require a big up-front investment in training, Statestep is relatively simple. In practice, completed models have been found to be understandable with little or no explanation. (Caveat: formulas, which are optional, can cause difficulties. If you do use formulas then it is a good idea to supplement them with textual descriptions.)

How does Statestep compare to formal methods?

Statestep shares the advantages common to formal methods in that it helps to create specifications that are precise, consistent and unambiguous. However, a recognized weakness of formal methods in general is that they are weak at exposing errors of omission. Statestep addresses this weakness by forcing you to think about how your system should respond in every case in an explicitly comprehensive way.

Statestep is also much easier to learn and use than most formal methods. On the negative side, it is typically less scalable, at least in its basic form, and also rather more specialized, which reduces the number of situations in which it is appropriate.

How does Statestep compare to other tabular methods?

There are several other tabular methods for specifying behaviour such as SCR and SpecTRM-RL. The principal difference is that each table in such methods describes the behaviour of a small part of the system; the response of the overall system is determined by the composition of the parts, taking into account any dependencies between them. For particular cases, this response can be worked out by the reader or be seen by testing or simulating the model. In contrast, each rule in Statestep shows the response of the whole system at once (or a comparatively large part of it) for some set of cases. This makes for an appreciably simpler notation. Moreover, this is what allows systematic consideration of how the overall system behaves in every possible case.

Thus, despite the superficial similarity, Statestep differs from other tabular approaches in much the same way that it differs from formal methods in general (see the previous question).

Is Statestep a model checker?

If you use Statestep to model a finite state machine then you will find that it automatically performs the most useful function of a model checker, that is, to check that no undesirable state can be reached. Unlike an ordinary model checker, Statestep can interactively check this reachability property (and other properties) even while the model is still being created, without reporting spurious errors. Another advantage is that it can explain its reasoning so you can see how the problem arises.

On the other hand, a model checker will allow you to describe (in temporal logic) more general properties to check, for example, is it possible to get from state X to state Y without going through state Z? Thus, while there is some overlap in functionality, Statestep is not a model checker as the term is ordinarily understood.

Should Statestep be used for safety-critical systems?

Statestep's unusually explicit and comprehensive approach can reveal problems that might not be detected during testing or even using more sophisticated techniques like model checking. (For an example of the subtle anomalies that can be found in even a small model, see the document on the digital watch model.) Thus, Statestep can provide added confidence in the quality of complex reactive systems intended for critical domains.

Naturally, there are the usual caveats. In particular, it cannot be guaranteed that Statestep itself or the Java platform on which it runs are bug-free: the licence agreements for both disclaim any warranty or liability for bugs. Therefore, while it may be used in addition to other procedures and tools to help ensure quality, Statestep does not claim to displace them.

What does Statestep check that other tools don't?

In terms of checking correctness, what uniquely distinguishes Statestep is that it gives you the power needed to validate a model's response in all circumstances - something that, fundamentally, cannot be checked automatically. For example, if you create a standard finite state machine model which includes components A, B, ..., then you can get a model checker to check an invariant such as A is never in state a1 at the same time as B is in state b1. However, suppose there is some unusual scenario in which the model enters the state [a1,b2,...]. Even if this state is allowed, it does not follow that for the system to enter it is the desired or correct response in this particular case. The problem may also be missed in testing which, for non-trivial systems, is necessarily sample-based.

With Statestep, even as you create the model you are already validating the overall system response in every case, including the most obscure scenarios.

What's the most combinations Statestep can handle?

Statestep continually checks the model you're working on and the unusual power of its interface depends on its performing a lot of calculations at interactive speed. You might expect that there would be some fixed limit to the size of the model that can be handled in this way, e.g., that the program would suddenly fall over when the number of possible combinations exceeded some number X. However, there is no fixed limit: Statestep operates symbolically, that is, it uses reasoning to avoid having to list out every possibility in the computer's memory.

What this means is that whether a model is too big for Statestep on a given machine is a complex function of the model's characteristics: how many constraints there are and how they relate to each other, how many rules for a given event, whether complicated formulas are used, etc. Experience suggests that the limiting factor in practice is more likely to be a desire to restrict the number of variables so that they fit within the width of the display. As an example, one real (i.e., taken from industry) model has 16 variables whose values can be combined in 1,428,840,000 ways; running on a 1.4GHz PC, Statestep provides instant feedback to any edit operation.

How many variables (values, rules...) does Statestep allow?

There is no fixed limit on the number of variables, values, constraints, events or rules. However, the maximum number of values per variable is 30: if more than this were allowed, the height of displayed table rows would become impractical anyway - and enforcing a fixed limit on variable size allows Statestep to maximize responsiveness and memory efficiency. (Note that a lower limit may potentially apply for code generation, depending on implementation. It's a good idea to check whatever code template you plan to use before creating your model.)

This is not much of a restriction since most finite state or other enumerated variables of the kind likely to be part of a Statestep model will not have this many values. In those rare cases where a larger variable might be useful it can always be replaced by two or more variables with fewer values.

What does Statestep cost?

Free licences are available for educational and non-commercial use. These are renewable annually, in return for which a brief description on how you have used Statestep and any other comments are appreciated (but not required).

For commercial use, licensing is flexible, for example, a license can be limited to named users or a named project, etc. For an evaluation licence and a quote, please send an email.

Aren't these tabular models very big and inefficient?

Big, yes – initially, a model can be intimidating (or impressive) and printing one out won't do the trees any favours. Inefficient, no! In fact, in every respect that matters, the specification should be efficient: after all, if you took a regular, text document, removed all the paragraph breaks and used the same small font for headings and text throughout, you might save some pages but nobody would call that "efficiency".

A printed Statestep model is optimized for reference, its most important use as a product. Even when read cover to cover for review purposes, a page of tables can be scanned much more quickly than a page of text. Also, the process needs to be considered as well as the product – and the Statestep method is designed to ensure you consider everything you need to consider exactly once. This makes specification as well as development more efficient: if the specification phase of a project takes longer, it is because you have done everything you needed to do in that phase; and you avoid surprises later on, when errors would take much more work to put right.

My customer is sceptical, what can I tell them?

A stakeholder is entitled to be wary of agreeing to a specification being prepared in an unfamiliar way. Unfortunately, fashion and exaggerated claims are not uncommon in the software industry and so scepticism is natural and sensible.

One thing to consider doing is to create a Statestep model separate from the specification you deliver to your customer (actually, a Statestep model will normally be only part of your specification deliverables anyway; see the User Guide). This might seem like more work than you want to do. However, you should find that using Statestep allows you to substantially improve the quality of the specification that you do deliver. Indeed, it may elicit so many questions that your customers start to wonder at your talent for thinking up scenarios that never occurred to them. They will also better appreciate the effort involved in that small extra feature they wanted. So, even if the model is never delivered, creating it can still be a significant benefit to you as well as your customer.

Shouldn't these two rules overlap (or conflict)? Is this a bug?

Rules for the same event overlap (or, where their outcomes are different, conflict) only if the cases covered by both rules are possible. If you look at the two rules again, you should find that, according to the constraints, the cases in which both rules apply can never occur.

To confirm this, first change one of the rules so that it covers only the cases covered by both rules. All the values should change to an orange colour. Next, right-click in any of the table cells and click 'Explain'. If you now delete one of the constraints listed in the explanation then the colours of the values will change and the overlap will be reported. (Note: if this doesn't happen immediately then there are additional reasons for the cases being impossible; just click 'Explain' again.)

Why do you provide links to competing tools?

In a sense, Statestep doesn't really have any competitors as it is unique in several respects. In particular, no other tool makes it easy to deal systematically with a potentially unlimited number of possibilities: if you use something like a decision table tool, the number of combinations that can practically be managed is probably in the hundreds; and whereas the rules in a rules engine may address enormous numbers of possibilities, they don't do this in the explicit, systematic way that Statestep allows.

On the other hand, rules engines (for instance) are designed to manipulate large, general databases of facts, something which Statestep was never meant to do. In short, the links may be useful to those who don't need (and so won't appreciate) the power provided by Statestep and to those who are looking for quite a different kind of tool or method.

Why is FreeMarker used for automatic code generation?

There are many potential ways to generate code from a Statestep model in XML. Some of the alternatives include XSLT, Perl and Apache Velocity. XSLT, however, is more suitable for generating XML or HTML output than text; as well-formed XML itself, XSLT is also verbose and some people find the language constructs counter-intuitive. Perl, while powerful and concise, is intimidating to the newcomer; it's a solution that's much bigger than the problem. Velocity is similar to FreeMarker but lacks a standard equivalent to FreeMarker's FMPP to load and process XML data; also, it is not powerful enough for general use.

While FMPP is normally used to generate web page content, it works perfectly well as a generic template engine. Used for code generation, it strikes the right balance: simple things are reasonably simple to do while more powerful features, such as recursive macros, are available when needed.

Questions, comments? info3@statestep.com