Module eqc_mocking

Description
Data Types
Function Index
Function Details

This module provides functionality for mocking Erlang modules.

Version: 1.42.1

Authors: Hans Svensson.

Description

This module provides functionality for mocking Erlang modules. It is designed to work together with eqc_component where callouts can be mocked conveniently using the functionality in this module. However, it is also possible to use this module for traditional mocking, but the documentation focus on the callout use-case.

The mocking technique we use in this module is based on research results from the Prowess project - An Expressive Semantics of Mocking. In short we use a CSP-like language to specify the behaviour of mocked modules. The behaviour can be described using the following macros:

?EVENT(Mod, Fun, Args, Result)
Defines an event. An event consists of an Action, which is matched, and if the action matches the defined result is returned. The action contains module, function and argument.
?SEQ(Ls)
Composes a list of languages sequentially.
?SEQ(L1, L2)
Composes two languages sequentially.
?PAR(Ls)
Composes a list of languages in parallel.
?PAR(L1, L2)
Composes two languages in parallel.
?XALT(L1, L2)
Introduces a language where either L1 or L2 is executed.
?REPLICATE(L)
Replication of a language, corresponds roughly to (*) in a regular expression.
?PERM(Ls)
Permutation, the language accepts any non-interleaving execution of the languages in Ls.
?SUCCESS
The empty (successful) language.

How to mock a module

To mock a module we need to specify its API in an API specification. Given the API we can also specify the particular behaviour of the defined functions, by defining a mocking langauage specification. As an example we give the mocking specification of a stack; its API consists of three functions new/0, push/2, and pop/1. Its API specification looks like:

  api_spec() -> #api_spec{
    language = erlang,
    modules  = [ #api_module{
      name = stack,
      functions = [ #api_fun{ name = new,  arity = 0 },
                    #api_fun{ name = push, arity = 2 },
                    #api_fun{ name = pop,  arity = 1 } ]
      }]}.

In one test case we expect the stack to be called four times; (1) creating a new stack, (2) pushing the integer 4 onto the stack, (3) pushing 6 onto the stack, and (4) popping one element from the stack expecting 6 to be returned. This could be expressed as:

  lang() -> ?SEQ([?EVENT(stack, new,  [],             stack_ref),
                  ?EVENT(stack, push, [stack_ref, 4], ok),
                  ?EVENT(stack, push, [stack_ref, 6], ok),
                  ?EVENT(stack, pop,  [stack_ref],    6)
                 ]).

Where we use an abstract stack reference stack_ref to represent the stack during the execution.

The stack can now be 'used' as follows (assuming that the functions defined above reside in the module stack_mock):

  2> eqc_mocking:start_mocking(stack_mock:api_spec()).
  {ok,<0.116.0>}
  3> eqc_mocking:init_lang(stack_mock:lang(), stack_mock:api_spec()).
  ok
  4> S = stack:new(), stack:push(S, 4), stack:push(S, 6), stack:pop(S).
  6
  5> eqc_mocking:check_callouts(stack_mock:lang()).
  true
  6> eqc_mocking:init_lang(stack_mock:lang(), stack_mock:api_spec()).
  ok
  7> f(S), S = stack:new(), stack:push(S, 4), stack:push(S, 6).
  ok
  8> eqc_mocking:check_callouts(stack_mock:lang()).
  {expected,{call,stack,pop,[stack_ref]}}
  9> eqc_mocking:init_lang(stack_mock:lang(), stack_mock:api_spec()).
  ok
  10> f(S), S = stack:new(), stack:pop(S).
  ** exception exit: {{mocking_error,{unexpected,{call,stack,pop,[stack_ref]}}}},
                      [{eqc_mocking,f5735660_0,
                                   [stack,pop,[stack_ref]],
                                   [{file,"../src/eqc_mocking.erl"},{line,375}]},
                       ... ]}
       in function  eqc_mocking:do_action/3 (../src/eqc_mocking.erl, line 371)

Noteable in the usage above is that we can call init_lang/2 many times without restarting the mocking framework in between. Also note that the check check_callouts/1 checks both that all expected calls where made, and that the calls were made with the expected arguments. If a call is made out of order a failure is reported immediately. If a call is made where the arguments does not match, the default behavior is to also report a failure immediately:

  19> eqc_mocking:init_lang(stack_mock:lang(), stack_mock:api_spec()).
  ok
  20> f(S), S = stack:new(), stack:push(S, 7), stack:push(S, -3), stack:pop(S).
  ** exception exit: {{mocking_error,{unexpected,{call,stack,push,[stack_ref,7]}}},
                     [{eqc_mocking,f5735660_0,
                                   [stack,push,[stack_ref,7]],
                                   [{file,"../src/eqc_mocking.erl"},{line,375}]},
                      ...]}
       in function  eqc_mocking:do_action/3 (../src/eqc_mocking.erl, line 371)

However, it is sometimes useful not to abort tests prematurely, and a small change to the API specification enables this:

  api_spec() -> #api_spec{
    language = erlang,
    modules  = [ #api_module{
      name = stack,
      functions = [ #api_fun{ name = new,  arity = 0 },
                    #api_fun{ name = push, arity = 2, matched = [] },
                    #api_fun{ name = pop,  arity = 1 } ]
      }]}.

Note the added matched = [] that tells eqc_mocking to not check any of the arguments at call time. (Default is matched = all). With this API specification the previous example behaves as follows:

  21> c(stack_mock).
  {ok, stack_mock}
  22> eqc_mocking:start_mocking(stack_mock:api_spec()).
  {ok,<0.116.0>}
  23> eqc_mocking:init_lang(stack_mock:lang(), stack_mock:api_spec()).
  ok
  24> f(S), S = stack:new(), stack:push(S, 7), stack:push(S, -3), stack:pop(S).
  6
  25> eqc_mocking:check_callouts(stack_mock:lang()).
  {unexpected,{call,stack,push,[stack_ref,7]},
              expected,
              {call,stack,push,[stack_ref,4]}}

Stop mocking

Once the mocked modules are no longer needed, the function stop_mocking/0 stops the mocking server and restores (i.e. unloads previously non-existing modules and reverting previously existing modules) the modules to their pre-mocking state.

Advanced usage - matching arguments

As we saw in the example above, strict argument checking is normally done during execution. This default behaviour is not always the wanted one, it is possible in the API specification to give the matching function that one wants to use!

  ... #api_fun{ name = foo, arity = 2, matched = fun([X1, _Y1], [X2, _Y2]) -> X1 == X2 end }

Where the first list of arguments is the actual arguments used in the call of the mocked function and the second list of arguments comes from the language specification. A particular use-case is to check for equality of (some of) the arguments, there is a short-hand notation for this, namely:

  ... #api_fun{ name = bar, arity = 3, matched = [1,2] }

where we are going to match on the first two arguments of bar during execution.

Advanced usage - ?WILDCARD arguments

Sometimes it is not practical to fully describe the expected arguments of a function call. Therefore, it is possible to use ?WILDCARD as an argument in a mocking language specification. This means that any argument is accepted in this position. (Note, that the user has to be very careful when mixing ?WILDCARD's and custom made matching functions.) For example:

?EVENT(module1, fun1, [Arg1, ?WILDCARD, Arg3], Result)

Advanced usage - fallback/passthrough module

Sometimes it is not interesting, or practical, to mock all functions in a module/API. For example some functions are called extremely often, or a function is only called for a side effect that is not interesting at the moment (the canonical example is logging). eqc_mocking provide two alternatives in this case, either the user can specify a custom fallback module, or an existing module can be used as a passthrough module. Functionality wise both options are similar, the only difference is where the fallback functions are defined. (In the passthrough case, behind the scene the existing module is temporary re-named, but this is transparent to the user.) As an example, consider the following API specification:

  api_spec() -> #api_spec{
    language = erlang,
    modules  = [
      #api_module{ name = modX, fallback = modY,
                   functions = [#api_fun{ name = f1, arity = 1},
                                #api_fun{ name = f2, arity = 2}] } ]}.

When used, calls to modX:f1/1 and modX:f2/2 are handled by the mocking framework, while calls to, for example, modX:g/2 would just result in a call to modY:g/2. If the user wants to use a custom fallback module, modY should differ in name from the mocked module modX. The API spec is used to generate a module that handles all calls to modX. This module will simply call modY for all functions not present in API spec. If instead the user want to use the existing functions in modX as fallback functions modY should be modX. Note: To use a module as a passthrough module it must be compiled with debug_info, orelse eqc_mocking cannot properly re-name it.

Note that, in both cases, it is possible to have a function that is mocked (i.e. is in the API specification) and also exists in the fallback/passthrough module. For this case there is an attribute in the #api_fun-record: fallback.

  ... #api_fun{ name = foo, arity = 1, fallback = true }

If fallback = true the call is first handled by the mocking framework, and only if the call cannot be handled (i.e. it is not expected to be called at that point) is it forwarded to the fallback module. If fallback = false the fallback is ignored for this function and an unexpected call will result in a mocking error.

Silent mocking

Sometimes you want to mock a module, but are not interested in calls to functions in this module to show up as events. (This is a special case of fallback/passthrough.) For example, if you do not want to start your logging framwork lager and want to silently accept all calls to lager:info/2.

This kind of silent mocking is best performed by implementing your own lager_mock module, which is less work than specifying the API for each function as a record. The only thing you add to the API spec is:

        #api_module{ name = lager, fallback = lager_mock }.

To quickly create a stub module corresponding to an existing module you can use create_stubs/2 and create_api_spec/1:

  17> eqc_mocking:create_stubs(eqc_mocking:create_api_spec(lager), "/tmp/").
  Writing stub file to: /tmp/lager.erl
  ok

Known limitations

It is not yet possible to safely validate (using validate/1) a language where custom matching functions are used. (It is not possible to know whether two actions may potentially overlap.)
The name of the mocked module(s) should be chosen with care. It should not be a sticky module in Erlang.

Data Types

api_arg_c()

api_arg_c() = #api_arg_c{type = undefined | atom() | string(), stored_type = atom() | string(), name = undefined | atom() | string() | {atom(), string()}, dir = in | out, buffer = false | true | {true, non_neg_integer()} | {true, non_neg_integer(), string()}, phantom = boolean(), matched = boolean(), default_val = no | string(), code = no | string()}

api_fun_c()

api_fun_c() = #api_fun_c{name = undefined | atom(), classify = undefined | any(), ret = atom() | api_arg_c(), args = [api_arg_c()], silent = false | {true, any()}}

api_fun_erl()

api_fun_erl() = #api_fun{name = undefined | atom(), classify = undefined | any(), arity = undefined | non_neg_integer(), fallback = boolean(), matched = [non_neg_integer()] | fun((any(), any()) -> boolean()) | all}

api_module()

api_module() = #api_module{name = undefined | atom(), fallback = undefined | atom(), functions = [api_fun_erl()] | [api_fun_c()]}

api_spec()

api_spec() = #api_spec{language = erlang | c, mocking = atom(), config = undefined | any(), modules = [api_module()]}

event()

event(Action, Result) = {event, Action, Result}

An event. When the language is 'run' an action is matched, and the result (of type Result) is returned.

lang()

The type of a mocking language, parameterized on the Action and the Result.

par()

par(L1, L2) = {par, L1, L2}

Parallel composition of two languages.

perm()

perm(Ls) = {perm, Ls}

Permutation, the language accepts any non-interleaving execution of the languages in Ls.

repl()

repl(L) = {repl, L}

Replication of a language, corresponds rougly to (*) in a regular expression.

seq()

seq(L1, L2) = {seq, L1, L2}

Sequential composition of two languages.

xalt()

xalt(L1, L2) = {xalt, L1, L2} | {xalt, term(), L1, L2}

Choice between two languages. Possibly tagged with a term. All conditionals tagged with the same term must make the same choice.

Function Index

callouts_to_mocking/1	Translate a callout-term, as return by eqc_component, to a mocking-language.
check_callouts/1	Checks that the correct callouts have been made.
create_api_spec/1	Create an API spec for existing module/modules (mocking all functions listed by Module:module_info(exports).
create_stubs/2	Create stub skeletons (.erl file) for modules in an API spec.
get_choices/0	Returns the branches taken in any tagged choices.
get_trace/1	Returns the trace, i.e.
get_trace_within/1	Same as 'get_trace' but waits (for Timeout ms) for enough actions to match the current language.
info/0	Information about what is currently being mocked.
init_lang/1	Initialize the mocking language.
is_lang/1	Recognizer for the mocking language.
merge_spec_modules/1	Safely merge multiple specifications of the same module.
provide_return/2	Equivalent to `provide_return(L, As, fun (X, Y) -> X == Y end)`.
provide_return/3	Given a language and a sequence of actions tries to provide a list of the respective results.
small_step/2	Equivalent to `small_step(A, L, fun (X, Y) -> X == Y end)`.
small_step/3	Implementation of the small step semantics for language.
start_global_mocking/1	Equivalent to `start_mocking(APISpec, [], [global])`.
start_mocking/1	Equivalent to `start_mocking(APISpec, [], [])`.
start_mocking/2	Equivalent to `start_mocking(APISpec, Components, [])`.
start_mocking/3	Initialize mocking, mocked modules are created as specified in the supplied callout specification.
stop_mocking/0	Gracefully stop mocking.
trace_verification/2	Equivalent to `trace_verification(L, As, fun (X, Y) -> X == Y end)`.
trace_verification/3	Similar to `provide_return/3`, but returns a boolean if the sequence of actions leads to an accepting state.
validate/1	Validate a mocking language.

Function Details

callouts_to_mocking/1

callouts_to_mocking(CalloutTerm::term()) -> lang(_Action, _Result)

Translate a callout-term, as return by eqc_component, to a mocking-language.

check_callouts/1

check_callouts(MockLang) -> true | Error

MockLang = lang(_Action, Result)
Error = {expected, Call} | {expected_one_of, [Call]} | {unexpected, Call} | {unexpected, Call, expected, Call} | {unexpected, Call, expected_one_of, [Call]} | {ambiguous, Call, cant_choose_between, [{Call, '->', Result}]}
Call = {call, module(), atom(), [any()]}

Checks that the correct callouts have been made. Given a language, checks that the mocked functions called so far matches the language (the argument checking during execution is normaly relaxed, so errors could be undiscovered until here). Also checks that the language is in an "accepting state", i.e. a state where it is ok to stop.

create_api_spec/1

create_api_spec(Module::module() | [module()]) -> api_spec()

Create an API spec for existing module/modules (mocking all functions listed by Module:module_info(exports). Don't forget to load the record definitions in your shell or you will not get well formatted output.

 4> rr(code:lib_dir(eqc) ++ "/include/eqc_mocking_api.hrl").
 [api_arg_c,api_fun,api_fun_c,api_module,api_spec]
 5> eqc_mocking:create_api_spec(foo)
 #api_spec{language = erlang,mocking = eqc_mocking,config = undefined,
           modules = [#api_module{name = foo,fallback = undefined,
                                  functions = [#api_fun{name = bar,classify = undefined,arity = 1,
                                                        fallback = false,matched = all},
                                               #api_fun{name = baz,classify = undefined,arity = 3,
                                                        fallback = false,matched = all}]}]}
 6>

create_stubs/2

create_stubs(APISpec::api_spec(), Path::string()) -> ok

Create stub skeletons (.erl file) for modules in an API spec.

get_choices/0

get_choices() -> [{term(), left | right}]

Returns the branches taken in any tagged choices.

get_trace/1

get_trace(APISpec::api_spec()) -> [_Action]

Returns the trace, i.e. the {Action, Result}-pairs seen so far for the current language.

get_trace_within/1

get_trace_within(Timeout::integer()) -> reference()

Same as 'get_trace' but waits (for Timeout ms) for enough actions to match the current language. The trace is returned in a message {trace, Ref, Trace} sent to the calling process, where Ref is the result of the call.

info/0

info() -> [{atom(), #linfo{}}]

Information about what is currently being mocked.

init_lang/1

init_lang(MockLang) -> ok | no_return()

MockLang = lang(_Action, _Result)

Initialize the mocking language. This sets the language describing how the mocked modules should behave. Calling this function does not reset the collected trace. Some sanity checks are made, throws an error if the language and the callout specification are inconsistent.

is_lang/1

is_lang(MockLang::lang(_Action, _Result)) -> boolean()

Recognizer for the mocking language.

merge_spec_modules/1

merge_spec_modules(MockMods::[api_module()]) -> [api_module()] | no_return()

Safely merge multiple specifications of the same module.

provide_return/2

provide_return(MockLang, Actions) -> [Result] | Error

MockLang = lang(Action, Result)
Actions = [Action]
Error = {expected, Call} | {expected_one_of, [Call]} | {unexpected, Call} | {unexpected, Call, expected, Call} | {unexpected, Call, expected_one_of, [Call]} | {ambiguous, Call, cant_choose_between, [{Call, '->', Result}]}
Call = {call, module(), atom(), [any()]}

Equivalent to provide_return(L, As, fun (X, Y) -> X == Y end).

provide_return/3

provide_return(MockLang, Actions, MatchFun) -> [Result] | Error

MockLang = lang(Action, Result)
Actions = [Action]
MatchFun = fun((Action, Action) -> boolean())
Error = {expected, Call} | {expected_one_of, [Call]} | {unexpected, Call} | {unexpected, Call, expected, Call} | {unexpected, Call, expected_one_of, [Call]} | {ambiguous, Call, cant_choose_between, [{Call, '->', Result}]}
Call = {call, module(), atom(), [any()]}

Given a language and a sequence of actions tries to provide a list of the respective results. If the actions does lead to an inconsistent state or if the sequence of actions does not lead to an accepting state a descriptive error is returned.

small_step/2

small_step(Step, MockLang) -> Res

Step = Action
MockLang = lang(Action, Result)
Res = {ok, Result, lang(Action, Result)} | fail | {ambiguous_callout, boolean()}

Equivalent to small_step(A, L, fun (X, Y) -> X == Y end).

small_step/3

small_step(Step, MockLang, MatchFun) -> Res

Step = Action
MockLang = lang(Action, Result)
MatchFun = fun((Action, Action) -> boolean())
Res = {ok, Result, lang(Action, Result)} | fail | {ambiguous_callout, boolean()}

Implementation of the small step semantics for language.

start_global_mocking/1

start_global_mocking(APISpec) -> Result

APISpec = #api_spec{language = erlang, mocking = atom(), config = undefined | any(), modules = [api_module()]}
Result = {ok, pid()} | ignore | {error, term()} | no_return()

Equivalent to start_mocking(APISpec, [], [global]).

start_mocking/1

start_mocking(APISpec) -> Result

APISpec = #api_spec{language = erlang, mocking = atom(), config = undefined | any(), modules = [api_module()]}
Result = {ok, pid()} | ignore | {error, term()} | no_return()

Equivalent to start_mocking(APISpec, [], []).

start_mocking/2

start_mocking(APISpec, Components) -> Result

APISpec = #api_spec{language = erlang, mocking = atom(), config = undefined | any(), modules = [api_module()]}
Components = [atom()]
Result = {ok, pid()} | ignore | {error, term()} | no_return()

Equivalent to start_mocking(APISpec, Components, []).

start_mocking/3

start_mocking(Spec, Components, Options) -> any()

Initialize mocking, mocked modules are created as specified in the supplied callout specification. Each mocked function is merely calling do_action/3. If the specification contains an error, an error is thrown. Functions that are modeled by a component in Components are not mocked!

Options:

global - Use a global gen_server to handle mocked calls. Required if mocked functions are called in a distributed setting.

stop_mocking/0

stop_mocking() -> ok

Gracefully stop mocking. Shutdown the mocking server, will try to restore mocked modules to its pre-mocking version.

trace_verification/2

trace_verification(MockLang, Actions) -> boolean()

MockLang = lang(Action, _Result)
Actions = [Action]

Equivalent to trace_verification(L, As, fun (X, Y) -> X == Y end).

trace_verification/3

trace_verification(MockLang, Actions, MatchFun) -> boolean()

MockLang = lang(Action, _Result)
Actions = [Action]
MatchFun = fun((Action, Action) -> boolean())

Similar to provide_return/3, but returns a boolean if the sequence of actions leads to an accepting state.

validate/1

validate(MockLang::lang(_Action, _Result)) -> boolean()

Validate a mocking language. Checks that the mocking language is not ambiguous.

Generated by EDoc, Sep 18 2017, 16:17:37.

Module eqc_mocking

?EVENT(Mod, Fun, Args, Result)

?SEQ(Ls)

?SEQ(L1, L2)

?PAR(Ls)

?PAR(L1, L2)

?XALT(L1, L2)

?REPLICATE(L)

?PERM(Ls)

?SUCCESS

How to mock a module

Stop mocking

Advanced usage - matching arguments

Advanced usage - ?WILDCARD arguments

Advanced usage - fallback/passthrough module

Silent mocking

Known limitations

`?EVENT(Mod, Fun, Args, Result)`

`?SEQ(Ls)`

`?SEQ(L1, L2)`

`?PAR(Ls)`

`?PAR(L1, L2)`

`?XALT(L1, L2)`

`?REPLICATE(L)`

`?PERM(Ls)`

`?SUCCESS`