Multiple expectations that can happen in every order

[jtomaszewski]

It's pretty cool that you can add expectations over time, like this:

expect(UserAPI, :get_user, 1, fn 123 -> {:ok, %{id: 123} end)
expect(UserAPI, :get_user, 1, fn 456 -> {:ok, %{id: 456} end)

In such a way you can define expectations that the UserAPI.get_user will be called exactly twice, once with 123 and once with 456 argument.

Problem appears when you don't care about the exact order of function invocations, but you still care about how many and with what arguments they will be called. For example, imagine my code that retrieves users, does it in parallel (e.g. using Task.async). Then I'm not sure anymore if User API will first receive the 123 or 456 in the request.

Yet, I want to verify that it had been called twice - once with 123, once with 456, in any order.

Is that possible with mox?

For example, in JavaScript's Jest library it is possible in such a way:

expect(getUserMock).toHaveBeenCalledTimes(2);
expect(getUserMock).toHaveBeenCalledWith(123);
expect(getUserMock).toHaveBeenCalledWith(456);

or like this

expect(getUserMock.calls).toEqual( expect.arrayContaining([123, 456]) );

[josevalim]

Yes:

expect(UserAPI, :get_user, 2, fn
  123 -> {:ok, %{id: 123}
  456 -> {:ok, %{id: 456}
end)

:)

[jtomaszewski]

It doesn't do what I need. It will pass if UserAPI.get_user gets called with 123 twice. I need it to pass only if it gets called once with 123, and once with 456.

Currently to make it happen, I'm afraid I'd have to maintain some data in the process (mutate it whenever the mocked function gets called, and verify it in the end of the test scenario). This sounds hard to do, and a lot of boilerplate to be written for such a simple expectation. I'd love the mocking library to do it for me instead.

What do you think?

[josevalim]

Ah yes, you are right. A pull request that adds this functionality is welcome then! But note it is not hard to achieve this functionality on your end:

parent = self()

expect(UserAPI, :get_user, 2, fn id ->
  send(parent, {:id, id})
  {:ok, %{id: id}}
end)

# do the call

assert_received {:id, 123}
assert_received {:id, 456}

[whatyouhide]

The API for this is not trivial IMO. The reason is that I don't think we should rely on pattern matching, because you could have something like:

UserAPI
|> expect(:get_user, fn user_id -> assert user_id == 123 end)
|> expect(:get_user, fn user_id -> assert user_id == 456 end)

If you have this, Mox kind of has no way to make sure that the two functions were called, right? So, I think what you're looking for is assertions on the arguments with which mock functions get called, which is something that Mox doesn't strictly provide right now in any of its APIs.

Maybe there's an alternative we could go for here. We could introduce something to retrieve the expectations that were called. A sketch of an API:

expect(UserAPI, :get_user, 2, fn id ->
  # Do your thing
  {:ok, %{id: id}}
end)

# Do the call

Mox.get_executed_calls(UserAPI)
#=> [
#=>   {:get_user, [123]},
#=>   {:get_user, [456]}
#=> ]

Thoughts @josevalim and @jtomaszewski?

[jtomaszewski]

The API for this is not trivial IMO.

Agree.

We could introduce something to retrieve the expectations that were called.

If that would be possible from implementation side, then yeah, why not, I think it would be great.

For example, something similar is possible in JS when you're testing with jest:

const fn = jest.fn();

fn(1);
fn(2);

expect(fn.mock.calls).toEqual([
  [1],
  [2]
]);

And I love using that API, it's super clear. I miss that in Elixir - I have to do that thing with send(parent, ...) and assert_received to assert all calls. (And sometimes I'm still not sure if I'm able to assert the exact order of how the calls came in.)