JavaScript Tests Mocha, Mocking, Sinon, Spies (by Joe Eames from Pluralsight) (in progress!)
-
Joe Eames @josepheames
Table of Contents
Chapter 1 - Mocha
- TDD Style Mocha
- BDD Style Mocha
- Exclusive Run
describe
block of tests withdescribe.only
- Exclusive Run
it
test withit.only
- Skip
describe
block of tests withdescribe.skip
- Skip test with
it.skip
- Pending tests by excluding the callback i.e.
it('...');
- Ignore specific global variables
- Ignore all global variables (not throw errors when found)
- Asynchronous Tests using BDD Style Mocha
- Use with
setTimeout
andsetInterval
calls) - Use to test UI effects
- Do not use unit tests for AJAX calls, instead use Mocking
- Use with
- Asychronous Tests
- Globally override Mocha default timeout is 2 seconds.
- Cont’d
- Locally override Mocha default timeout is 2 seconds.
- Integration with CI using Mocha, TeamCity and PhantomJS executable (capture output)
- Download PhantomJS headless Webkit
phantomjs
executable and copy to project - Multiple TeamCity Reporters setup for HTML test runners, i.e.
- Download PhantomJS headless Webkit
* Create custom `run-mocha.js` file
* Note: WebKit (engine behind Chrome)
* Note: PhantomJS (JS runtime environment that wraps around a version of WebKit engine without browser overhead)
* Note: PhantomJS is insufficient for cross-browser compatibility testing, as it
wraps around WebKit version that does not verify JS will run on multiple browsers and versions
* Note: Cross-browser testing with CI using 3rd parties
* PhantomJS translates JS console.log into format to expected by CI server to determine if tests passing/failing
- Failing test that completes before assert runs, unless stop running tests
until notified
stop();
and instruct tostart();
so it does not run code after thesetTimeout
first
- Definitions:
- System Under Test (SUT)
- Mock Object Types (Test Doubles)
- Dummy objects - passed around but not actually used other than to fill parameter lists.
- Fake objects - simplified working implementation so not suitable for production
- Stubs - return canned/predefined answers to test calls
- Spies (enhanded Stub) - spies hold/record info on how method called and with what parameters. (i.e. email service records quantity of messages received)
- Mock - pre-programmed specification of calls expected to receive, throwing exceptions if a method receives unexpected calls, verification checks that mocks receive all expected calls
- Mocking Benefits
- Test component isolated from SUT (i.e. remove costs of interaction with external or complex systems with associated versions)
- Test interactions between Components
- Flow control simple (i.e. avoid declaring state to return true/false, instead create Mock objects to so)
- Methodology
- No interface of class for mocking against in JS
- Freestanding functions outside JS classes
- Mocking Functions - replace function with Test Double
- Mock Object - Options (2 OFF) for creating a Mock Interface from which to create a Mocking Object:
- Create Fake object (Test Double) manually with same interface as real object being mocked
-
Disadvantage: requires updating Test Double to match changes to Object
-
Example: Given prototypical inheritance class defined using constructor function and prototype modification. Mock the class by creating a new object with the same interface as the class
-
- Create Fake object (Test Double) manually with same interface as real object being mocked
* Create Real object instance using existing object and mock it (to defined an interface of it)
* Disadvantage: Object constructor may have side-effects we do not want in our test
* Example: Create live instance of class and use as interface definition to be mocked
- Spy Options in Mocking Implementation (allows testing how SUT calls its collaborators to check performed correctly)
- Replacing Methods with Spies
- Spy methods replace all methods of the Mocked class
- Intercepting Actual Methods with Spies
- Insert Spy method between class method and real implementation
- Note: Both Spy implementations return predefined response, whilst tracking quantity of times called and parameters provided
- Replacing Methods with Spies
- Example 1 Mocking Library Implementation (Manual Mocking)
- Example 2 Mocking Library Implementation (Pass-Through Mocking)
- Passes context and original function through as parameters
- Spies
- Jasmine Spies
- Jasmine Spies are subset of Sinon.JS
- Jasmine Spies are built for mocking single functions
- Spying on freestanding functions (i.e. callbacks)
- Jasmine Spies
- Cont’d
- Cont’d
spyOn
global function of Jasmine Spies (attached to window object)- Used to spy on method of object
- Create a spy on a dependency’s functions that is used in a class being tested (replacing the old function)
- Use
spyOn(objectWithMethodToFake, "methodNameAsString")
withandReturn
- Use
andCallFake
to call an actual fake implementation for the spy - Use a spy that dispatches call to the real implementation using
andCallThrough
, and monitor how many times it is called, and monitor arguments passed - Cause spy to throw error
andThrow
to test class exception handling
- Cont’d
- Cont’d
- Cont’d
createSpyObj
function of Jasmine Spies- Creates new object with only spy methods (not need hijack existing object)
- Usage is when object under test has dependency with many
functions we may be calling (instead of having to repeatedly call
spyOn
we just callcreateSpyObj
and all spies already set up)
- Cont’d
- Jasmine Spy supported Matchers
toHaveBeenCalled
- asserts spy called at least oncenot.toHaveBeenCalled
- asserts spy never calledtoHaveBeenCalledWith(arg1, arg2...)
- asserts spy called at least once with all specific args provided (no guarantee about multiple calls)
- Cont’d
not.toHaveBeenCalledWith(arg1, arg2...)
- asserts spy never called with specific set of args
- Jasmine Spy supported Metadata in assert calls
calls[]
- array of objects representing each time spy called. each object has 2 OFF properties with info about specific invocationcalls[].args[]
property is array of each arg passed for invocationcalls[].object
property is object that was context of the call (i.e. object that wasthis
then spy was invoked), and is useful for callbacks to determine object they were called oncallCount
convenience method instead ofcalls.length
mostRecentCall
gets last callargsForCall[]
- Jasmine Spy supported Utility methods
jasmine.isSpy
- check if object is a spy or notreset
- method on spies to reset them
Chapter 2 - SinonJS
- SinonJS (mocking library, not a test framework)
- Configure to use Sinon for spies in HTML spec runner
- Sinon Spies Usages
- Option 1 - test double for single function (i.e. taking callback)
- Option 2 - watch object’s existing methods (i.e. using original implementation)
similar to
andCallThrough
- Create Sinon Spies by either:
sinon.spy
method without params creates anon function without return value suitable when callback required for testing
- Sinon Spy API
- Supports Sinon Stubs and Sinon Mocks
- Sinon Call API
- Sinon Assertions
- Benefit of usage: Use optionally instead of assertions that come with testing framework since more explicit/expressive/readable and error messages clearer upon assertion failure
- Example with deliberate failure for each test as they both create the Sinon Spy but neither of them actually call it before asserting it was called
- Sinon Assertion API - assertions built-in to Sinon
- Available by using
sinon.assert.___
.called
.notCalled
- spy was never called.calledOnce
- takes spy as single parameter.calledTwice
- as above.calledThrice
- as above.callCount(spy, num
- takes the spy and a number of times we expect spy was called, as parameters.callOrder(spy1, spy2...)
- checks certain set of spies called in specific order.calledOn(spy, obj)
- checks a spy was called with a given context.alwaysCalledOn(spy, obj)
- checks a spy was always called with a given context.calledWith(spy, args...)
- checks a spy was called with specific set of args.alwaysCalledWith(spy, args...)
- checks every call on a spy.neverCalledWith(spy, args...)
- checks spy was never called with given args.calledWithExact(spy, args...)
- checks spy was called with exact set of args.calledWithMatch(spy, matches...)
.alwaysCalledWithMatch(spy, matches...)
.neverCalledWithMatch(spy, matches...)
.threw(spy)
- checks spy threw an exception.threw(spy, exception)
- checks spy threw a specific exception.alwaysThrew(spy, exception)
- checks spy always threw a specific exception
- Available by using
- Sinon Stubs
- Definition:
- Sinon Stubs are the second form of Test Double provided by SinonJS
- Sinon Stubs are Sinon Spies that have pre-programmed behaviour
- Sinon Stubs support Sinon Spy API + additional methods to control behaviour
- Sinon Stubs may be anonymous or wrap existing methods
- Sinon Stubs DO NOT call the original function, whereas Sinon Spies DO
- Sinon Stub usage purposes:
- Control behaviour
- Suppress existing behaviour (since actual underlying implementation is not called)
- Verification of behaviour (i.e. check Sinon Stubs were called in specific manner)
- Definition:
- Sinon Stub Creation (3 OFF ways)
- Sinon Stub API
.returns(obj)
- specify that whenever call Stub it will return the Object passed as param.throws
- tells Sinon to throw general exception whenever given Stub is called.throws("type")
- tells Sinon to throw a particular type of exception whenever given Stub is called.throws(obj)
- tells Sinon to throw particular object whenever given Stub is called.withArgs(args...)
- allows customise particular Sinon Stub on a per-invocation basis (i.e. tell it to act a certain way when called with certain set of args).returnsArg(index)
- tells Sinon to return the arg at index in array of provided args.callsArg(index)
- instruct Sinon to call arg at index in array of provided args (implies that the arg at that index is a Function).callsArgOn(index, context)
- same as above, but can pass in context for Sinon to call the Function arg at given index.callsArgWith(index, args...)
- same as above, but can pass a set of args to the Function arg at given index that gets called.callsArgOnWith(index, context, args...)
- same as above, but can pass in context for Sinon to call the Function arg at given index, passing a given set of args to it- Stub method call one or more Callbacks that are passed to it (i.e. more complex than
callsArg
- See docs for other more advanced API methods
- Sinon Stub Example
- Objective is to test using Stubs whether the Boxing class
.punch
method works correctly - Note that implementations of methods irrelevant to testing the
.punch
method are commented out as they don’t matter
- Objective is to test using Stubs whether the Boxing class
- Sinon Mocks
- Definition:
- Sinon Mocks are the third form of Test Double provided by SinonJS
- Sinon Mocks are like Sinon Stubs but have pre-programmed expectations
- Sinon Mocks fail tests if pre-programmed expectations not met
- Sinon Mocks state assertions before/upfront (instead of afterwards)
- Guidelines
- Try to Only use ONE Sinon Mock Object per test
- Try to Only use 1-2 Expectations per test
- Try to Only use 1-2 Assertions that are External to the mock (try to only use the Sinon Mock Object with the test, not others)
- Definition:
- Different Architectures between Spies, Stubs, and Mocks
- Sinon Spy Archi - Wrap old fn with new fn and use new fn in place of old one
- WITHOUT SPY - MyFn —-> Orig Fn
- WITH SPY - MyFn —-> Spy Wrap Fn ( Orig Fn + Spy API )
- Sinon Stub Archi - Sinon takes original method on existing object,
and replaces reference to the original method with a brand new method,
then set expectations (AFTER actual action takes place)
- WITHOUT STUB - MyObj —-> Orig Fn
- WITH STUB - MyObj —-> Stub Fn ( + Spy API + Stub API )
- Sinon Mock Archi - Create a mock of an object, and then create expectations (BEFORE actual action takes place) on the methods of the mocked object, such that each expectation causes the orig fn to be replaced with a mock fn (discarding orig implementation) (instead of wrapping orig fns or replacing orig fns) * WITHOUT STUB - MyObj —-> Orig Fn 1 | ‘—-> Orig Fn 2 * WITH STUB - MyObj —-> Orig Fn 1 <———————————-, | | | | `—-> Orig Fn 2 <——————————-, | | | | Mock – Orig Fn 1 Expectation ( + Spy API + Stub API )——’ | | |__ Orig Fn 2 Expectation ( + Spy API + Stub API ) __|
- Sinon Spy Archi - Wrap old fn with new fn and use new fn in place of old one
- Sinon Mock Usage
- Sinon Mock API
- Note: Each method returns the expectation to allow chaining into compound expressions
expectation.atLeast(number)
- tell mock to fail test if specific mock fn is not invoked at least n timesexpectation.atMost(number)
expectation.never
- tell mock to fail test if mock method is never calledexpectation.once
- tell mock to fail test if an expectation isn’t called exactly onceexpectation.twice
expectation.thrice
expectation.exactly(number)
expectation.withArgs(args...)
- tell mock to fail test if mock method is not called at least once with given argsexpectation.withExactArgs(args...)
expectation.on(obj)
- tell mock to fail test if mock method is not called at least once with given object as the contextexpectation.verify
- checks that all expectations have been met, and if any not met then the test fails
- Sinon Mock Example
- Objective is to test using Mock instead of Stub to assert
whether the Boxing class
.punch
method works correctly - Note that either Stubs or Mocks may be used to VERIFY BEHAVIOR
- Note that implementations of methods irrelevant to testing the
.punch
method are commented out as they don’t matter - Note: Using Stubs, Functions are replaced with Stubs in-place. Using Mocks however, we have a separate Mock Object that we talk to
- Objective is to test using Mock instead of Stub to assert
whether the Boxing class
- Sinon Matchers
- Definition: Fuzzy matching of calls to a Test Double based on args used
(without exactly specifying the args). Use them in place of an arg to check
if Test Double called correctly
- i.e. matcher to check spy called with any number as an arg
- i.e. matcher to check spy called with a function as an arg
- Example usage: Create spy, call the spy, and call
calledWithMatch
method on the spy (passing in one or more Matchers)
- Definition: Fuzzy matching of calls to a Test Double based on args used
(without exactly specifying the args). Use them in place of an arg to check
if Test Double called correctly
- Sinon Matchers API
sinon.match(number)
- verifies arg matches the given number exactlysinon.match(string)
- verifies arg matches the given string or a substring to match the expectation provided. SEE EXAMPLE BELOWsinon.match(regexp)
- matches arg if was string matching the given regexsinon.match(object)
- matches arg if was object matching the given objectsinon.match(function)
- Creates a Custom Matchersinon.match.any
- matches any arg givensinon.match.defined
- matches any arg was anything but not undefinedsinon.match.truthy
- matches any arg that is truthysinon.match.falsy
sinon.match.bool
sinon.match.number
sinon.match.string
sinon.match.object
- matches any arg that is an object (not a primitive value)sinon.match.func
- match any arg that is a functionsinon.match.array
sinon.match.regexp
sinon.match.date
sinon.match.same(ref)
- matches only if object that was passed as arg is the same as the arg given to thesame
matcher. SEE EXAMPLE BELOWsinon.match.typeOf(type)
- matches when arg is of given type. type can be either:"undefined"
,"null"
,"boolean"
,"number"
,"string"
,"object"
,"function"
,"array"
,"regex"
,"date"
sinon.match.instanceOf(type)
- requires value to be an instance of given typesinon.match.has(property[,expectation])
- matches when object of arg has at least one of the properties given are the same as given tohas
. SEE EXAMPLE BELOW another parameter may be given as the ‘value’ of the property (to be more exact)sinon.match.hasOwn(property[,expectation])
- same ashas
but property must be defined on the object itself, and cannot be inherited
- Cont’d
- Custom Sinon Matcher
- Purpose: create for flexibility when Sinon Matchers API insufficient for use case
- Example Usage: Create custom matcher that passes “successfully” if given value is within a range (for example.
- Note: second param is just a readable version of the name that is used when given arg fails to match against the Matcher
- Custom Sinon Matcher
- Cont’d
- Combining Multiple Sinon Matchers
- Example: Support passing either a ‘string’ or a ‘function’ as first param
of a method by using the
and
andor
functions (available on every matcher)sinon.match.func.or(sinon.match.string)
- Example: Support passing either a ‘string’ or a ‘function’ as first param
of a method by using the
- Combining Multiple Sinon Matchers
- Mock Timers with Sinon Fake Timers
- Similar to Jasmine’s mock clock
- Clock Object allows control time during tests with
setTimeout
andsetInterval
calls - Allows control of Dates (i.e.
Date.now
) - Integrates with jQuery animations (i.e. move forward with time to when jQuery animation callback fires)
- Note: Import
sinon-ie
(Custom Sinon Library) when working with IE
- Sinon Fake XMLHttpRequest (Test Double)
- Sinon controls XHR Object in browser but not all requests will receive a response
- Fake implementation of XMLHttpRequest (XHR) Object
- Tests that verify/inspect each request from SUT making AJAX calls and associated responses
- Tests that alternatively use XHR Object for canned responses
- Create Fake Request
- Option 1
useFakeXmlHttpRequest
API low-level method - Option 2
fakeServer
API higher-level method for hijacking XHR object in browser that sets up a pattern for responses to allow inspection of specific requests/responses
- Option 1
- Option 1
useFakeXmlHttpRequest
- Cont’d
- Purpose of Test and what is being tested
- Inspect request object (Fake XHR Object) to test code is correctly sending requests/data
- Testing original objects but want Fake XHR Object to respond the way a server would
- Purpose of Test and what is being tested
- FakeXMLHttpRequest API
- Use Fake XHR Object properties to inspect the request object and asset made correctly
request.url
- url that made requestrequest.method
- http method maderequest.requestHeaders
- http request headersrequest.requestBody
request.status
- status of Response to http request depending on how Sinon Fake XHR Object setuprequest.statusText
request.username
request.password
request.responseXml
- may have a value based on http request headersrequest.responseText
- text of response if not an XML based responserequest.getResponseHeader(header)
- inspect specific response headerrequest.getAllResponseHeaders()
- all response headers as string
- Use Fake XHR Object properties to inspect the request object and asset made correctly
- FakeXMLHttpRequest Response API
request.respond(status, headers, body)
// status (int), headers (obj), body/data (string) Sinon responds when call on Fake XHRrequest
object causing callbacks to fire or promises to completerequest.setResponseHeaders(object)
// set request header separatelyrequest.setResponseBody(body)
- Option 2
fakeServer
- Example
- Fake Server API
server.respondWith(response)
server.respondWith(url, response)
- with overload for custom response to urlserver.respondWith(method, url, response)
- with overload for custom response to url for specific methodsserver.respondWith(urlRegExp, response)
- with overload, so vague as to which urls to respond toserver.respondWith(method, urlRegExp, response)
server.respond()
- causes all requests to be responded to
- Sandboxing
- Post-testing ensure global objects restored to original state, including:
- Spies, Stubs, Mocks, Fake Timers, Fake XHRs
- Implement Sandboxing with either
- Option 1:
sinon.sandbox.create
- Option 2:
sinon.test()
- use this method as wrapper to test callback
- Option 1:
- Post-testing ensure global objects restored to original state, including:
Links
Written on December 7, 2016