summaryrefslogtreecommitdiffstats
path: root/runtime/doc/testing.txt
diff options
context:
space:
mode:
Diffstat (limited to 'runtime/doc/testing.txt')
-rw-r--r--runtime/doc/testing.txt633
1 files changed, 633 insertions, 0 deletions
diff --git a/runtime/doc/testing.txt b/runtime/doc/testing.txt
new file mode 100644
index 0000000..dabdd32
--- /dev/null
+++ b/runtime/doc/testing.txt
@@ -0,0 +1,633 @@
+*testing.txt* For Vim version 9.1. Last change: 2023 May 18
+
+
+ VIM REFERENCE MANUAL by Bram Moolenaar
+
+
+Testing Vim and Vim script *testing-support*
+
+Expression evaluation is explained in |eval.txt|. This file goes into details
+about writing tests in Vim script. This can be used for testing Vim itself
+and for testing plugins.
+
+1. Testing Vim |testing|
+2. Test functions |test-functions-details|
+3. Assert functions |assert-functions-details|
+
+==============================================================================
+1. Testing Vim *testing*
+
+Vim can be tested after building it, usually with "make test".
+The tests are located in the directory "src/testdir".
+
+There are two types of tests added over time:
+ test20.in oldest, only for tiny builds
+ test_something.vim new style tests
+
+ *new-style-testing*
+New tests should be added as new style tests. The test scripts are named
+test_<feature>.vim (replace <feature> with the feature under test). These use
+functions such as |assert_equal()| to keep the test commands and the expected
+result in one place.
+ *old-style-testing*
+These tests are used only for testing Vim without the |+eval| feature.
+
+Find more information in the file src/testdir/README.txt.
+
+==============================================================================
+2. Test functions *test-functions-details*
+
+test_alloc_fail({id}, {countdown}, {repeat}) *test_alloc_fail()*
+ This is for testing: If the memory allocation with {id} is
+ called, then decrement {countdown}, and when it reaches zero
+ let memory allocation fail {repeat} times. When {repeat} is
+ smaller than one it fails one time.
+
+ Can also be used as a |method|: >
+ GetAllocId()->test_alloc_fail()
+
+
+test_autochdir() *test_autochdir()*
+ Set a flag to enable the effect of 'autochdir' before Vim
+ startup has finished.
+
+
+test_feedinput({string}) *test_feedinput()*
+ Characters in {string} are queued for processing as if they
+ were typed by the user. This uses a low level input buffer.
+ This function works only when with |+unix| or GUI is running.
+
+ Can also be used as a |method|: >
+ GetText()->test_feedinput()
+
+
+test_garbagecollect_now() *test_garbagecollect_now()*
+ Like garbagecollect(), but executed right away. This must
+ only be called directly to avoid any structure to exist
+ internally, and |v:testing| must have been set before calling
+ any function. *E1142*
+ This will not work when called from a :def function, because
+ variables on the stack will be freed.
+
+
+test_garbagecollect_soon() *test_garbagecollect_soon()*
+ Set the flag to call the garbagecollector as if in the main
+ loop. Only to be used in tests.
+
+
+test_getvalue({name}) *test_getvalue()*
+ Get the value of an internal variable. These values for
+ {name} are supported:
+ need_fileinfo
+
+ Can also be used as a |method|: >
+ GetName()->test_getvalue()
+<
+ *test_gui_event()*
+test_gui_event({event}, {args})
+ Generate a GUI {event} with arguments {args} for testing Vim
+ functionality. This function works only when the GUI is
+ running.
+
+ {event} is a String and the supported values are:
+ "dropfiles" drop one or more files in a window.
+ "findrepl" search and replace text.
+ "mouse" mouse button click event.
+ "scrollbar" move or drag the scrollbar.
+ "key" send a low-level keyboard event.
+ "tabline" select a tab page by mouse click.
+ "tabmenu" select a tabline menu entry.
+
+ {args} is a Dict and contains the arguments for the event.
+
+ "dropfiles":
+ Drop one or more files in a specified window. The supported
+ items in {args} are:
+ files: List of file names
+ row: window row number
+ col: window column number
+ modifiers: key modifiers. The supported values are:
+ 0x4 Shift
+ 0x8 Alt
+ 0x10 Ctrl
+ The files are added to the |argument-list| and the first
+ file in {files} is edited in the window. See |drag-n-drop|
+ for more information. This event works only when the
+ |drop_file| feature is present.
+
+ "findrepl":
+ {only available when the GUI has a find/replace dialog}
+ Perform a search and replace of text. The supported items
+ in {args} are:
+ find_text: string to find.
+ repl_text: replacement string.
+ flags: flags controlling the find/replace. Supported
+ values are:
+ 1 search next string (find dialog)
+ 2 search next string (replace dialog)
+ 3 replace string once
+ 4 replace all matches
+ 8 match whole words only
+ 16 match case
+ forward: set to 1 for forward search.
+
+ "mouse":
+ Inject either a mouse button click, or a mouse move, event.
+ The supported items in {args} are:
+ button: mouse button. The supported values are:
+ 0 left mouse button
+ 1 middle mouse button
+ 2 right mouse button
+ 3 mouse button release
+ 4 scroll wheel down
+ 5 scroll wheel up
+ 6 scroll wheel left
+ 7 scroll wheel right
+ row: mouse click row number. The first row of the
+ Vim window is 1 and the last row is 'lines'.
+ col: mouse click column number. The maximum value
+ of {col} is 'columns'.
+ multiclick: set to 1 to inject a multiclick mouse event.
+ modifiers: key modifiers. The supported values are:
+ 4 shift is pressed
+ 8 alt is pressed
+ 16 ctrl is pressed
+ move: Optional; if used and TRUE then a mouse move
+ event can be generated.
+ Only {args} row: and col: are used and
+ required; they are interpreted as pixels or
+ screen cells, depending on "cell".
+ Only results in an event when 'mousemoveevent'
+ is set or a popup uses mouse move events.
+ cell: Optional: when present and TRUE then "move"
+ uses screen cells instead of pixel positions
+
+ "scrollbar":
+ Set or drag the left, right or horizontal scrollbar. Only
+ works when the scrollbar actually exists. The supported
+ items in {args} are:
+ which: Selects the scrollbar. The supported values
+ are:
+ left Left scrollbar of the current window
+ right Right scrollbar of the current window
+ hor Horizontal scrollbar
+ value: Amount to scroll. For the vertical scrollbars
+ the value can be between 0 to the line-count
+ of the buffer minus one. For the horizontal
+ scrollbar the value can be between 1 and the
+ maximum line length, assuming 'wrap' is not
+ set.
+ dragging: 1 to drag the scrollbar and 0 to click in the
+ scrollbar.
+
+ "key":
+ Send a low-level keyboard event (e.g. key-up or down).
+ Currently only supported on MS-Windows.
+ The supported items in {args} are:
+ event: The supported string values are:
+ keyup generate a keyup event
+ keydown generate a keydown event
+ keycode: Keycode to use for a keyup or a keydown event.
+ *E1291*
+
+ "tabline":
+ Inject a mouse click event on the tabline to select a
+ tabpage. The supported items in {args} are:
+ tabnr: tab page number
+
+ "tabmenu":
+ Inject an event to select a tabline menu entry. The
+ supported items in {args} are:
+ tabnr: tab page number
+ item: tab page menu item number. 1 for the first
+ menu item, 2 for the second item and so on.
+
+ After injecting the GUI events you probably should call
+ |feedkeys()| to have them processed, e.g.: >
+ call feedkeys("y", 'Lx!')
+<
+ Returns TRUE if the event is successfully added, FALSE if
+ there is a failure.
+
+ Can also be used as a |method|: >
+ GetEvent()->test_gui_event({args})
+<
+test_ignore_error({expr}) *test_ignore_error()*
+ Ignore any error containing {expr}. A normal message is given
+ instead.
+ This is only meant to be used in tests, where catching the
+ error with try/catch cannot be used (because it skips over
+ following code).
+ {expr} is used literally, not as a pattern.
+ When the {expr} is the string "RESET" then the list of ignored
+ errors is made empty.
+
+ Can also be used as a |method|: >
+ GetErrorText()->test_ignore_error()
+
+
+test_mswin_event({event}, {args}) *test_mswin_event()*
+ Generate a low-level MS-Windows {event} with arguments {args}
+ for testing Vim functionality. It works for MS-Windows GUI
+ and for the console.
+
+ {event} is a String and the supported values are:
+ "mouse" mouse event.
+ "key" keyboard event.
+
+ "mouse":
+ Inject either a mouse button click, or a mouse move, event.
+ The supported items in {args} are:
+ button: mouse button. The supported values are:
+ 0 right mouse button
+ 1 middle mouse button
+ 2 left mouse button
+ 3 mouse button release
+ 4 scroll wheel down
+ 5 scroll wheel up
+ 6 scroll wheel left
+ 7 scroll wheel right
+ row: mouse click row number. The first row of the
+ Vim window is 1 and the last row is 'lines'.
+ col: mouse click column number. The maximum value
+ of {col} is 'columns'.
+ Note: row and col are always interpreted as
+ screen cells for the console application.
+ But, they may be interpreted as pixels
+ for the GUI, depending on "cell".
+ multiclick: set to 1 to inject a double-click mouse event.
+ modifiers: key modifiers. The supported values are:
+ 4 shift is pressed
+ 8 alt is pressed
+ 16 ctrl is pressed
+ move: Optional; if used and TRUE then a mouse move
+ event can be generated.
+ Only {args} row: and col: are used and
+ required.
+ Only results in an event when 'mousemoveevent'
+ is set or a popup uses mouse move events.
+ cell: Optional for the GUI: when present and TRUE
+ then "move" uses screen cells instead of pixel
+ positions. Not used by the console.
+
+ "key":
+ Send a low-level keyboard event (e.g. keyup or keydown).
+ The supported items in {args} are:
+ event: The supported string values are:
+ keyup generate a keyup event
+ keydown generate a keydown event
+ keycode: Keycode to use for a keyup or a keydown event.
+ modifiers: Optional; key modifiers.
+ The supported values are:
+ 2 shift is pressed
+ 4 ctrl is pressed
+ 8 alt is pressed
+ Note: These values are different from the
+ mouse modifiers.
+ execute: Optional. Similar to |feedkeys()| mode x.
+ When this is included and set to true
+ (non-zero) then Vim will process any buffered
+ unprocessed key events. All other {args}
+ items are optional when this is set and true.
+
+ Returns TRUE if the event is successfully added or executed,
+ FALSE if there is a failure.
+
+ Can also be used as a |method|: >
+ GetEvent()->test_mswin_event({args})
+<
+
+test_null_blob() *test_null_blob()*
+ Return a |Blob| that is null. Only useful for testing.
+
+
+test_null_channel() *test_null_channel()*
+ Return a |Channel| that is null. Only useful for testing.
+ {only available when compiled with the +channel feature}
+
+
+test_null_dict() *test_null_dict()*
+ Return a |Dict| that is null. Only useful for testing.
+
+
+test_null_function() *test_null_function()*
+ Return a |Funcref| that is null. Only useful for testing.
+
+
+test_null_job() *test_null_job()*
+ Return a |Job| that is null. Only useful for testing.
+ {only available when compiled with the +job feature}
+
+
+test_null_list() *test_null_list()*
+ Return a |List| that is null. Only useful for testing.
+
+
+test_null_partial() *test_null_partial()*
+ Return a |Partial| that is null. Only useful for testing.
+
+
+test_null_string() *test_null_string()*
+ Return a |String| that is null. Only useful for testing.
+
+
+test_option_not_set({name}) *test_option_not_set()*
+ Reset the flag that indicates option {name} was set. Thus it
+ looks like it still has the default value. Use like this: >
+ set ambiwidth=double
+ call test_option_not_set('ambiwidth')
+< Now the 'ambiwidth' option behaves like it was never changed,
+ even though the value is "double".
+ Only to be used for testing!
+
+ Can also be used as a |method|: >
+ GetOptionName()->test_option_not_set()
+
+
+test_override({name}, {val}) *test_override()*
+ Overrides certain parts of Vim's internal processing to be able
+ to run tests. Only to be used for testing Vim!
+ The override is enabled when {val} is non-zero and removed
+ when {val} is zero.
+ Current supported values for {name} are:
+
+ {name} effect when {val} is non-zero ~
+ alloc_lines make a copy of every buffer line into allocated
+ memory, so that memory access errors can be found
+ by valgrind
+ autoload `import autoload` will load the script right
+ away, not postponed until an item is used
+ char_avail disable the char_avail() function
+ nfa_fail makes the NFA regexp engine fail to force a
+ fallback to the old engine
+ no_query_mouse do not query the mouse position for "dec"
+ terminals
+ no_wait_return set the "no_wait_return" flag. Not restored
+ with "ALL".
+ redraw disable the redrawing() function
+ redraw_flag ignore the RedrawingDisabled flag
+ starting reset the "starting" variable, see below
+ term_props reset all terminal properties when the version
+ string is detected
+ ui_delay time in msec to use in ui_delay(); overrules a
+ wait time of up to 3 seconds for messages
+ unreachable no error for code after `:throw` and `:return`
+ uptime overrules sysinfo.uptime
+ vterm_title setting the window title by a job running in a
+ terminal window
+ ALL clear all overrides, except alloc_lines ({val} is
+ not used)
+
+ "starting" is to be used when a test should behave like
+ startup was done. Since the tests are run by sourcing a
+ script the "starting" variable is non-zero. This is usually a
+ good thing (tests run faster), but sometimes this changes
+ behavior in a way that the test doesn't work properly.
+ When using: >
+ call test_override('starting', 1)
+< The value of "starting" is saved. It is restored by: >
+ call test_override('starting', 0)
+
+< To make sure the flag is reset later using `:defer` can be
+ useful: >
+ call test_override('unreachable', 1)
+ defer call test_override('unreachable', 0)
+
+< Can also be used as a |method|: >
+ GetOverrideVal()-> test_override('starting')
+
+
+test_refcount({expr}) *test_refcount()*
+ Return the reference count of {expr}. When {expr} is of a
+ type that does not have a reference count, returns -1. Only
+ to be used for testing.
+
+ Can also be used as a |method|: >
+ GetVarname()->test_refcount()
+
+
+test_setmouse({row}, {col}) *test_setmouse()*
+ Set the mouse position to be used for the next mouse action.
+ {row} and {col} are one based.
+ For example: >
+ call test_setmouse(4, 20)
+ call feedkeys("\<LeftMouse>", "xt")
+
+
+test_settime({expr}) *test_settime()*
+ Set the time Vim uses internally. Currently only used for
+ timestamps in the history, as they are used in viminfo, and
+ for undo.
+ Using a value of 1 makes Vim not sleep after a warning or
+ error message.
+ {expr} must evaluate to a number. When the value is zero the
+ normal behavior is restored.
+
+ Can also be used as a |method|: >
+ GetTime()->test_settime()
+
+
+test_srand_seed([seed]) *test_srand_seed()*
+ When [seed] is given this sets the seed value used by
+ `srand()`. When omitted the test seed is removed.
+
+
+test_unknown() *test_unknown()*
+ Return a value with unknown type. Only useful for testing.
+
+
+test_void() *test_void()*
+ Return a value with void type. Only useful for testing.
+
+==============================================================================
+3. Assert functions *assert-functions-details*
+
+
+assert_beeps({cmd}) *assert_beeps()*
+ Run {cmd} and add an error message to |v:errors| if it does
+ NOT produce a beep or visual bell.
+ Also see |assert_fails()|, |assert_nobeep()| and
+ |assert-return|.
+
+ Can also be used as a |method|: >
+ GetCmd()->assert_beeps()
+<
+ *assert_equal()*
+assert_equal({expected}, {actual} [, {msg}])
+ When {expected} and {actual} are not equal an error message is
+ added to |v:errors| and 1 is returned. Otherwise zero is
+ returned. |assert-return|
+ The error is in the form "Expected {expected} but got
+ {actual}". When {msg} is present it is prefixed to that.
+
+ There is no automatic conversion, the String "4" is different
+ from the Number 4. And the number 4 is different from the
+ Float 4.0. The value of 'ignorecase' is not used here, case
+ always matters.
+ Example: >
+ assert_equal('foo', 'bar')
+< Will result in a string to be added to |v:errors|:
+ test.vim line 12: Expected 'foo' but got 'bar' ~
+
+ Can also be used as a |method|, the base is passed as the
+ second argument: >
+ mylist->assert_equal([1, 2, 3])
+
+< *assert_equalfile()*
+assert_equalfile({fname-one}, {fname-two} [, {msg}])
+ When the files {fname-one} and {fname-two} do not contain
+ exactly the same text an error message is added to |v:errors|.
+ Also see |assert-return|.
+ When {fname-one} or {fname-two} does not exist the error will
+ mention that.
+ Mainly useful with |terminal-diff|.
+
+ Can also be used as a |method|: >
+ GetLog()->assert_equalfile('expected.log')
+
+assert_exception({error} [, {msg}]) *assert_exception()*
+ When v:exception does not contain the string {error} an error
+ message is added to |v:errors|. Also see |assert-return|.
+ This can be used to assert that a command throws an exception.
+ Using the error number, followed by a colon, avoids problems
+ with translations: >
+ try
+ commandthatfails
+ call assert_false(1, 'command should have failed')
+ catch
+ call assert_exception('E492:')
+ endtry
+<
+ *assert_fails()*
+assert_fails({cmd} [, {error} [, {msg} [, {lnum} [, {context}]]]])
+ Run {cmd} and add an error message to |v:errors| if it does
+ NOT produce an error or when {error} is not found in the
+ error message. Also see |assert-return|.
+ *E856*
+ When {error} is a string it must be found literally in the
+ first reported error. Most often this will be the error code,
+ including the colon, e.g. "E123:". >
+ assert_fails('bad cmd', 'E987:')
+<
+ When {error} is a |List| with one or two strings, these are
+ used as patterns. The first pattern is matched against the
+ first reported error: >
+ assert_fails('cmd', ['E987:.*expected bool'])
+< The second pattern, if present, is matched against the last
+ reported error.
+ If there is only one error then both patterns must match. This
+ can be used to check that there is only one error.
+ To only match the last error use an empty string for the first
+ error: >
+ assert_fails('cmd', ['', 'E987:'])
+<
+ If {msg} is empty then it is not used. Do this to get the
+ default message when passing the {lnum} argument.
+ *E1115*
+ When {lnum} is present and not negative, and the {error}
+ argument is present and matches, then this is compared with
+ the line number at which the error was reported. That can be
+ the line number in a function or in a script.
+ *E1116*
+ When {context} is present it is used as a pattern and matched
+ against the context (script name or function name) where
+ {lnum} is located in.
+
+ Note that beeping is not considered an error, and some failing
+ commands only beep. Use |assert_beeps()| for those.
+
+ Can also be used as a |method|: >
+ GetCmd()->assert_fails('E99:')
+
+assert_false({actual} [, {msg}]) *assert_false()*
+ When {actual} is not false an error message is added to
+ |v:errors|, like with |assert_equal()|.
+ The error is in the form "Expected False but got {actual}".
+ When {msg} is present it is prepended to that.
+ Also see |assert-return|.
+
+ A value is false when it is zero. When {actual} is not a
+ number the assert fails.
+
+ Can also be used as a |method|: >
+ GetResult()->assert_false()
+
+assert_inrange({lower}, {upper}, {actual} [, {msg}]) *assert_inrange()*
+ This asserts number and |Float| values. When {actual} is lower
+ than {lower} or higher than {upper} an error message is added
+ to |v:errors|. Also see |assert-return|.
+ The error is in the form "Expected range {lower} - {upper},
+ but got {actual}". When {msg} is present it is prefixed to
+ that.
+
+ *assert_match()*
+assert_match({pattern}, {actual} [, {msg}])
+ When {pattern} does not match {actual} an error message is
+ added to |v:errors|. Also see |assert-return|.
+ The error is in the form "Pattern {pattern} does not match
+ {actual}". When {msg} is present it is prefixed to that.
+
+ {pattern} is used as with |=~|: The matching is always done
+ like 'magic' was set and 'cpoptions' is empty, no matter what
+ the actual value of 'magic' or 'cpoptions' is.
+
+ {actual} is used as a string, automatic conversion applies.
+ Use "^" and "$" to match with the start and end of the text.
+ Use both to match the whole text.
+
+ Example: >
+ assert_match('^f.*o$', 'foobar')
+< Will result in a string to be added to |v:errors|:
+ test.vim line 12: Pattern '^f.*o$' does not match 'foobar' ~
+
+ Can also be used as a |method|: >
+ getFile()->assert_match('foo.*')
+<
+assert_nobeep({cmd}) *assert_nobeep()*
+ Run {cmd} and add an error message to |v:errors| if it
+ produces a beep or visual bell.
+ Also see |assert_beeps()|.
+
+ Can also be used as a |method|: >
+ GetCmd()->assert_nobeep()
+<
+ *assert_notequal()*
+assert_notequal({expected}, {actual} [, {msg}])
+ The opposite of `assert_equal()`: add an error message to
+ |v:errors| when {expected} and {actual} are equal.
+ Also see |assert-return|.
+
+ Can also be used as a |method|: >
+ mylist->assert_notequal([1, 2, 3])
+
+< *assert_notmatch()*
+assert_notmatch({pattern}, {actual} [, {msg}])
+ The opposite of `assert_match()`: add an error message to
+ |v:errors| when {pattern} matches {actual}.
+ Also see |assert-return|.
+
+ Can also be used as a |method|: >
+ getFile()->assert_notmatch('bar.*')
+
+
+assert_report({msg}) *assert_report()*
+ Report a test failure directly, using String {msg}.
+ Always returns one.
+
+ Can also be used as a |method|: >
+ GetMessage()->assert_report()
+
+
+assert_true({actual} [, {msg}]) *assert_true()*
+ When {actual} is not true an error message is added to
+ |v:errors|, like with |assert_equal()|.
+ Also see |assert-return|.
+ A value is TRUE when it is a non-zero number. When {actual}
+ is not a number the assert fails.
+ When {msg} is given it precedes the default message.
+
+ Can also be used as a |method|: >
+ GetResult()->assert_true()
+<
+
+ vim:tw=78:ts=8:noet:ft=help:norl:
generated by cgit v1.2.3 (git 2.39.1) at 2024-11-23 16:00:58 +0000