patch 8.2.0149: maintaining a Vim9 branch separately is more workv8.2.0149

Problem:    Maintaining a Vim9 branch separately is more work.
Solution:   Merge the Vim9 script changes.

65 files changed, 11791 insertions, 1341 deletions

diff --git a/README.md b/README.md
index c87a26558b..4e3d8433c3 100644
--- a/README.md
+++ b/README.md
@@ -31,6 +31,8 @@ flavours of UNIX.  Porting to other systems should not be very difficult.
 Older versions of Vim run on MS-DOS, MS-Windows 95/98/Me/NT/2000, Amiga DOS,
 Atari MiNT, BeOS, RISC OS and OS/2.  These are no longer maintained.
 
+For Vim9 script see [README_VIM9](README_VIM9.md).
+
 ## Distribution ##
 
 You can often use your favorite package manager to install Vim.  On Mac and
diff --git a/README_VIM9.md b/README_VIM9.md
new file mode 100644
index 0000000000..b77d013c05
--- /dev/null
+++ b/README_VIM9.md
@@ -0,0 +1,344 @@
+![Vim Logo](https://github.com/vim/vim/blob/master/runtime/vimlogo.gif)
+
+# What is Vim9?
+
+This is an experimental side of [Vim](https://github.com/vim/vim).
+It explores ways of making Vim script faster and better.
+
+WARNING: The Vim9 script features are in the early stages of development,
+anything can break!
+
+# Why Vim9?
+
+## 1. FASTER VIM SCRIPT
+
+The third item on the poll results of 2018, after popup windows and text
+properties, is faster Vim script.  So how do we do that?
+
+I have been throwing some ideas around, and soon came to the conclusion
+that the current way functions are called and executed, with
+dictionaries for the arguments and local variables, is never going to be
+very fast.  We're lucky if we can make it twice as fast.  The overhead
+of a function call and executing every line is just too high.
+
+So what then?  We can only make something fast by having a new way of
+defining a function, with similar but different properties of the old
+way:
+* Arguments are only available by name, not through the a: dictionary or
+  the a:000 list.
+* Local variables are not available in an l: dictionary.
+* A few more things that slow us down, such as exception handling details.
+
+I Implemented a "proof of concept" and measured the time to run a simple
+for loop with an addition (Justin used this example in his presentation,
+full code is below):
+
+``` vim
+  let sum = 0
+  for i in range(1, 2999999)
+    let sum += i
+  endfor
+```
+
+| how     | time in sec |
+| --------| -------- |
+| Vim old | 5.018541 |
+| Python  | 0.369598 |
+| Lua     | 0.078817 |
+| Vim new | 0.073595 |
+
+That looks very promising!  It's just one example, but it shows how much
+we can gain, and also that Vim script can be faster than builtin
+interfaces.
+
+In practice the script would not do something useless as counting but change
+the text.  For example, re-indent all the lines:
+
+``` vim
+  let totallen = 0
+  for i in range(1, 100000)
+    call setline(i, '    ' .. getline(i))
+    let totallen += len(getline(i))
+  endfor
+```
+
+| how     | time in sec |
+| --------| -------- |
+| Vim old | 0.853752 |
+| Python  | 0.304584 |
+| Lua     | 0.286573 |
+| Vim new | 0.190276 |
+
+The differences are smaller, but Vim 9 script is clearly the fastest.
+
+How does Vim9 script work?  The function is first compiled into a sequence of
+instructions.  Each instruction has one or two parameters and a stack is
+used to store intermediate results.  Local variables are also on the
+stack, space is reserved during compilation.  This is a fairly normal
+way of compilation into an intermediate format, specialized for Vim,
+e.g. each stack item is a typeval_T.  And one of the instructions is
+"execute Ex command", for commands that are not compiled.
+
+
+## 2. PHASING OUT INTERFACES
+
+Attempts have been made to implement functionality with built-in script
+languages such as Python, Perl, Lua, Tcl and Ruby.  This never gained much
+foothold, for various reasons.
+
+Instead of using script language support in Vim:
+* Encourage implementing external tools in any language and communicate
+  with them.  The job and channel support already makes this possible.
+  Really any language can be used, also Java and Go, which are not
+  available built-in.
+* Phase out the built-in language interfaces, make maintenance a bit easier
+  and executables easier to build.  They will be kept for backwards
+  compatibility, no new features.
+* Improve the Vim script language, it is used to communicate with the external
+  tool and implements the Vim side of the interface.  Also, it can be used when
+  an external tool is undesired.
+
+All together this creates a clear situation: Vim with the +eval feature
+will be sufficient for most plugins, while some plugins require
+installing a tool that can be written in any language.  No confusion
+about having Vim but the plugin not working because some specific
+language is missing.  This is a good long term goal.
+
+Rationale: Why is it better to run a tool separately from Vim than using a
+built-in interface and interpreter?  Take for example something that is
+written in Python:
+* The built-in interface uses the embedded python interpreter.  This is less
+  well maintained than the python command.  Building Vim with it requires
+  installing developer packages.  If loaded dynamically there can be a version
+  mismatch.
+* When running the tool externally the standard python command can be used,
+  which is quite often available by default or can be easily installed.
+* The built-in interface has an API that is unique for Vim with Python. This is
+  an extra API to learn.
+* A .py file can be compiled into a .pyc file and execute much faster.
+* Inside Vim multi-threading can cause problems, since the Vim core is single
+  threaded.  In an external tool there are no such problems.
+* The Vim part is written in .vim files, the Python part is in .py files, this
+  is nicely separated.
+* Disadvantage: An interface needs to be made between Vim and Python.
+  JSON is available for this, and it's fairly easy to use.  But it still
+  requires implementing asynchronous communication.
+
+
+## 3. BETTER VIM SCRIPT
+
+To make Vim faster a new way of defining a function needs to be added.
+While we are doing that, since the lines in this function won't be fully
+backwards compatible anyway, we can also make Vim script easier to use.
+In other words: "less weird".  Making it work more like modern
+programming languages will help.  No surprises.
+
+A good example is how in a function the arguments are prefixed with
+"a:". No other language I know does that, so let's drop it.
+
+Taking this one step further is also dropping "s:" for script-local variables;
+everything at the script level is script-local by default.  Since this is not
+backwards compatible it requires a new script style: Vim9 script!
+
+It should be possible to convert code from other languages to Vim
+script.  We can add functionality to make this easier.  This still needs
+to be discussed, but we can consider adding type checking and a simple
+form of classes.  If you look at JavaScript for example, it has gone
+through these stages over time, adding real class support and now
+TypeScript adds type checking.  But we'll have to see how much of that
+we actually want to include in Vim script.  Ideally a conversion tool
+can take Python, JavaScript or TypeScript code and convert it to Vim
+script, with only some things that cannot be converted.
+
+Vim script won't work the same as any specific language, but we can use
+mechanisms that are commonly known, ideally with the same syntax.  One
+thing I have been thinking of is assignments without ":let".  I often
+make that mistake (after writing JavaScript especially).  I think it is
+possible, if we make local variables shadow commands.  That should be OK,
+if you shadow a command you want to use, just rename the variable.
+Using "let" and "const" to declare a variable, like in JavaScript and
+TypeScript, can work:
+
+
+``` vim
+def MyFunction(arg: number): number
+   let local = 1
+   let todo = arg
+   const ADD = 88
+   while todo > 0
+      local += ADD
+      --todo
+   endwhile
+   return local
+enddef
+```
+
+The similarity with JavaScript/TypeScript can also be used for dependencies
+between files.  Vim currently uses the `:source` command, which has several
+disadvantages:
+*   In the sourced script, is not clear what it provides.  By default all
+    functions are global and can be used elsewhere.
+*   In a script that sources other scripts, it is not clear what function comes
+    from what sourced script.  Finding the implementation is a hassle.
+*   Prevention of loading the whole script twice must be manually implemented.
+
+We can use the `:import` and `:export` commands from the JavaScript standard to
+make this much better.  For example, in script "myfunction.vim" define a
+function and export it:
+
+``` vim
+vim9script  " Vim9 script syntax used here
+
+let local = 'local variable is not exported, script-local'
+
+export def MyFunction()  " exported function
+...
+
+def LocalFunction() " not exported, script-local
+...
+```
+
+And in another script import the function:
+
+``` vim
+vim9script  " Vim9 script syntax used here
+
+import MyFunction from 'myfunction.vim'
+```
+
+This looks like JavaScript/TypeScript, thus many users will understand the
+syntax.
+
+These are ideas, this will take time to design, discuss and implement.
+Eventually this will lead to Vim 9!
+
+
+## Code for sum time measurements
+
+Vim was build with -O2.
+
+``` vim
+func VimOld()
+  let sum = 0
+  for i in range(1, 2999999)
+    let sum += i
+  endfor
+  return sum
+endfunc
+
+func Python()
+  py3 << END
+sum = 0
+for i in range(1, 3000000):
+  sum += i
+END
+  return py3eval('sum')
+endfunc
+
+func Lua()
+  lua << END
+    sum = 0
+    for i = 1, 2999999 do
+      sum = sum + i
+    end
+END
+  return luaeval('sum')
+endfunc
+
+def VimNew()
+  let sum = 0
+  for i in range(1, 2999999)
+    let sum += i
+  endfor
+  return sum
+enddef
+
+let start = reltime()
+echo VimOld()
+echo 'Vim old: ' .. reltimestr(reltime(start))
+
+let start = reltime()
+echo Python()
+echo 'Python: ' .. reltimestr(reltime(start))
+
+let start = reltime()
+echo Lua()
+echo 'Lua: ' .. reltimestr(reltime(start))
+
+let start = reltime()
+echo VimNew()
+echo 'Vim new: ' .. reltimestr(reltime(start))
+```
+
+## Code for indent time measurements
+
+``` vim
+def VimNew(): number
+  let totallen = 0
+  for i in range(1, 100000)
+    setline(i, '    ' .. getline(i))
+    totallen += len(getline(i))
+  endfor
+  return totallen
+enddef
+
+func VimOld()
+  let totallen = 0
+  for i in range(1, 100000)
+    call setline(i, '    ' .. getline(i))
+    let totallen += len(getline(i))
+  endfor
+  return totallen
+endfunc
+
+func Lua()
+  lua << END
+    b = vim.buffer()
+    totallen = 0
+    for i = 1, 100000 do
+      b[i] = "    " .. b[i]
+      totallen = totallen + string.len(b[i])
+    end
+END
+  return luaeval('totallen')
+endfunc
+
+func Python()
+  py3 << END
+cb = vim.current.buffer
+totallen = 0
+for i in range(0, 100000):
+  cb[i] = '    ' + cb[i]
+  totallen += len(cb[i])
+END
+  return py3eval('totallen')
+endfunc
+
+new
+call setline(1, range(100000))
+let start = reltime()
+echo VimOld()
+echo 'Vim old: ' .. reltimestr(reltime(start))
+bwipe!
+
+new
+call setline(1, range(100000))
+let start = reltime()
+echo Python()
+echo 'Python: ' .. reltimestr(reltime(start))
+bwipe!
+ 
+new
+call setline(1, range(100000))
+let start = reltime()
+echo Lua()
+echo 'Lua: ' .. reltimestr(reltime(start))
+bwipe!
+
+new
+call setline(1, range(100000))
+let start = reltime()
+echo VimNew()
+echo 'Vim new: ' .. reltimestr(reltime(start))
+bwipe!
+```
diff --git a/runtime/doc/Makefile b/runtime/doc/Makefile
index 514a01237f..65dc8436c6 100644
--- a/runtime/doc/Makefile
+++ b/runtime/doc/Makefile
@@ -149,6 +149,7 @@ DOCS = \
 	version7.txt \
 	version8.txt \
 	vi_diff.txt \
+	vim9.txt \
 	visual.txt \
 	windows.txt \
 	workshop.txt
@@ -289,6 +290,7 @@ HTMLS = \
 	version8.html \
 	vi_diff.html \
 	vimindex.html \
+	vim9.html \
 	visual.html \
 	windows.html \
 	workshop.html
diff --git a/runtime/doc/eval.txt b/runtime/doc/eval.txt
index b580f2fabf..071f250807 100644
--- a/runtime/doc/eval.txt
+++ b/runtime/doc/eval.txt
@@ -12,6 +12,10 @@ Note: Expression evaluation can be disabled at compile time.  If this has been
 done, the features in this document are not available.  See |+eval| and
 |no-eval-feature|.
 
+This file is about the backwards compatible Vim script.  For Vim9 script,
+which executes much faster, supports type checking and much more, see
+|vim9.txt|.
+
 1.  Variables			|variables|
     1.1 Variable types
     1.2 Function references		|Funcref|
@@ -2512,8 +2516,8 @@ haslocaldir([{winnr} [, {tabnr}]])
 					or |:tcd|
 hasmapto({what} [, {mode} [, {abbr}]])
 				Number	|TRUE| if mapping to {what} exists
-histadd({history}, {item})	String	add an item to a history
-histdel({history} [, {item}])	String	remove an item from a history
+histadd({history}, {item})	Number	add an item to a history
+histdel({history} [, {item}])	Number	remove an item from a history
 histget({history} [, {index}])	String	get the item {index} from a history
 histnr({history})		Number	highest index of a history
 hlexists({name})		Number	|TRUE| if highlight group {name} exists
@@ -10894,6 +10898,9 @@ New functions can be defined.  These can be called just like builtin
 functions.  The function executes a sequence of Ex commands.  Normal mode
 commands can be executed with the |:normal| command.
 
+This section is about the legacy functions. For the Vim9 functions, which
+execute much faster, support type checking and more, see |vim9.txt|.
+
 The function name must start with an uppercase letter, to avoid confusion with
 builtin functions.  To prevent from using the same name in different scripts
 avoid obvious, short names.  A good habit is to start the function name with
diff --git a/runtime/doc/options.txt b/runtime/doc/options.txt
index 876ddf3588..847290e29f 100644
--- a/runtime/doc/options.txt
+++ b/runtime/doc/options.txt
@@ -6221,6 +6221,7 @@ A jump table for the options with a short description can be found at |Q_op|.
 	  compiler/	compiler files |:compiler|
 	  doc/		documentation |write-local-help|
 	  ftplugin/	filetype plugins |write-filetype-plugin|
+	  import/	files that are found by `:import`
 	  indent/	indent scripts |indent-expression|
 	  keymap/	key mapping files |mbyte-keymap|
 	  lang/		menu translations |:menutrans|
diff --git a/runtime/doc/vim9.txt b/runtime/doc/vim9.txt
new file mode 100644
index 0000000000..c6c674da83
--- /dev/null
+++ b/runtime/doc/vim9.txt
@@ -0,0 +1,561 @@
+*vim9.txt*	For Vim version 8.2.  Last change: 2019 Dec 06
+
+
+		  VIM REFERENCE MANUAL	  by Bram Moolenaar
+
+
+THIS IS STILL UNDER DEVELOPMENT - ANYTHING CAN BREAK - ANYTHING CAN CHANGE
+
+Vim9 script commands and expressions.
+
+Most expression help is in |eval.txt|.  This file is about the new syntax and
+features in Vim9 script.
+
+THIS IS STILL UNDER DEVELOPMENT - ANYTHING CAN BREAK - ANYTHING CAN CHANGE
+
+
+1   What is Vim9 script?		|vim9-script|
+2.  Differences				|vim9-differences|
+3.  New style functions			|fast-functions|
+4.  Types				|vim9-types|
+5.  Namespace, Import and Export	|vim9script|
+
+9.  Rationale				|vim9-rationale|
+
+==============================================================================
+
+1. What is Vim9 script?					*vim9-script*
+
+THIS IS STILL UNDER DEVELOPMENT - ANYTHING CAN BREAK - ANYTHING CAN CHANGE
+
+Vim script has been growing over time, while keeping backwards compatibility.
+That means bad choices from the past often can't be changed.  Execution is
+quite slow, every line is parsed every time it is executed.
+
+The main goal of Vim9 script is to drastically improve performance.  An
+increase in execution speed of 10 to 100 times can be expected.  A secondary
+goal is to avoid Vim-specific constructs and get closer to commonly used
+programming languages, such as JavaScript, TypeScript and Java.
+
+The performance improvements can only be achieved by not being 100% backwards
+compatible.  For example, in a function the arguments are not available in the
+"a:" dictionary, as creating that dictionary adds quite a lot of overhead.
+Other differences are more subtle, such as how errors are handled.
+
+The Vim9 script syntax and semantics are used in:
+- a function defined with the `:def` command
+- a script file where the first command is `vim9script`
+
+When using `:function` in a Vim9 script file the legacy syntax is used.
+However, this is discouraged.
+
+Vim9 script and legacy Vim script can be mixed.  There is no need to rewrite
+old scripts, they keep working as before.
+
+==============================================================================
+
+2. Differences from legacy Vim script			*vim9-differences*
+
+THIS IS STILL UNDER DEVELOPMENT - ANYTHING CAN BREAK - ANYTHING CAN CHANGE
+
+Vim9 functions ~
+
+`:def` has no extra arguments like `:function` does: "range", "abort", "dict"
+or "closure".  A `:def` function always aborts on an error, does not get a
+range passed and cannot be a "dict" function.
+
+In the function body:
+- Arguments are accessed by name, without "a:".
+- There is no "a:" dictionary or "a:000" list.  Variable arguments are defined
+  with a name and have a list type: >
+  	def MyFunc(...itemlist: list<type>)
+	   for item in itemlist
+	     ...
+
+
+Variable declarations with :let and :const ~
+
+Local variables need to be declared with `:let`.  Local constants need to be
+declared with `:const`.  We refer to both as "variables".
+
+Variables can be local to a script, function or code block: >
+	vim9script
+	let script_var = 123
+	def SomeFunc()
+	  let func_var = script_var
+	  if cond
+	    let block_var = func_var
+	  ...
+
+The variables are only visible in the block where they are defined and nested
+blocks.  Once the block ends the variable is no longer accessible: >
+	if cond
+	   let inner = 5
+	else
+	   let inner = 0
+	endif
+	echo inner  " Error!
+
+The declaration must be done earlier: >
+	let inner: number
+	if cond
+	   inner = 5
+	else
+	   inner = 0
+	endif
+	echo inner
+
+To intentionally use a variable that won't be available later, a block can be
+used: >
+	{
+	   let temp = 'temp'
+	   ...
+	}
+	echo temp  " Error!
+
+An existing variable cannot be assigend to with `:let`, since that implies a
+declaration.  An exception is global variables: these can be both used with
+and without `:let`, because there is no rule about where they are declared.
+
+Variables cannot shadow previously defined variables.
+Variables may shadow Ex commands, rename the variable if needed.
+
+Since "&opt = value" is now assigning a value to option "opt", ":&" cannot be
+used to repeat a `:substitute` command.
+
+
+Omitting :call and :eval ~
+
+Functions can be called without `:call`: >
+  	writefile(lines, 'file')
+Using `:call` is still posible, but this is discouraged.
+
+A method call without `eval` is possible, so long as the start is an
+identifier or can't be an Ex command.  It does not work for string constants: >
+	myList->add(123)		" works
+	g:myList->add(123)		" works
+	[1, 2, 3]->Process()		" works
+	#{a: 1, b: 2}->Process()	" works
+	{'a': 1, 'b': 2}->Process()	" works
+	"foobar"->Process()		" does NOT work
+	eval "foobar"->Process()	" works
+
+
+No curly braces expansion ~
+
+|curly-braces-names| cannot be used.
+
+
+Comperators ~
+
+The 'ignorecase' option is not used for comperators that use strings.
+
+
+White space ~
+
+Vim9 script enforces proper use of white space.  This is no longer allowed: >
+	let var=234	" Error!
+	let var= 234	" Error!
+	let var =234	" Error!
+There must be white space before and after the "=": >
+	let var = 234	" OK
+
+White space is required around most operators.
+
+White space is not allowed:
+- Between a function name and the "(": >
+  	call Func (arg)	   " Error!
+  	call Func
+	     \ (arg)	   " Error!
+  	call Func(arg)	   " OK
+  	call Func(
+	     \ arg)	   " OK
+
+
+Conditions and expressions ~
+
+Conditions and expression are mostly working like they do in JavaScript.  A
+difference is made where JavaScript does not work like most people expect.
+Specifically, an empty list is falsey.
+
+Any type of variable can be used as a condition, there is no error, not even
+for using a list or job.  This is very much like JavaScript, but there are a
+few exceptions.
+
+	type		TRUE when ~
+	bool		v:true
+	number		non-zero
+	float		non-zero
+	string		non-empty
+	blob		non-empty
+	list		non-empty (different from JavaScript)
+	dictionary	non-empty (different from JavaScript)
+	funcref		when not NULL
+	partial		when not NULL
+	special		v:true
+	job		when not NULL
+	channel		when not NULL
+	class		when not NULL
+	object		when not NULL (TODO: when isTrue() returns v:true)
+
+The boolean operators "||" and "&&" do not change the value: >
+	8 || 2   == 8
+	0 || 2   == 2
+	0 || ''  == ''
+	8 && 2   == 2
+	0 && 2   == 0
+	[] && 2  == []
+
+When using `..` for string concatenation the arguments are always converted to
+string. >
+	'hello ' .. 123  == 'hello 123'
+	'hello ' .. v:true  == 'hello true'
+
+In Vim9 script one can use "true" for v:true and "false" for v:false.
+
+
+==============================================================================
+
+3. New style functions					*fast-functions*
+
+THIS IS STILL UNDER DEVELOPMENT - ANYTHING CAN BREAK - ANYTHING CAN CHANGE
+
+							*:def*
+:def[!] {name}([arguments])[: {return-type}
+			Define a new function by the name {name}.  The body of
+			the function follows in the next lines, until the
+			matching `:enddef`.
+
+			When {return-type} is omitted the return type will be
+			decided upon by the first encountered `return`
+			statement in the function.  E.g., for: >
+				return 'messag