diff options
65 files changed, 11791 insertions, 1341 deletions
@@ -31,6 +31,8 @@ flavours of UNIX. Porting to other systems should not be very difficult. Older versions of Vim run on MS-DOS, MS-Windows 95/98/Me/NT/2000, Amiga DOS, Atari MiNT, BeOS, RISC OS and OS/2. These are no longer maintained. +For Vim9 script see [README_VIM9](README_VIM9.md). + ## Distribution ## You can often use your favorite package manager to install Vim. On Mac and diff --git a/README_VIM9.md b/README_VIM9.md new file mode 100644 index 0000000000..b77d013c05 --- /dev/null +++ b/README_VIM9.md @@ -0,0 +1,344 @@ +![Vim Logo](https://github.com/vim/vim/blob/master/runtime/vimlogo.gif) + +# What is Vim9? + +This is an experimental side of [Vim](https://github.com/vim/vim). +It explores ways of making Vim script faster and better. + +WARNING: The Vim9 script features are in the early stages of development, +anything can break! + +# Why Vim9? + +## 1. FASTER VIM SCRIPT + +The third item on the poll results of 2018, after popup windows and text +properties, is faster Vim script. So how do we do that? + +I have been throwing some ideas around, and soon came to the conclusion +that the current way functions are called and executed, with +dictionaries for the arguments and local variables, is never going to be +very fast. We're lucky if we can make it twice as fast. The overhead +of a function call and executing every line is just too high. + +So what then? We can only make something fast by having a new way of +defining a function, with similar but different properties of the old +way: +* Arguments are only available by name, not through the a: dictionary or + the a:000 list. +* Local variables are not available in an l: dictionary. +* A few more things that slow us down, such as exception handling details. + +I Implemented a "proof of concept" and measured the time to run a simple +for loop with an addition (Justin used this example in his presentation, +full code is below): + +``` vim + let sum = 0 + for i in range(1, 2999999) + let sum += i + endfor +``` + +| how | time in sec | +| --------| -------- | +| Vim old | 5.018541 | +| Python | 0.369598 | +| Lua | 0.078817 | +| Vim new | 0.073595 | + +That looks very promising! It's just one example, but it shows how much +we can gain, and also that Vim script can be faster than builtin +interfaces. + +In practice the script would not do something useless as counting but change +the text. For example, re-indent all the lines: + +``` vim + let totallen = 0 + for i in range(1, 100000) + call setline(i, ' ' .. getline(i)) + let totallen += len(getline(i)) + endfor +``` + +| how | time in sec | +| --------| -------- | +| Vim old | 0.853752 | +| Python | 0.304584 | +| Lua | 0.286573 | +| Vim new | 0.190276 | + +The differences are smaller, but Vim 9 script is clearly the fastest. + +How does Vim9 script work? The function is first compiled into a sequence of +instructions. Each instruction has one or two parameters and a stack is +used to store intermediate results. Local variables are also on the +stack, space is reserved during compilation. This is a fairly normal +way of compilation into an intermediate format, specialized for Vim, +e.g. each stack item is a typeval_T. And one of the instructions is +"execute Ex command", for commands that are not compiled. + + +## 2. PHASING OUT INTERFACES + +Attempts have been made to implement functionality with built-in script +languages such as Python, Perl, Lua, Tcl and Ruby. This never gained much +foothold, for various reasons. + +Instead of using script language support in Vim: +* Encourage implementing external tools in any language and communicate + with them. The job and channel support already makes this possible. + Really any language can be used, also Java and Go, which are not + available built-in. +* Phase out the built-in language interfaces, make maintenance a bit easier + and executables easier to build. They will be kept for backwards + compatibility, no new features. +* Improve the Vim script language, it is used to communicate with the external + tool and implements the Vim side of the interface. Also, it can be used when + an external tool is undesired. + +All together this creates a clear situation: Vim with the +eval feature +will be sufficient for most plugins, while some plugins require +installing a tool that can be written in any language. No confusion +about having Vim but the plugin not working because some specific +language is missing. This is a good long term goal. + +Rationale: Why is it better to run a tool separately from Vim than using a +built-in interface and interpreter? Take for example something that is +written in Python: +* The built-in interface uses the embedded python interpreter. This is less + well maintained than the python command. Building Vim with it requires + installing developer packages. If loaded dynamically there can be a version + mismatch. +* When running the tool externally the standard python command can be used, + which is quite often available by default or can be easily installed. +* The built-in interface has an API that is unique for Vim with Python. This is + an extra API to learn. +* A .py file can be compiled into a .pyc file and execute much faster. +* Inside Vim multi-threading can cause problems, since the Vim core is single + threaded. In an external tool there are no such problems. +* The Vim part is written in .vim files, the Python part is in .py files, this + is nicely separated. +* Disadvantage: An interface needs to be made between Vim and Python. + JSON is available for this, and it's fairly easy to use. But it still + requires implementing asynchronous communication. + + +## 3. BETTER VIM SCRIPT + +To make Vim faster a new way of defining a function needs to be added. +While we are doing that, since the lines in this function won't be fully +backwards compatible anyway, we can also make Vim script easier to use. +In other words: "less weird". Making it work more like modern +programming languages will help. No surprises. + +A good example is how in a function the arguments are prefixed with +"a:". No other language I know does that, so let's drop it. + +Taking this one step further is also dropping "s:" for script-local variables; +everything at the script level is script-local by default. Since this is not +backwards compatible it requires a new script style: Vim9 script! + +It should be possible to convert code from other languages to Vim +script. We can add functionality to make this easier. This still needs +to be discussed, but we can consider adding type checking and a simple +form of classes. If you look at JavaScript for example, it has gone +through these stages over time, adding real class support and now +TypeScript adds type checking. But we'll have to see how much of that +we actually want to include in Vim script. Ideally a conversion tool +can take Python, JavaScript or TypeScript code and convert it to Vim +script, with only some things that cannot be converted. + +Vim script won't work the same as any specific language, but we can use +mechanisms that are commonly known, ideally with the same syntax. One +thing I have been thinking of is assignments without ":let". I often +make that mistake (after writing JavaScript especially). I think it is +possible, if we make local variables shadow commands. That should be OK, +if you shadow a command you want to use, just rename the variable. +Using "let" and "const" to declare a variable, like in JavaScript and +TypeScript, can work: + + +``` vim +def MyFunction(arg: number): number + let local = 1 + let todo = arg + const ADD = 88 + while todo > 0 + local += ADD + --todo + endwhile + return local +enddef +``` + +The similarity with JavaScript/TypeScript can also be used for dependencies +between files. Vim currently uses the `:source` command, which has several +disadvantages: +* In the sourced script, is not clear what it provides. By default all + functions are global and can be used elsewhere. +* In a script that sources other scripts, it is not clear what function comes + from what sourced script. Finding the implementation is a hassle. +* Prevention of loading the whole script twice must be manually implemented. + +We can use the `:import` and `:export` commands from the JavaScript standard to +make this much better. For example, in script "myfunction.vim" define a +function and export it: + +``` vim +vim9script " Vim9 script syntax used here + +let local = 'local variable is not exported, script-local' + +export def MyFunction() " exported function +... + +def LocalFunction() " not exported, script-local +... +``` + +And in another script import the function: + +``` vim +vim9script " Vim9 script syntax used here + +import MyFunction from 'myfunction.vim' +``` + +This looks like JavaScript/TypeScript, thus many users will understand the +syntax. + +These are ideas, this will take time to design, discuss and implement. +Eventually this will lead to Vim 9! + + +## Code for sum time measurements + +Vim was build with -O2. + +``` vim +func VimOld() + let sum = 0 + for i in range(1, 2999999) + let sum += i + endfor + return sum +endfunc + +func Python() + py3 << END +sum = 0 +for i in range(1, 3000000): + sum += i +END + return py3eval('sum') +endfunc + +func Lua() + lua << END + sum = 0 + for i = 1, 2999999 do + sum = sum + i + end +END + return luaeval('sum') +endfunc + +def VimNew() + let sum = 0 + for i in range(1, 2999999) + let sum += i + endfor + return sum +enddef + +let start = reltime() +echo VimOld() +echo 'Vim old: ' .. reltimestr(reltime(start)) + +let start = reltime() +echo Python() +echo 'Python: ' .. reltimestr(reltime(start)) + +let start = reltime() +echo Lua() +echo 'Lua: ' .. reltimestr(reltime(start)) + +let start = reltime() +echo VimNew() +echo 'Vim new: ' .. reltimestr(reltime(start)) +``` + +## Code for indent time measurements + +``` vim +def VimNew(): number + let totallen = 0 + for i in range(1, 100000) + setline(i, ' ' .. getline(i)) + totallen += len(getline(i)) + endfor + return totallen +enddef + +func VimOld() + let totallen = 0 + for i in range(1, 100000) + call setline(i, ' ' .. getline(i)) + let totallen += len(getline(i)) + endfor + return totallen +endfunc + +func Lua() + lua << END + b = vim.buffer() + totallen = 0 + for i = 1, 100000 do + b[i] = " " .. b[i] + totallen = totallen + string.len(b[i]) + end +END + return luaeval('totallen') +endfunc + +func Python() + py3 << END +cb = vim.current.buffer +totallen = 0 +for i in range(0, 100000): + cb[i] = ' ' + cb[i] + totallen += len(cb[i]) +END + return py3eval('totallen') +endfunc + +new +call setline(1, range(100000)) +let start = reltime() +echo VimOld() +echo 'Vim old: ' .. reltimestr(reltime(start)) +bwipe! + +new +call setline(1, range(100000)) +let start = reltime() +echo Python() +echo 'Python: ' .. reltimestr(reltime(start)) +bwipe! + +new +call setline(1, range(100000)) +let start = reltime() +echo Lua() +echo 'Lua: ' .. reltimestr(reltime(start)) +bwipe! + +new +call setline(1, range(100000)) +let start = reltime() +echo VimNew() +echo 'Vim new: ' .. reltimestr(reltime(start)) +bwipe! +``` diff --git a/runtime/doc/Makefile b/runtime/doc/Makefile index 514a01237f..65dc8436c6 100644 --- a/runtime/doc/Makefile +++ b/runtime/doc/Makefile @@ -149,6 +149,7 @@ DOCS = \ version7.txt \ version8.txt \ vi_diff.txt \ + vim9.txt \ visual.txt \ windows.txt \ workshop.txt @@ -289,6 +290,7 @@ HTMLS = \ version8.html \ vi_diff.html \ vimindex.html \ + vim9.html \ visual.html \ windows.html \ workshop.html diff --git a/runtime/doc/eval.txt b/runtime/doc/eval.txt index b580f2fabf..071f250807 100644 --- a/runtime/doc/eval.txt +++ b/runtime/doc/eval.txt @@ -12,6 +12,10 @@ Note: Expression evaluation can be disabled at compile time. If this has been done, the features in this document are not available. See |+eval| and |no-eval-feature|. +This file is about the backwards compatible Vim script. For Vim9 script, +which executes much faster, supports type checking and much more, see +|vim9.txt|. + 1. Variables |variables| 1.1 Variable types 1.2 Function references |Funcref| @@ -2512,8 +2516,8 @@ haslocaldir([{winnr} [, {tabnr}]]) or |:tcd| hasmapto({what} [, {mode} [, {abbr}]]) Number |TRUE| if mapping to {what} exists -histadd({history}, {item}) String add an item to a history -histdel({history} [, {item}]) String remove an item from a history +histadd({history}, {item}) Number add an item to a history +histdel({history} [, {item}]) Number remove an item from a history histget({history} [, {index}]) String get the item {index} from a history histnr({history}) Number highest index of a history hlexists({name}) Number |TRUE| if highlight group {name} exists @@ -10894,6 +10898,9 @@ New functions can be defined. These can be called just like builtin functions. The function executes a sequence of Ex commands. Normal mode commands can be executed with the |:normal| command. +This section is about the legacy functions. For the Vim9 functions, which +execute much faster, support type checking and more, see |vim9.txt|. + The function name must start with an uppercase letter, to avoid confusion with builtin functions. To prevent from using the same name in different scripts avoid obvious, short names. A good habit is to start the function name with diff --git a/runtime/doc/options.txt b/runtime/doc/options.txt index 876ddf3588..847290e29f 100644 --- a/runtime/doc/options.txt +++ b/runtime/doc/options.txt @@ -6221,6 +6221,7 @@ A jump table for the options with a short description can be found at |Q_op|. compiler/ compiler files |:compiler| doc/ documentation |write-local-help| ftplugin/ filetype plugins |write-filetype-plugin| + import/ files that are found by `:import` indent/ indent scripts |indent-expression| keymap/ key mapping files |mbyte-keymap| lang/ menu translations |:menutrans| diff --git a/runtime/doc/vim9.txt b/runtime/doc/vim9.txt new file mode 100644 index 0000000000..c6c674da83 --- /dev/null +++ b/runtime/doc/vim9.txt @@ -0,0 +1,561 @@ +*vim9.txt* For Vim version 8.2. Last change: 2019 Dec 06 + + + VIM REFERENCE MANUAL by Bram Moolenaar + + +THIS IS STILL UNDER DEVELOPMENT - ANYTHING CAN BREAK - ANYTHING CAN CHANGE + +Vim9 script commands and expressions. + +Most expression help is in |eval.txt|. This file is about the new syntax and +features in Vim9 script. + +THIS IS STILL UNDER DEVELOPMENT - ANYTHING CAN BREAK - ANYTHING CAN CHANGE + + +1 What is Vim9 script? |vim9-script| +2. Differences |vim9-differences| +3. New style functions |fast-functions| +4. Types |vim9-types| +5. Namespace, Import and Export |vim9script| + +9. Rationale |vim9-rationale| + +============================================================================== + +1. What is Vim9 script? *vim9-script* + +THIS IS STILL UNDER DEVELOPMENT - ANYTHING CAN BREAK - ANYTHING CAN CHANGE + +Vim script has been growing over time, while keeping backwards compatibility. +That means bad choices from the past often can't be changed. Execution is +quite slow, every line is parsed every time it is executed. + +The main goal of Vim9 script is to drastically improve performance. An +increase in execution speed of 10 to 100 times can be expected. A secondary +goal is to avoid Vim-specific constructs and get closer to commonly used +programming languages, such as JavaScript, TypeScript and Java. + +The performance improvements can only be achieved by not being 100% backwards +compatible. For example, in a function the arguments are not available in the +"a:" dictionary, as creating that dictionary adds quite a lot of overhead. +Other differences are more subtle, such as how errors are handled. + +The Vim9 script syntax and semantics are used in: +- a function defined with the `:def` command +- a script file where the first command is `vim9script` + +When using `:function` in a Vim9 script file the legacy syntax is used. +However, this is discouraged. + +Vim9 script and legacy Vim script can be mixed. There is no need to rewrite +old scripts, they keep working as before. + +============================================================================== + +2. Differences from legacy Vim script *vim9-differences* + +THIS IS STILL UNDER DEVELOPMENT - ANYTHING CAN BREAK - ANYTHING CAN CHANGE + +Vim9 functions ~ + +`:def` has no extra arguments like `:function` does: "range", "abort", "dict" +or "closure". A `:def` function always aborts on an error, does not get a +range passed and cannot be a "dict" function. + +In the function body: +- Arguments are accessed by name, without "a:". +- There is no "a:" dictionary or "a:000" list. Variable arguments are defined + with a name and have a list type: > + def MyFunc(...itemlist: list<type>) + for item in itemlist + ... + + +Variable declarations with :let and :const ~ + +Local variables need to be declared with `:let`. Local constants need to be +declared with `:const`. We refer to both as "variables". + +Variables can be local to a script, function or code block: > + vim9script + let script_var = 123 + def SomeFunc() + let func_var = script_var + if cond + let block_var = func_var + ... + +The variables are only visible in the block where they are defined and nested +blocks. Once the block ends the variable is no longer accessible: > + if cond + let inner = 5 + else + let inner = 0 + endif + echo inner " Error! + +The declaration must be done earlier: > + let inner: number + if cond + inner = 5 + else + inner = 0 + endif + echo inner + +To intentionally use a variable that won't be available later, a block can be +used: > + { + let temp = 'temp' + ... + } + echo temp " Error! + +An existing variable cannot be assigend to with `:let`, since that implies a +declaration. An exception is global variables: these can be both used with +and without `:let`, |