Change readfile() to ignore byte order marks, unless in binary mode.

4 files changed, 21 insertions, 6 deletions

diff --git a/runtime/doc/eval.txt b/runtime/doc/eval.txt
index 51ec4e1558..37a935cf01 100644
--- a/runtime/doc/eval.txt
+++ b/runtime/doc/eval.txt
@@ -1,4 +1,4 @@
-*eval.txt*	For Vim version 7.3e.  Last change: 2010 Aug 04
+*eval.txt*	For Vim version 7.3e.  Last change: 2010 Aug 07
 
 
 		  VIM REFERENCE MANUAL	  by Bram Moolenaar
@@ -4524,6 +4524,7 @@ readfile({fname} [, {binary} [, {max}]])
 		as an item.  Lines broken at NL characters.  Macintosh files
 		separated with CR will result in a single long line (unless a
 		NL appears somewhere).
+		All NUL characters are replaced with a NL character.
 		When {binary} is equal to "b" binary mode is used:
 		- When the last line ends in a NL an extra empty list item is
 		  added.
@@ -4531,7 +4532,8 @@ readfile({fname} [, {binary} [, {max}]])
 		Otherwise:
 		- CR characters that appear before a NL are removed.
 		- Whether the last line ends in a NL or not does not matter.
-		All NUL characters are replaced with a NL character.
+		- When 'encoding' is Unicode any UTF-8 byte order mark is
+		  removed from the text.
 		When {max} is given this specifies the maximum number of lines
 		to be read.  Useful if you only want to check the first ten
 		lines of a file: >
diff --git a/runtime/doc/tags b/runtime/doc/tags
index f72af0e845..e16c8e9160 100644
--- a/runtime/doc/tags
+++ b/runtime/doc/tags
@@ -7917,6 +7917,7 @@ tex-package	syntax.txt	/*tex-package*
 tex-runon	syntax.txt	/*tex-runon*
 tex-slow	syntax.txt	/*tex-slow*
 tex-style	syntax.txt	/*tex-style*
+tex-verb	syntax.txt	/*tex-verb*
 tex.vim	syntax.txt	/*tex.vim*
 text-functions	usr_41.txt	/*text-functions*
 text-objects	motion.txt	/*text-objects*
diff --git a/runtime/doc/todo.txt b/runtime/doc/todo.txt
index 299ea625b1..e4bdbc5134 100644
--- a/runtime/doc/todo.txt
+++ b/runtime/doc/todo.txt
@@ -33,10 +33,6 @@ be worked on, but only if you sponsor Vim development.  See |sponsor|.
 Before release 7.3:
 - Rename vim73 branch to default (hints: Xavier de Gaye, 2010 May 23)
 
-Cursor positioning wrong with 0x200e character. (John Becket, 2010 May 6)
-
-Patch to make more characters work in dialogs. (Yankwei Jia, 2010 Aug 4)
-
 Should readfile() ignore BOM when not in binary mode?
 
 Bug: searching for tags file uses 'suffixesadd', should not happen. (Dominique
@@ -64,6 +60,8 @@ accented character. (Tony Mechelynck, 2010 Apr 15)
 Patch: Let rare word highlighting overrule good word highlighting.
 (Jakson A. Aquino, 2010 Jul 30)
 
+Patch to make more characters work in dialogs. (Yankwei Jia, 2010 Aug 4)
+
 ":drop" does not respect 'autochdir'. (Peter Odding, 2010 Jul 24)
 
 Problem with cursor in the wrong column. (SungHyun Nam, 2010 Mar 11)
diff --git a/src/eval.c b/src/eval.c
index 8a26b6690c..5dfb7f2266 100644
--- a/src/eval.c
+++ b/src/eval.c
@@ -14267,6 +14267,20 @@ f_readfile(argvars, rettv)
 	    }
 	    else if (buf[filtd] == NUL)
 		buf[filtd] = '\n';
+#ifdef FEAT_MBYTE
+	    else if (buf[filtd] == 0xef
+		    && enc_utf8
+		    && filtd + 2 < buflen
+		    && !binary
+		    && buf[filtd + 1] == 0xbb
+		    && buf[filtd + 2] == 0xbf)
+	    {
+		/* remove utf-8 byte order mark */
+		mch_memmove(buf + filtd, buf + filtd + 3, buflen - filtd - 3);
+		--filtd;
+		buflen -= 3;
+	    }
+#endif
 	}
 	if (readlen <= 0)
 	    break;