OBJ_obj2txt(): fix off-by-one documentation of the result

This backports the doc improvements of #17188.

Reviewed-by: Paul Dale <pauli@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/17189)

1 files changed, 11 insertions, 15 deletions

diff --git a/doc/man3/OBJ_nid2obj.pod b/doc/man3/OBJ_nid2obj.pod
index c22117738e..482cc320c7 100644
--- a/doc/man3/OBJ_nid2obj.pod
+++ b/doc/man3/OBJ_nid2obj.pod
@@ -71,12 +71,14 @@ as well as numerical forms. If I<no_name> is 1 only the numerical form
 is acceptable.
 
 OBJ_obj2txt() converts the B<ASN1_OBJECT> I<a> into a textual representation.
-The representation is written as a null terminated string to I<buf>
+Unless I<buf> is NULL,
+the representation is written as a NUL-terminated string to I<buf>, where
 at most I<buf_len> bytes are written, truncating the result if necessary.
-The total amount of space required is returned. If I<no_name> is 0 then
-if the object has a long or short name then that will be used, otherwise
-the numerical form will be used. If I<no_name> is 1 then the numerical
-form will always be used.
+In any case it returns the total string length, excluding the NUL character,
+required for non-truncated representation, or -1 on error.
+If I<no_name> is 0 then if the object has a long or short name
+then that will be used, otherwise the numerical form will be used.
+If I<no_name> is 1 then the numerical form will always be used.
 
 i2t_ASN1_OBJECT() is the same as OBJ_obj2txt() with the I<no_name> set to zero.
 
@@ -152,9 +154,10 @@ a NID or B<NID_undef> on error.
 
 OBJ_add_sigid() returns 1 on success or 0 on error.
 
-OBJ_obj2txt() returns the number of bytes written to I<buf> if I<buf_len> is big enough. 
-Otherwise, the result is truncated and the total amount of space required is returned. 
-It also returns -1 on error.
+i2t_ASN1_OBJECT() an OBJ_obj2txt() return -1 on error.
+On success, they return the length of the string written to I<buf> if I<buf> is
+not NULL and I<buf_len> is big enough, otherwise the total string length.
+Note that this does not count the trailing NUL character.
 
 =head1 EXAMPLES
 
@@ -178,13 +181,6 @@ Create a new object directly:
 
 =head1 BUGS
 
-OBJ_obj2txt() is awkward and messy to use: it doesn't follow the
-convention of other OpenSSL functions where the buffer can be set
-to B<NULL> to determine the amount of data that should be written.
-Instead I<buf> must point to a valid buffer and I<buf_len> should
-be set to a positive value. A buffer length of 80 should be more
-than enough to handle any OID encountered in practice.
-
 Neither OBJ_create() nor OBJ_add_sigid() do any locking and are thus not
 thread safe.  Moreover, none of the other functions should be called while
 concurrent calls to these two functions are possible.