diff options
Diffstat (limited to 'doc')
| -rw-r--r-- | doc/man1/openssl-cmp.pod.in | 12 | ||||
| -rw-r--r-- | doc/man3/ASN1_item_d2i_bio.pod | 2 | ||||
| -rw-r--r-- | doc/man3/OCSP_sendreq_new.pod | 46 | ||||
| -rw-r--r-- | doc/man3/OSSL_HTTP_REQ_CTX.pod | 157 | ||||
| -rw-r--r-- | doc/man3/OSSL_HTTP_transfer.pod | 272 | ||||
| -rw-r--r-- | doc/man3/X509_load_http.pod | 2 |
6 files changed, 300 insertions, 191 deletions
diff --git a/doc/man1/openssl-cmp.pod.in b/doc/man1/openssl-cmp.pod.in index 28ea4ee6a5..49105ca315 100644 --- a/doc/man1/openssl-cmp.pod.in +++ b/doc/man1/openssl-cmp.pod.in @@ -52,6 +52,7 @@ Message transfer options: [B<-proxy> I<[http[s]://][userinfo@]host[:port][/path][?query][#fragment]>] [B<-no_proxy> I<addresses>] [B<-recipient> I<name>] +[B<-keep_alive> I<value>] [B<-msg_timeout> I<seconds>] [B<-total_timeout> I<seconds>] @@ -488,11 +489,20 @@ as far as any of those is present, else the NULL-DN as last resort. The argument must be formatted as I</type0=value0/type1=value1/type2=...>. For details see the description of the B<-subject> option. +=item B<-keep_alive> I<value> + +If the given value is 0 then HTTP connections are not kept open +after receiving a response, which is the default behavior for HTTP 1.0. +If the value is 1 or 2 then persistent connections are requested. +If the value is 2 then persistent connections are required, +i.e., in case the server does not grant them an error occurs. +The default value is 1, which means preferring to keep the connection open. + =item B<-msg_timeout> I<seconds> Number of seconds (or 0 for infinite) a CMP request-response message round trip is allowed to take before a timeout error is returned. -Default is 120. +Default is to use the B<-total_timeout> setting. =item B<-total_timeout> I<seconds> diff --git a/doc/man3/ASN1_item_d2i_bio.pod b/doc/man3/ASN1_item_d2i_bio.pod index 4ec7da838c..bd3c9b06c2 100644 --- a/doc/man3/ASN1_item_d2i_bio.pod +++ b/doc/man3/ASN1_item_d2i_bio.pod @@ -31,7 +31,7 @@ ASN1_item_i2d_mem_bio() returns a pointer to a memory BIO or NULL on error. =head1 HISTORY -The functions described here were added in OpenSSL 3.0. +ASN1_item_i2d_mem_bio() was added in OpenSSL 3.0. =head1 COPYRIGHT diff --git a/doc/man3/OCSP_sendreq_new.pod b/doc/man3/OCSP_sendreq_new.pod index 10c6131f86..51469661de 100644 --- a/doc/man3/OCSP_sendreq_new.pod +++ b/doc/man3/OCSP_sendreq_new.pod @@ -18,10 +18,7 @@ OCSP_REQ_CTX_set1_req #include <openssl/ocsp.h> OSSL_HTTP_REQ_CTX *OCSP_sendreq_new(BIO *io, const char *path, - const OCSP_REQUEST *req, int maxline); - - int OCSP_sendreq_nbio(OCSP_RESPONSE **presp, OSSL_HTTP_REQ_CTX *rctx); - + const OCSP_REQUEST *req, int buf_size); OCSP_RESPONSE *OCSP_sendreq_bio(BIO *io, const char *path, OCSP_REQUEST *req); Deprecated since OpenSSL 3.0, can be hidden entirely by defining @@ -29,12 +26,12 @@ B<OPENSSL_API_COMPAT> with a suitable version value, see L<openssl_user_macros(7)>: typedef OSSL_HTTP_REQ_CTX OCSP_REQ_CTX; + int OCSP_sendreq_nbio(OCSP_RESPONSE **presp, OSSL_HTTP_REQ_CTX *rctx); int OCSP_REQ_CTX_i2d(OCSP_REQ_CT *rctx, const ASN1_ITEM *it, ASN1_VALUE *req); int OCSP_REQ_CTX_add1_header(OCSP_REQ_CT *rctx, const char *name, const char *value); void OCSP_REQ_CTX_free(OCSP_REQ_CTX *rctx); - void OCSP_set_max_response_length(OCSP_REQ_CT *rctx, - unsigned long len); + void OCSP_set_max_response_length(OCSP_REQ_CT *rctx, unsigned long len); int OCSP_REQ_CTX_set1_req(OCSP_REQ_CTX *rctx, const OCSP_REQUEST *req); =head1 DESCRIPTION @@ -42,28 +39,32 @@ L<openssl_user_macros(7)>: These functions perform an OCSP POST request / response transfer over HTTP, using the HTTP request functions described in L<OSSL_HTTP_REQ_CTX(3)>. -The function OCSP_sendreq_new() builds a complete B<OSSL_HTTP_REQ_CTX> -structure using connection B<BIO> I<io>, the URL path I<path>, the OCSP -request I<req>, and with a response header maximum line length of I<maxline>. -If I<maxline> is zero a default value of 4k is used. +The function OCSP_sendreq_new() builds a complete B<OSSL_HTTP_REQ_CTX> structure +with the B<BIO> I<io> to be used for requests and reponse, the URL path I<path>, +optionally the OCSP request I<req>, and a response header maximum line length +of I<buf_size>. If I<buf_size> is zero a default value of 4KiB is used. The I<req> may be set to NULL and provided later using OCSP_REQ_CTX_set1_req() -or L<OSSL_HTTP_REQ_CTX_set1_req(3)> . - +or L<OSSL_HTTP_REQ_CTX_set1_req(3)>. The I<io> and I<path> arguments to OCSP_sendreq_new() correspond to the components of the URL. For example if the responder URL is C<http://example.com/ocspreq> the BIO -I<io> should be connected to host C<example.com> on port 80 and I<path> +I<io> should haven been connected to host C<example.com> on port 80 and I<path> should be set to C</ocspreq>. -OCSP_sendreq_nbio() performs I/O on the OCSP request context I<rctx>. -When the operation is complete it assigns the response, a pointer to a -B<OCSP_RESPONSE> structure, in I<*presp>. - -OCSP_sendreq_bio() is the same as a call to OCSP_sendreq_new() followed by -OCSP_sendreq_nbio() and then OCSP_REQ_CTX_free() in a single call, with a +OCSP_sendreq_nbio() attempts to send the request prepared in I<rctx> +and to gather the response via HTTP, using the BIO I<io> and I<path> +that were given when calling OCSP_sendreq_new(). +If the operation gets completed it assigns the response, +a pointer to a B<OCSP_RESPONSE> structure, in I<*presp>. +The function may need to be called again if its result is -1, which indicates +L<BIO_should_retry(3)>. In such a case it is advisable to sleep a little in +between, using L<BIO_wait(3)> on the read BIO to prevent a busy loop. + +OCSP_sendreq_bio() combines OCSP_sendreq_new() with as many calls of +OCSP_sendreq_nbio() as needed and then OCSP_REQ_CTX_free(), with a response header maximum line length 4k. It waits indefinitely on a response. It does not support setting a timeout or adding headers and is retained -for compatibility; use OCSP_sendreq_nbio() instead. +for compatibility; use L<OSSL_HTTP_transfer(3)> instead. OCSP_REQ_CTX_i2d(rctx, it, req) is equivalent to the following: @@ -88,15 +89,14 @@ L<OSSL_HTTP_REQ_CTX_set_max_response_length(3)>. OCSP_sendreq_new() returns a valid B<OSSL_HTTP_REQ_CTX> structure or NULL if an error occurred. -OCSP_sendreq_nbio(), OCSP_REQ_CTX_i2d(), and OCSP_REQ_CTX_set1_req() -return 1 for success and 0 for failure. +OCSP_sendreq_nbio() returns 1 for success, 0 on error, -1 if retry is needed. OCSP_sendreq_bio() returns the B<OCSP_RESPONSE> structure sent by the responder or NULL if an error occurred. =head1 SEE ALSO -L<OSSL_HTTP_REQ_CTX(3)> +L<OSSL_HTTP_REQ_CTX(3)>, L<OSSL_HTTP_transfer(3)>, L<OCSP_cert_to_id(3)>, L<OCSP_request_add1_nonce(3)>, L<OCSP_REQUEST_new(3)>, diff --git a/doc/man3/OSSL_HTTP_REQ_CTX.pod b/doc/man3/OSSL_HTTP_REQ_CTX.pod index 8e928f19fa..a09b9b81a9 100644 --- a/doc/man3/OSSL_HTTP_REQ_CTX.pod +++ b/doc/man3/OSSL_HTTP_REQ_CTX.pod @@ -7,11 +7,15 @@ OSSL_HTTP_REQ_CTX_new, OSSL_HTTP_REQ_CTX_free, OSSL_HTTP_REQ_CTX_set_request_line, OSSL_HTTP_REQ_CTX_add1_header, +OSSL_HTTP_REQ_CTX_set_expected, OSSL_HTTP_REQ_CTX_set1_req, OSSL_HTTP_REQ_CTX_nbio, -OSSL_HTTP_REQ_CTX_sendreq_d2i, +OSSL_HTTP_REQ_CTX_nbio_d2i, +OSSL_HTTP_REQ_CTX_exchange, OSSL_HTTP_REQ_CTX_get0_mem_bio, -OSSL_HTTP_REQ_CTX_set_max_response_length +OSSL_HTTP_REQ_CTX_get_resp_len, +OSSL_HTTP_REQ_CTX_set_max_response_length, +OSSL_HTTP_is_alive - HTTP client low-level functions =head1 SYNOPSIS @@ -20,11 +24,7 @@ OSSL_HTTP_REQ_CTX_set_max_response_length typedef struct ossl_http_req_ctx_st OSSL_HTTP_REQ_CTX; - OSSL_HTTP_REQ_CTX *OSSL_HTTP_REQ_CTX_new(BIO *wbio, BIO *rbio, - int maxline, unsigned long max_resp_len, - int timeout, - const char *expected_content_type, - int expect_asn1); + OSSL_HTTP_REQ_CTX *OSSL_HTTP_REQ_CTX_new(BIO *wbio, BIO *rbio, int buf_size); void OSSL_HTTP_REQ_CTX_free(OSSL_HTTP_REQ_CTX *rctx); int OSSL_HTTP_REQ_CTX_set_request_line(OSSL_HTTP_REQ_CTX *rctx, int method_POST, @@ -33,42 +33,41 @@ OSSL_HTTP_REQ_CTX_set_max_response_length int OSSL_HTTP_REQ_CTX_add1_header(OSSL_HTTP_REQ_CTX *rctx, const char *name, const char *value); + int OSSL_HTTP_REQ_CTX_set_expected(OSSL_HTTP_REQ_CTX *rctx, + const char *content_type, int asn1, + int timeout, int keep_alive); int OSSL_HTTP_REQ_CTX_set1_req(OSSL_HTTP_REQ_CTX *rctx, const char *content_type, - const ASN1_ITEM *it, ASN1_VALUE *req); + const ASN1_ITEM *it, const ASN1_VALUE *req); int OSSL_HTTP_REQ_CTX_nbio(OSSL_HTTP_REQ_CTX *rctx); - ASN1_VALUE *OSSL_HTTP_REQ_CTX_sendreq_d2i(OSSL_HTTP_REQ_CTX *rctx, - const ASN1_ITEM *it); + int OSSL_HTTP_REQ_CTX_nbio_d2i(OSSL_HTTP_REQ_CTX *rctx, + ASN1_VALUE **pval, const ASN1_ITEM *it); + BIO *OSSL_HTTP_REQ_CTX_exchange(OSSL_HTTP_REQ_CTX *rctx); BIO *OSSL_HTTP_REQ_CTX_get0_mem_bio(const OSSL_HTTP_REQ_CTX *rctx); + size_t OSSL_HTTP_REQ_CTX_get_resp_len(const OSSL_HTTP_REQ_CTX *rctx); void OSSL_HTTP_REQ_CTX_set_max_response_length(OSSL_HTTP_REQ_CTX *rctx, unsigned long len); + int OSSL_HTTP_is_alive(const OSSL_HTTP_REQ_CTX *rctx); + =head1 DESCRIPTION -B<OSSL_HTTP_REQ_CTX> is a context structure for an HTTP request, used to -collect all the necessary data to perform that request. +B<OSSL_HTTP_REQ_CTX> is a context structure for an HTTP request and response, +used to collect all the necessary data to perform that request. This file documents low-level HTTP functions rarely used directly. High-level HTTP client functions like L<OSSL_HTTP_get(3)> and L<OSSL_HTTP_transfer(3)> should be preferred. OSSL_HTTP_REQ_CTX_new() allocates a new HTTP request context structure, -which gets populated with the B<BIO> to send the request to (I<wbio>), -the B<BIO> to read the response from (I<rbio>, which may be equal to I<wbio>), -the maximum expected response header line length (I<maxline>, where a value <= 0 -indicates that the B<HTTP_DEFAULT_MAX_LINE_LENGTH> of 4KiB should be used; -this length is also used as the number of content bytes read at a time), -the maximum allowed response content length (I<max_resp_len>, where 0 means -that the B<HTTP_DEFAULT_MAX_RESP_LEN> is used, which currently is 100 KiB), -a response timeout measure in seconds (I<timeout>, -where 0 indicates no timeout, i.e., waiting indefinitely), -the expected MIME content type of the response (I<expected_content_type>, -which may be NULL for no expectation), -and a flag indicating that the response is expected to be -a DER encoded ASN.1 structure (I<expect_asn1>). +which gets populated with the B<BIO> to write/send the request to (I<wbio>), +the B<BIO> to read/receive the response from (I<rbio>, which may be equal to +I<wbio>), and the maximum expected response header line length I<buf_size>. +A value <= 0 indicates that +the B<HTTP_DEFAULT_MAX_LINE_LENGTH> of 4KiB should be used. +I<buf_size> is also used as the number of content bytes that are read at a time. The allocated context structure is also populated with an internal allocated memory B<BIO>, which collects the HTTP request and additional headers as text. -The returned context should only be used for a single HTTP request/response. OSSL_HTTP_REQ_CTX_free() frees up the HTTP request context I<rctx>. The I<wbio> and I<rbio> are not free'd and it is up to the application @@ -87,33 +86,71 @@ For example, to add a C<Host> header for C<example.com> you would call: OSSL_HTTP_REQ_CTX_add1_header(ctx, "Host", "example.com"); +OSSL_HTTP_REQ_CTX_set_expected() optionally sets in I<rctx> some expectations +of the HTTP client on the response. +Due to the structure of an HTTP request, if the I<keep_alive> argument is +nonzero the function must be used before calling OSSL_HTTP_REQ_CTX_set1_req(). +If the I<content_type> parameter +is not NULL then the client will check that the given content type string +is included in the HTTP header of the response and return an error if not. +If the I<asn1> parameter is nonzero a structure in ASN.1 encoding will be +expected as the response content and input streaming is disabled. This means +that an ASN.1 sequence header is required, its length field is checked, and +OSSL_HTTP_REQ_CTX_get0_mem_bio() should be used to get the buffered response. +Else any form of input is allowed without length checks, which is the default. +In this case the BIO given as I<rbio> argument to OSSL_HTTP_REQ_CTX_new() should +be used directly to read the response contents, which may support streaming. +If the I<timeout> parameter is > 0 this indicates the maximum number of seconds +the subsequent HTTP transfer (sending the request and receiving a response) +is allowed to take. +A value <= 0 enables waiting indefinitely, i.e., no timeout can occur. +This is the default. +If the I<keep_alive> parameter is 0, which is the default, the connection is not +kept open after receiving a response. This is the default behavior for HTTP 1.0. +If the value is 1 or 2 then a persistent connection is requested. +If the value is 2 then a persistent connection is required, +i.e., an error occurs in case the server does not grant it. + OSSL_HTTP_REQ_CTX_set1_req() is to be used if and only if the I<method_POST> -parameter in the OSSL_HTTP_REQ_CTX_set_request_line() call was 1. +parameter in the OSSL_HTTP_REQ_CTX_set_request_line() call was 1 +and an ASN.1-encoded request should be sent, which does not support streaming. It finalizes the HTTP request context by adding the DER encoding of I<req>, using the ASN.1 template I<it> to do the encoding. The HTTP header C<Content-Length> is filled out with the length of the request. If I<content_type> isn't NULL, -the HTTP header C<Content-Type> is also added with its content as value. +the HTTP header C<Content-Type> is also added with the given string value. All of this ends up in the internal memory B<BIO>. -OSSL_HTTP_REQ_CTX_nbio() attempts to send the request prepared I<rctx> -and gathering the response via HTTP, using the I<rbio> and I<wbio> +OSSL_HTTP_REQ_CTX_nbio() attempts to send the request prepared in I<rctx> +and to gather the response via HTTP, using the I<wbio> and I<rbio> that were given when calling OSSL_HTTP_REQ_CTX_new(). -When successful, the contents of the internal memory B<BIO> contains -the contents of the HTTP response, without the response headers. -It may need to be called again if its result is -1, which indicates +The function may need to be called again if its result is -1, which indicates L<BIO_should_retry(3)>. In such a case it is advisable to sleep a little in -between using L<BIO_wait(3)> on the read BIO to prevent a busy loop. - -OSSL_HTTP_REQ_CTX_sendreq_d2i() calls OSSL_HTTP_REQ_CTX_nbio(), possibly -several times until a timeout is reached, and DER decodes the received -response using the ASN.1 template I<it>. +between, using L<BIO_wait(3)> on the read BIO to prevent a busy loop. + +OSSL_HTTP_REQ_CTX_nbio_d2i() is like OSSL_HTTP_REQ_CTX_nbio() but on successs +in addition parses the response, which must be a DER-encoded ASN.1 structure, +using the ASN.1 template I<it> and places the result in I<*pval>. + +OSSL_HTTP_REQ_CTX_exchange() calls OSSL_HTTP_REQ_CTX_nbio() as often as needed +in order to exchange a request and response or until a timeout is reached. +If successful and an ASN.1-encoded response was expected, the response contents +should be read via the BIO returned by OSSL_HTTP_REQ_CTX_get0_mem_bio(). +Else the I<rbio> that was given when calling OSSL_HTTP_REQ_CTX_new() +represents the current state of reading the response. +If OSSL_HTTP_REQ_CTX_exchange() was successful, this BIO has been read past the +end of the response headers, such that the actual response contents can be read +via this BIO, which may support streaming. OSSL_HTTP_REQ_CTX_get0_mem_bio() returns the internal memory B<BIO>. Before sending the request, this could used to modify the HTTP request text. I<Use with caution!> -After receiving a response via HTTP, the BIO represents -the current state of reading the response headers and contents. +After receiving a response via HTTP, the BIO represents the current state of +reading the response headers. If the response was expected to be ASN.1 encoded, +its contents can be read via this BIO, which does not support streaming. + +OSSL_HTTP_REQ_CTX_get_resp_len() returns the size of the response contents +in I<rctx> if provided by the server as <Content-Length> header field, else 0. OSSL_HTTP_REQ_CTX_set_max_response_length() sets the maximum allowed response content length for I<rctx> to I<len>. If not set or I<len> is 0 @@ -122,6 +159,18 @@ If the C<Content-Length> header is present and exceeds this value or the content is an ASN.1 encoded structure with a length exceeding this value or both length indications are present but disagree then an error occurs. +OSSL_HTTP_is_alive() can be used to query if the HTTP connection +given by I<rctx> is still alive, i.e., has not been closed. +It returns 0 if I<rctx> is NULL. + +If the client application requested or required a persistent connection +and this was granted by the server, it can keep I<rctx> as long as it wants +to send further requests and OSSL_HTTP_is_alive() returns nonzero, +else it should call I<OSSL_HTTP_REQ_CTX_free(rctx)> or L<OSSL_HTTP_close(3)>. +In case the client application keeps I<rctx> but the connection then dies +for any reason at the server side, it will notice this obtaining an +I/O error when trying to send the next request via I<rctx>. + =head1 WARNINGS The server's response may be unexpected if the hostname that was used to @@ -155,7 +204,7 @@ and must be done exactly once in that case. =back When the request context is fully prepared, the HTTP exchange may be performed -with OSSL_HTTP_REQ_CTX_nbio() or OSSL_HTTP_REQ_CTX_sendreq_d2i(). +with OSSL_HTTP_REQ_CTX_nbio() or OSSL_HTTP_REQ_CTX_exchange(). =head1 RETURN VALUES @@ -166,20 +215,36 @@ OSSL_HTTP_REQ_CTX_free() and OSSL_HTTP_REQ_CTX_set_max_response_length() do not return values. OSSL_HTTP_REQ_CTX_set_request_line(), OSSL_HTTP_REQ_CTX_add1_header(), -OSSL_HTTP_REQ_CTX_set1_req() and OSSL_HTTP_REQ_CTX_nbio +OSSL_HTTP_REQ_CTX_set1_req(), and OSSL_HTTP_REQ_CTX_set_expected() return 1 for success and 0 for failure. -OSSL_HTTP_REQ_CTX_sendreq_d2i() returns a pointer to an B<ASN1_VALUE> for -success and NULL for failure. +OSSL_HTTP_REQ_CTX_nbio() and OSSL_HTTP_REQ_CTX_nbio_d2i() +return 1 for success, 0 on error or redirection, -1 if retry is needed. -OSSL_HTTP_REQ_CTX_get0_mem_bio() returns the internal memory B<BIO>. +OSSL_HTTP_REQ_CTX_exchange() and OSSL_HTTP_REQ_CTX_get0_mem_bio() +returns a pointer to a B<BIO> on success and NULL on failure. + +OSSL_HTTP_REQ_CTX_get_resp_len() returns the size of the response contents +or 0 if not available or an error occurred. + +OSSL_HTTP_is_alive() returns 1 if its argument is non-NULL +and the client requested a persistent connection +and the server did not disagree on keeping the connection open, else 0. =head1 SEE ALSO L<BIO_should_retry(3)>, L<BIO_wait(3)>, +L<ASN1_item_d2i_bio(3)>, +L<ASN1_item_i2d_mem_bio(3)>, +L<OSSL_HTTP_open(3)>, L<OSSL_HTTP_get(3)>, -L<OSSL_HTTP_transfer(3)> +L<OSSL_HTTP_transfer(3)>, +L<OSSL_HTTP_close(3)> + +=head1 HISTORY + +The functions described here were added in OpenSSL 3.0. =head1 COPYRIGHT diff --git a/doc/man3/OSSL_HTTP_transfer.pod b/doc/man3/OSSL_HTTP_transfer.pod index d2ff8eeebc..da84789472 100644 --- a/doc/man3/OSSL_HTTP_transfer.pod +++ b/doc/man3/OSSL_HTTP_transfer.pod @@ -2,13 +2,15 @@ =head1 NAME +OSSL_HTTP_open, +OSSL_HTTP_bio_cb_t, +OSSL_HTTP_proxy_connect, +OSSL_HTTP_set_request, +OSSL_HTTP_exchange, OSSL_HTTP_get, -OSSL_HTTP_get_asn1, -OSSL_HTTP_post_asn1, OSSL_HTTP_transfer, -OSSL_HTTP_bio_cb_t, -OSSL_HTTP_proxy_connect -- http client functions +OSSL_HTTP_close +- HTTP client high-level functions =head1 SYNOPSIS @@ -16,91 +18,53 @@ OSSL_HTTP_proxy_connect typedef BIO *(*OSSL_HTTP_bio_cb_t)(BIO *bio, void *arg, int connect, int detail); + OSSL_HTTP_REQ_CTX *OSSL_HTTP_open(const char *server, const char *port, + const char *proxy, const char *no_proxy, + int use_ssl, BIO *bio, BIO *rbio, + OSSL_HTTP_bio_cb_t bio_update_fn, void *arg, + int buf_size, int overall_timeout); + int OSSL_HTTP_proxy_connect(BIO *bio, const char *server, const char *port, + const char *proxyuser, const char *proxypass, + int timeout, BIO *bio_err, const char *prog); + int OSSL_HTTP_set_request(OSSL_HTTP_REQ_CTX *rctx, const char *path, + const STACK_OF(CONF_VALUE) *headers, + const char *content_type, BIO *req, + const char *expected_content_type, int expect_asn1, + size_t max_resp_len, int timeout, int keep_alive); + BIO *OSSL_HTTP_exchange(OSSL_HTTP_REQ_CTX *rctx, char **redirection_url); BIO *OSSL_HTTP_get(const char *url, const char *proxy, const char *no_proxy, BIO *bio, BIO *rbio, OSSL_HTTP_bio_cb_t bio_update_fn, void *arg, - const STACK_OF(CONF_VALUE) *headers, - int maxline, unsigned long max_resp_len, int timeout, - const char *expected_ct, int expect_asn1); - ASN1_VALUE *OSSL_HTTP_get_asn1(const char *url, - const char *proxy, const char *no_proxy, - BIO *bio, BIO *rbio, - OSSL_HTTP_bio_cb_t bio_update_fn, void *arg, - const STACK_OF(CONF_VALUE) *headers, - int maxline, unsigned long max_resp_len, - int timeout, const char *expected_ct, - const ASN1_ITEM *rsp_it); - ASN1_VALUE *OSSL_HTTP_post_asn1(const char *server, const char *port, - const char *path, int use_ssl, - const char *proxy, const char *no_proxy, - BIO *bio, BIO *rbio, - OSSL_HTTP_bio_cb_t bio_update_fn, void *arg, - const STACK_OF(CONF_VALUE) *headers, - const char *content_type, - const ASN1_VALUE *req, const ASN1_ITEM *req_it, - int maxline, unsigned long max_resp_len, - int timeout, const char *expected_ct, - const ASN1_ITEM *rsp_it); - BIO *OSSL_HTTP_transfer(const char *server, const char *port, const char *path, - int use_ssl, const char *proxy, const char *no_proxy, + int buf_size, const STACK_OF(CONF_VALUE) *headers, + const char *expected_content_type, int expect_asn1, + size_t max_resp_len, int timeout); + BIO *OSSL_HTTP_transfer(OSSL_HTTP_REQ_CTX **prctx, + const char *server, const char *port, + const char *path, int use_ssl, + const char *proxy, const char *no_proxy, BIO *bio, BIO *rbio, OSSL_HTTP_bio_cb_t bio_update_fn, void *arg, - const STACK_OF(CONF_VALUE) *headers, - const char *content_type, BIO *req_mem, - int maxline, unsigned long max_resp_len, int timeout, - const char *expected_ct, int expect_asn1, - char **redirection_url); - int OSSL_HTTP_proxy_connect(BIO *bio, const char *server, const char *port, - const char *proxyuser, const char *proxypass, - int timeout, BIO *bio_err, const char *prog); + int buf_size, const STACK_OF(CONF_VALUE) *headers, + const char *content_type, BIO *req, + const char *expected_content_type, int expect_asn1, + size_t max_resp_len, int timeout, int keep_alive); + int OSSL_HTTP_close(OSSL_HTTP_REQ_CTX *rctx, int ok); =head1 DESCRIPTION -OSSL_HTTP_get() uses HTTP GET to obtain data (of any type) from the given I<url> -and returns it as a memory BIO. -If the schema component of the I<url> is C<https> a TLS connection is requested -and the I<bio_update_fn> parameter, described below, must be provided. -Any userinfo and fragment components in the I<url> are ignored. -Any query component is handled as part of the path component. - -OSSL_HTTP_get_asn1() is like OSSL_HTTP_get() but in addition -parses the received contents (e.g., an X.509 certificate) -as an ASN.1 DER encoded value with the expected structure specified by I<rsp_it> -and returns it on success as a pointer to I<ASN1_VALUE>. - -OSSL_HTTP_post_asn1() is like OSSL_HTTP_get_asn1() but uses the HTTP POST method -to send a request I<req> with the ASN.1 structure defined in I<req_it> and the -given I<content_type> to the given I<server> and optional I<port> and I<path>. -If I<use_ssl> is nonzero a TLS connection is requested and the I<bio_update_fn> -parameter, described below, must be provided. +OSSL_HTTP_open() initiates an HTTP session using the I<bio> argument if not +NULL, else by connecting to a given I<server> optionally via a I<proxy>. -OSSL_HTTP_transfer() exchanges any form of HTTP request and response. -It implements the core of the functions described above. -If I<path> parameter is NULL it defaults to "/". -If I<use_ssl> is nonzero a TLS connection is requested -and the I<bio_update_fn> parameter, described below, must be provided. -If I<req_mem> is NULL it uses the HTTP GET method, else it uses HTTP POST to -send a request with the contents of the memory BIO and optional I<content_type>. -The optional list I<headers> may contain additional custom HTTP header lines. -If I<req_mem> is NULL (i.e., the HTTP method is GET) and I<redirection_url> -is not NULL the latter pointer is used to provide any new location that -the server may return with HTTP code 301 (MOVED_PERMANENTLY) or 302 (FOUND). -In this case the caller is responsible for deallocating this URL with -L<OPENSSL_free(3)>. - -The above functions have the following parameters in common. - -Typically the OpenSSL build supports sockets -and the I<bio> and I<rbio> parameters are both NULL. -In this case the client creates a network BIO internally -for connecting to the given I<server> -at the specified I<port> (if any, defaulting to 80 for HTTP or 443 for HTTPS), -optionally via a I<proxy> (respecting I<no_proxy>) as described below. -Then the client uses this internal BIO for exchanging the request and response. -If I<bio> is given and I<rbio> is NULL then the client uses this I<bio> instead. +Typically the OpenSSL build supports sockets and the I<bio> parameter is NULL. +In this case I<rbio> must be NULL as well, and the +library creates a network BIO internally for connecting to the given I<server> +at the specified I<port> if any, defaulting to 80 for HTTP or 443 for HTTPS. +Then this internal BIO is used for setting up a connection +and for exchanging one or more request and response. +If I<bio> is given and I<rbio> is NULL then this I<bio> is used instead. If both I<bio> and I<rbio> are given (which may be memory BIOs for instance) -then no explicit connection is attempted, -I<bio> is used for writing the request, and I<rbio> for reading the response. +then no explicit connection is set up, but +I<bio> is used for writing requests and I<rbio> for reading responses. As soon as the client has flushed I<bio> the server must be ready to provide a response or indicate a waiting condition via I<rbio>. @@ -121,33 +85,12 @@ Proxying plain HTTP is supported directly, while using a proxy for HTTPS connections requires a suitable callback function such as OSSL_HTTP_proxy_connect(), described below. -The I<maxline> parameter specifies the response header maximum line length, -where a value <= 0 indicates that the B<HTTP_DEFAULT_MAX_LINE_LENGTH> of 4KiB -should be used. -This length is also used as the number of content bytes that are read at a time. -The I<max_resp_len> parameter specifies the maximum response length, -where 0 indicates B<HTTP_DEFAULT_MAX_RESP_LEN>, which currently is 100 KiB. - -An ASN.1-encoded response is expected by OSSL_HTTP_get_asn1() and -OSSL_HTTP_post_asn1(), while for OSSL_HTTP_get() or OSSL_HTTP_transfer() -this is only the case if the I<expect_asn1> parameter is nonzero. -If the response header contains one or more "Content-Length" header lines and/or -an ASN.1-encoded response is expected, which should include a total length, -the length indications received are checked for consistency -and for not exceeding the maximum response length. - -If the parameter I<expected_ct> -is not NULL then the HTTP client checks that the given content type string -is included in the HTTP header of the response and returns an error if not. - -If the I<timeout> parameter is > 0 this indicates the maximum number of seconds -to wait until the transfer is complete. -A value of 0 enables waiting indefinitely, -while a value < 0 immediately leads to a timeout condition. +If I<use_ssl> is nonzero a TLS connection is requested +and the I<bio_update_fn> parameter must be provided. -The optional parameter I<bio_update_fn> with its optional argument I<arg> may -be used to modify the connection BIO used by the HTTP client (and cannot be -used when both I<bio> and I<rbio> are given). +The parameter I<bio_update_fn>, which is optional if I<use_ssl> is 0, +may be used to modify the connection BIO used by the HTTP client, +but cannot be used when both I<bio> and I<rbio> are given. I<bio_update_fn> is a BIO connect/disconnect callback function with prototype BIO *(*OSSL_HTTP_bio_cb_t)(BIO *bio, void *arg, int connect, int detail) @@ -157,7 +100,7 @@ whereby it may make use of a custom defined argument I<arg>, which may for instance refer to an I<SSL_CTX> structure. During connection establishment, just after calling BIO_do_connect_retry(), the function is invoked with the I<connect> argument being 1 and the I<detail> -argument being 1 if HTTPS is requested, i.e., SSL/TLS should be enabled. +argument being 1 if HTTPS is requested, i.e., SSL/TLS should be enabled, else 0. On disconnect I<connect> is 0 and I<detail> is 1 if no error occurred, else 0. For instance, on connect the function may prepend a TLS BIO to implement HTTPS; after disconnect it may do some diagnostic output and/or specific cleanup. @@ -166,10 +109,10 @@ Here is a simple example that supports TLS connections (but not via a proxy): BIO *http_tls_cb(BIO *hbio, void *arg, int connect, int detail) { - SSL_CTX *ctx = (SSL_CTX *)arg; - if (connect && detail) { /* connecting with TLS */ + SSL_CTX *ctx = (SSL_CTX *)arg; BIO *sbio = BIO_new_ssl(ctx, 1); + hbio = sbio != NULL ? BIO_push(sbio, hbio) : NULL; } else if (!connect && !detail) { /* disconnecting after error */ /* optionally add diagnostics here */ @@ -179,6 +122,16 @@ Here is a simple example that supports TLS connections (but not via a proxy): After disconnect the modified BIO will be deallocated using BIO_free_all(). +The I<buf_size> parameter specifies the response header maximum line length. +A value <= 0 indicates that +the B<HTTP_DEFAULT_MAX_LINE_LENGTH> of 4KiB should be used. +I<buf_size> is also used as the number of content bytes that are read at a time. + +If the I<overall_timeout> parameter is > 0 this indicates the maximum number of +seconds the overall HTTP transfer (i.e., connection setup if needed, +sending requests, and receiving responses) is allowed to take until completion. +A value <= 0 enables waiting indefinitely, i.e., no timeout. + OSSL_HTTP_proxy_connect() may be used by an above BIO connect callback function to set up an SSL/TLS connection via an HTTPS proxy. It promotes the given BIO I<bio> representing a connection @@ -186,11 +139,86 @@ pre-established with a TLS proxy using the HTTP CONNECT method, optionally using proxy client credentials I<proxyuser> and I<proxypass>, to connect with TLS protection ultimately to I<server> and I<port>. If the I<port> argument is NULL or the empty string it defaults to "443". -The I<timeout> parameter is used as described above. +If the I<timeout> parameter is > 0 this indicates the maximum number of +seconds the connection setup is allowed to take. +A value <= 0 enables waiting indefinitely, i.e., no timeout. Since this function is typically called by applications such as L<openssl-s_client(1)> it uses the I<bio_err> and I<prog> parameters (unless NULL) to print additional diagnostic information in a user-oriented way. +OSSL_HTTP_set_request() sets up in I<rctx> the request header and content data +and expectations on the response using the following parameters. +If I<path> is NULL it defaults to "/". +If I<req> is NULL the HTTP GET method will be used to send the request +else HTTP POST with the contents of I<req> and optional I<content_type>, where +the length of the data in I<req> does not need to be determined in advance: the +BIO will be read on-the-fly while sending the request, which supports streaming. +The optional list I<headers> may contain additional custom HTTP header lines. +If the parameter I<expected_content_type> +is not NULL then the client will check that the given content type string +is included in the HTTP header of the response and return an error if not. +If the I<expect_asn1> parameter is nonzero, +a structure in ASN.1 encoding will be expected as response content. +The I<max_resp_len> parameter specifies the maximum allowed +response content length, where the value 0 indicates no limit. +If the I<timeout> parameter is > 0 this indicates the maximum number of seconds +the subsequent HTTP transfer (sending the request and receiving a response) +is allowed to take. +A value of 0 enables waiting indefinitely, i.e., no timeout. +A value < 0 indicates that the I<overall_timeout> parameter value given +when opening the HTTP transfer will be used instead. +If I<keep_alive> is 0 the connection is not kept open +after receiving a response, which is the default behavior for HTTP 1.0. +If the value is 1 or 2 then a persistent connection is requested. +If the value is 2 then a persistent connection is required, +i.e., an error occurs in case the server does not grant it. + +OSSL_HTTP_exchange() exchanges any form of HTTP request and response +as specified by I<rctx>, which must include both connection and request data, +typically set up using OSSL_HTTP_open() and OSSL_HTTP_set_request(). +It implements the core of the functions described below. +If the HTTP method is GET and I<redirection_url> +is not NULL the latter pointer is used to provide any new location that +the server may return with HTTP code 301 (MOVED_PERMANENTLY) or 302 (FOUND). +In this case the function returns NULL and the caller is +responsible for deallocating the URL with L<OPENSSL_free(3)>. +If the response header contains one or more "Content-Length" header lines and/or +an ASN.1-encoded response is expected, which should include a total length, +the length indications received are checked for consistency +and for not exceeding any given maximum response length. +On receiving a response, the function returns the contents as a memory BIO, +which does not support streaming, in case an ASN.1-encoded response is expected. +Else it returns directly the read BIO that holds the response contents, +which allows a response of indefinite length and may support streaming. + +OSSL_HTTP_get() uses HTTP GET to obtain data from I<bio> if non-NULL, +else from the server contained in the I<url>, and returns it as a BIO. +It supports redirection via HTTP status code 301 or 302. It is meant for +transfers with a single round trip, so does not support persistent connections. +If I<bio> is non-NULL, any host and port components in the I<url> are not used +for connecting but the hostname is used, as usual, for the C<Host> header. +Any userinfo and fragment components in the I<url> are ignored. +Any query component is handled as part of the path component. +If the scheme component of the I<url> is C<https> a TLS connection is requested +and the I<bio_update_fn>, as described for OSSL_HTTP_open(), must be provided. +Also the remaining parameters are interpreted as described for OSSL_HTTP_open() +and OSSL_HTTP_set_request(), respectively. + +OSSL_HTTP_transfer() exchanges an HTTP request and response +over a connection managed via I<prctx> without supporting redirection. +It combines OSSL_HTTP_open(), OSSL_HTTP_set_request(), OSSL_HTTP_exchange(), +and OSSL_HTTP_close(). +If I<prctx> is not NULL it reuses any open connection represented by a non-NULL +I<*prctx>. It keeps the connection open if a persistent connection is requested +or required and this was granted by the server, else it closes the connection +and assigns NULL to I<*prctx>. +The remaining parameters are interpreted as described for OSSL_HTTP_open() +and OSSL_HTTP_set_request(), respectively. + +OSSL_HTTP_close() closes the connection and releases I<rctx>. +The I<ok> parameter is passed to any BIO update function +given during setup as described above for OSSL_HTTP_open(). + =head1 NOTES The names of the environment variables used by this implementation: @@ -200,23 +228,29 @@ other HTTP client implementations such as wget, curl, and |