From 90699176b07469e0b6b688ed88bc3f1deb5ccc26 Mon Sep 17 00:00:00 2001 From: Hugo Landau Date: Wed, 1 Mar 2023 16:52:40 +0000 Subject: QUIC CC: Major revisions to CC abstract interface Reviewed-by: Tomas Mraz Reviewed-by: Matt Caswell (Merged from https://github.com/openssl/openssl/pull/20423) --- doc/designs/quic-design/congestion-control.md | 293 +++++--------------------- 1 file changed, 47 insertions(+), 246 deletions(-) (limited to 'doc/designs') diff --git a/doc/designs/quic-design/congestion-control.md b/doc/designs/quic-design/congestion-control.md index d5bb4a1d79..d2c9323695 100644 --- a/doc/designs/quic-design/congestion-control.md +++ b/doc/designs/quic-design/congestion-control.md @@ -1,249 +1,50 @@ Congestion control API design ============================= -This is mostly inspired by the MSQUIC congestion control API -as it was the most isolated one among the libraries. - -The API is designed in a way to be later easily transformed into a -a fetchable implementation API. - -The API is centered around two structures - the `OSSL_CC_METHOD` -structure holding the API calls into the congestion control module -and `OSSL_CC_DATA` opaque type that holds the data of needed -for the operation of the module. - -Most of the information when the functions of the API are -supposed to be called and how the API is designed to operate -should be clear from the comments for the individual -functions in [Appendix A](#appendix-a). - -Changeables ------------ - -As some parameters that the congestion control algorithm needs might -be updated during the lifetime of the connection these parameters -are called changeables. - -The CC implementation will save the pointers to these parameters' -data and will use the current value of the data at the pointer when -it needs to recalculate the allowance and whether the sending is -blocked or not. - -Some examples of changeables - - - `payload_length` - the maximum length of payload that can be sent - in a single UDP packet (not counting UDP and IP headers). - - `smoothed_rtt` - the current round trip time of the connection - as computed from the acked packets. - - `rtt_variance` - the variance of the `smoothed_rtt` value. - -Thread handling ---------------- - -The `OSSL_CC_DATA` is created with the `new` function per each -connection. The expectation is that there is only a single thread -accessing the `ccdata` which should not be a limitation as the calls -into the module will be done from the packetizer layer of the QUIC -connection handling and there the eventual writes from multiple threads -handling individual streams of the connection have to be synchronized -already to create the packets. - -Congestion event handling -------------------------- - -The congestion state of a connection can change only after some event -happens - i.e., a packet is considered lost meaning the `on_data_lost()` -is called, or an ack arrives for a packet causing calls to -`on_data_acked()` or `on_spurious_congestion_event()` functions. -The congestion control does not produce any timer events by itself. - -Exemptions ----------- - -To facilitate probing and to avoid having to always special-case -probing packets when considering congestion on sending, the -`set_exemption()` function allows setting a number of packets that are -allowed to be sent even when forbidden by the eventual congestion state. - -The exemptions must be used if and only if a packet (or multiple packets) -has to be sent as required by the protocol regardless of the congestion state. - -Paths ------ - -Initially the design expects that only a single path per-connection is -actively sending data. In future when multiple active paths sending data -shall be supported the instances of `OSSL_CC_DATA` would be per-path. - -There might need to be further adjustments needed in that case. However -at least initially this API is intended to be internal to the -OpenSSL library allowing any necessary changes of the API. - -Appendix A ----------- - -Proposed header file with comments explaining the individual -functions. The API is meant to be internal initially so the method -accessors to set the individual functions will be added later once -the API is public. Alternatively this might be also implemented -as fetchable dispatch API. - -```C -/* - * Copyright 2022 The OpenSSL Project Authors. All Rights Reserved. - * - * Licensed under the Apache License 2.0 (the "License"). You may not use - * this file except in compliance with the License. You can obtain a copy - * in the file LICENSE in the source distribution or at - * https://www.openssl.org/source/license.html - */ - -#include - -typedef struct ossl_cc_method_st OSSL_CC_METHOD; -/* - * An OSSL_CC_DATA is an externally defined opaque pointer holding - * the internal data of the congestion control method - */ -typedef struct ossl_cc_data_st OSSL_CC_DATA; - -/* - * Once this becomes public API then we will need functions to create and - * free an OSSL_CC_METHOD, as well as functions to get/set the various - * function pointers....unless we make it fetchable. - */ -struct ossl_cc_method_st { - /* - * Create a new OSSL_CC_DATA object to handle the congestion control - * calculations. - * - * |settings| are mandatory settings that will cause the - * new() call to fail if they are not understood). - * |options| are optional settings that will not - * cause the new() call to fail if they are not understood. - * |changeables| contain additional parameters that the congestion - * control algorithms need that can be updated during the - * connection lifetime - for example size of the datagram payload. - * To avoid calling a function with OSSL_PARAM array every time - * these parameters are changed the addresses of these param - * values are considered permanent and the values can be updated - * any time. - */ - OSSL_CC_DATA *(*new)(OSSL_PARAM *settings, OSSL_PARAM *options, - OSSL_PARAM *changeables); - - /* - * Release the OSSL_CC_DATA. - */ - void (*free)(OSSL_CC_DATA *ccdata); - - /* - * Reset the congestion control state. - * |flags| to support different level of reset (partial/full). - */ - void (*reset)(OSSL_CC_DATA *ccdata, int flags); - - /* - * Set number of packets exempted from CC - used for probing - * |numpackets| is a small value (2). - * Returns 0 on error, 1 otherwise. - */ - int (*set_exemption)(OSSL_CC_DATA *ccdata, int numpackets); - - /* - * Get current number of packets exempted from CC. - * Returns negative value on error, the number otherwise. - */ - int (*get_exemption)(OSSL_CC_DATA *ccdata); - - /* - * Returns 1 if sending is allowed, 0 otherwise. - */ - int (*can_send)(OSSL_CC_DATA *ccdata); - - /* - * Returns number of bytes allowed to be sent. - * |time_since_last_send| is time since last send operation - * in microseconds. - * |time_valid| is 1 if the |time_since_last_send| holds - * a meaningful value, 0 otherwise. - */ - size_t (*get_send_allowance)(OSSL_CC_DATA *ccdata, - uint64_t time_since_last_send, - int time_valid); - - /* - * Returns the maximum number of bytes allowed to be in flight. - */ - size_t (*get_bytes_in_flight_max)(OSSL_CC_DATA *ccdata); - - /* - * Returns the next time at which the CC will release more budget for - * sending, or ossl_time_infinite(). - */ - OSSL_TIME (*get_next_credit_time)(OSSL_CC_DATA *ccdata); - - /* - * To be called when a packet with retransmittable data was sent. - * |num_retransmittable_bytes| is the number of bytes sent - * in the packet that are retransmittable. - * Returns 1 on success, 0 otherwise. - */ - int (*on_data_sent)(OSSL_CC_DATA *ccdata, - size_t num_retransmittable_bytes); - - /* - * To be called when retransmittable data was invalidated. - * I.E. they are not considered in-flight anymore but - * are neither acknowledged nor lost. In particular used when - * 0RTT data was rejected. - * |num_retransmittable_bytes| is the number of bytes - * of the invalidated data. - * Returns 1 if sending is unblocked (can_send returns 1), 0 - * otherwise. - */ - int (*on_data_invalidated)(OSSL_CC_DATA *ccdata, - size_t num_retransmittable_bytes); - - /* - * To be called when sent data was acked. - * |time_now| is current time in microseconds. - * |largest_pn_acked| is the largest packet number of the acked - * packets. - * |num_retransmittable_bytes| is the number of retransmittable - * packet bytes that were newly acked. - * Returns 1 if sending is unblocked (can_send returns 1), 0 - * otherwise. - */ - int (*on_data_acked)(OSSL_CC_DATA *ccdata, - uint64_t time_now, - uint64_t last_pn_acked, - size_t num_retransmittable_bytes); - - /* - * To be called when sent data is considered lost. - * |largest_pn_lost| is the largest packet number of the lost - * packets. - * |largest_pn_sent| is the largest packet number sent on this - * connection. - * |num_retransmittable_bytes| is the number of retransmittable - * packet bytes that are newly considered lost. - * |persistent_congestion| is 1 if the congestion is considered - * persistent (see RFC 9002 Section 7.6), 0 otherwise. - */ - void (*on_data_lost)(OSSL_CC_DATA *ccdata, - uint64_t largest_pn_lost, - uint64_t largest_pn_sent, - size_t num_retransmittable_bytes, - int persistent_congestion); - - /* - * To be called when all lost data from the previous call to - * on_data_lost() was actually acknowledged. - * This reverts the size of the congestion window to the state - * before the on_data_lost() call. - * Returns 1 if sending is unblocked, 0 otherwise. - */ - int (*on_spurious_congestion_event)(OSSL_CC_DATA *ccdata); -}; -``` +We use an abstract interface for the QUIC congestion controller to facilitate +use of pluggable QUIC congestion controllers in the future. The interface is +based on interfaces suggested by RFC 9002 and MSQUIC's congestion control APIs. + +`OSSL_CC_METHOD` provides a vtable of function pointers to congestion controller +methods. `OSSL_CC_DATA` is an opaque type representing a congestion controller +instance. + +For details on the API, see the comments in `include/internal/quic_cc.h`. + +Congestion controllers are not thread safe; the caller is responsible for +synchronisation. + +Congestion controllers may vary their state with respect to time. This is +faciliated via the `get_wakeup_deadline` method and the `now` argument to the +`new` method, which provides access to a clock. While no current congestion +controller makes use of this facility, it can be used by future congestion +controllers to implement packet pacing. + +Congestion controllers may expose integer configuration options via the +`set_option_uint` and `get_option_uint` methods. These options may be specific +to the congestion controller method, although there are some well known ones +intended to be common to all congestion controllers. The use of strings for +option names is avoided for performance reasons. + +Currently, the only dependency injected to a congestion controller is access to +a clock. In the future it is likely that access at least to the statistics +manager will be provided. Excessive futureproofing of the congestion controller +interface has been avoided as this is currently an internal API for which no API +stability guarantees are required; for example, no currently implemented +congestion control algorithm requires access to the statistics manager, but such +access can readily be added later as needed. + +QUIC congestion control state is per-path, per-connection. Currently we support +only a single path per connection, so there is one congestion control instance +per connection. This may change in future. + +While the congestion control API is roughly based around the arrangement of +functions as described by the congestion control psuedocode in RFC 9002, there +are some deliberate changes in order to obtain cleaner separation between the +loss detection and congestion control functions. Where a literal option of RFC +9002 psuedocode would require a congestion controller to access the ACK +manager's internal state directly, the interface between the two has been +changed to avoid this. This involves some small amounts of functionality which +RFC 9002 considers part of the congestion controller being part of the ACK +manager in our implementation. See the comments in `include/internal/quic_cc.h` +and `ssl/quic/quic_ackm.c` for more information. -- cgit v1.2.3