runtime: cleanup and add config options (#1807)

* runtime: cleanup and add config options

This patch finishes the cleanup as part of the transition to Tokio 0.2.
A number of changes were made to take advantage of having all Tokio
types in a single crate. Also, fixes using Tokio types from
`spawn_blocking`.

* Many threads, one resource driver

Previously, in the threaded scheduler, a resource driver (mio::Poll /
timer combo) was created per thread. This was more or less fine, except
it required balancing across the available drivers. When using a
resource driver from **outside** of the thread pool, balancing is
tricky. The change was original done to avoid having a dedicated driver
thread.

Now, instead of creating many resource drivers, a single resource driver
is used. Each scheduler thread will attempt to "lock" the resource
driver before parking on it. If the resource driver is already locked,
the thread uses a condition variable to park. Contention should remain
low as, under load, the scheduler avoids using the drivers.

* Add configuration options to enable I/O / time

New configuration options are added to `runtime::Builder` to allow
enabling I/O and time drivers on a runtime instance basis. This is
useful when wanting to create lightweight runtime instances to execute
compute only tasks.

* Bug fixes

The condition variable parker is updated to the same algorithm used in
`std`. This is motivated by some potential deadlock cases discovered by
`loom`.

The basic scheduler is fixed to fairly schedule tasks. `push_front` was
accidentally used instead of `push_back`.

I/O, time, and spawning now work from within `spawn_blocking` closures.

* Misc cleanup

The threaded scheduler is no longer generic over `P :Park`. Instead, it
is hard coded to a specific parker. Tests, including loom tests, are
updated to use `Runtime` directly. This provides greater coverage.

The `blocking` module is moved back into `runtime` as all usage is
within `runtime` itself.

1 files changed, 22 insertions, 12 deletions

diff --git a/tokio/src/runtime/io.rs b/tokio/src/runtime/io.rs
index 014d39de..e912f6f7 100644
--- a/tokio/src/runtime/io.rs
+++ b/tokio/src/runtime/io.rs
@@ -6,8 +6,12 @@
 /// Re-exported for convenience.
 pub(crate) use std::io::Result;
 
-cfg_io_driver! {
+pub(crate) use variant::*;
+
+#[cfg(all(feature = "io-driver", not(loom)))]
+mod variant {
     use crate::io::driver;
+    use crate::park::{Either, ParkThread};
 
     use std::io;
 
@@ -16,27 +20,33 @@ cfg_io_driver! {
     /// When the `io-driver` feature is enabled, this is the "real" I/O driver
     /// backed by Mio. Without the `io-driver` feature, this is a thread parker
     /// backed by a condition variable.
-    pub(crate) type Driver = driver::Driver;
+    pub(crate) type Driver = Either<driver::Driver, ParkThread>;
 
     /// The handle the runtime stores for future use.
     ///
     /// When the `io-driver` feature is **not** enabled, this is `()`.
-    pub(crate) type Handle = driver::Handle;
+    pub(crate) type Handle = Option<driver::Handle>;
 
-    pub(crate) fn create_driver() -> io::Result<(Driver, Handle)> {
-        let driver = driver::Driver::new()?;
-        let handle = driver.handle();
+    pub(crate) fn create_driver(enable: bool) -> io::Result<(Driver, Handle)> {
+        if enable {
+            let driver = driver::Driver::new()?;
+            let handle = driver.handle();
 
-        Ok((driver, handle))
+            Ok((Either::A(driver), Some(handle)))
+        } else {
+            let driver = ParkThread::new();
+            Ok((Either::B(driver), None))
+        }
     }
 
-    pub(crate) fn set_default(handle: &Handle) -> driver::DefaultGuard<'_> {
-        driver::set_default(handle)
+    pub(crate) fn set_default(handle: &Handle) -> Option<driver::DefaultGuard<'_>> {
+        handle.as_ref().map(|handle| driver::set_default(handle))
     }
 }
 
-cfg_not_io_driver! {
-    use crate::runtime::park::ParkThread;
+#[cfg(any(not(feature = "io-driver"), loom))]
+mod variant {
+    use crate::park::ParkThread;
 
     use std::io;
 
@@ -46,7 +56,7 @@ cfg_not_io_driver! {
     /// There is no handle
     pub(crate) type Handle = ();
 
-    pub(crate) fn create_driver() -> io::Result<(Driver, Handle)> {
+    pub(crate) fn create_driver(_enable: bool) -> io::Result<(Driver, Handle)> {
         let driver = ParkThread::new();
 
         Ok((driver, ()))