-
Notifications
You must be signed in to change notification settings - Fork 672
Executors and partly reenable futures crate #1037
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 4 commits
Commits
Show all changes
25 commits
Select commit
Hold shift + click to select a range
04842ae
Executors and partly reenable futures crate
MajorBreakfast 54d6b2f
Rename res to output
MajorBreakfast 8baad68
Rename Task to TaskContainer
MajorBreakfast 883ddff
Update to latest nightly
MajorBreakfast 95273d6
Split futures_unordered.rs
MajorBreakfast 52c0fe4
`Future + !Unpin` support for `FuturesUnordered`
MajorBreakfast cc6fa32
transmute type args
MajorBreakfast 146e818
Use TaskObj directly in LocalPool
MajorBreakfast ff85f87
Fix bug in stream iter doc
MajorBreakfast 251059d
Enable executors crate for future-util
MajorBreakfast 9c20c18
Remove Unpin restriction from LocalPool
MajorBreakfast 3fa5c6e
Lift Unpin restriction from Spawn
MajorBreakfast bec00f7
Move ThreadNotify into local_pool.rs
MajorBreakfast 2df964b
Prefix free function with module
MajorBreakfast 41853e7
Make some more doctests run successfully
MajorBreakfast ddbb39d
Make all enabled doc tests green
MajorBreakfast f0c8787
Additionally export `Poll` under `task` module (like in std)
MajorBreakfast d7cc312
Improve `Node` of `FuturesUnordered`
MajorBreakfast 5a3c6c1
Mostly updates to comments
MajorBreakfast ea55900
Use `cx.local_waker().wake()`
MajorBreakfast a8472bd
Bring back `SinkExt::Fanout" but keep its tests disabled for now
MajorBreakfast d4d5234
Rename `Inner` to `ReadyToRunQueue` for clarity
MajorBreakfast a2e07b0
Restrict `FuturesUnorderdered::iter_mut` to `Unpin` futures. Add `ite…
MajorBreakfast 7206d59
Add more `Unpin` bounds to `FuturesUnordered::iter_mut`
MajorBreakfast f235429
Implement `IterMut` in terms of `IterPinMut`
MajorBreakfast File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,6 +1,6 @@ | ||
| [package] | ||
| name = "futures-executor" | ||
| version = "0.2.0" | ||
| version = "0.3.0-alpha" | ||
| authors = ["Alex Crichton <[email protected]>"] | ||
| license = "MIT/Apache-2.0" | ||
| repository = "https://github.com/rust-lang-nursery/futures-rs" | ||
|
|
@@ -15,12 +15,12 @@ std = ["num_cpus", "futures-core/std", "futures-util/std", "futures-channel/std" | |
| default = ["std"] | ||
|
|
||
| [dependencies] | ||
| futures-core = { path = "../futures-core", version = "0.2.0", default-features = false} | ||
| futures-util = { path = "../futures-util", version = "0.2.0", default-features = false} | ||
| futures-channel = { path = "../futures-channel", version = "0.2.0", default-features = false} | ||
| futures-core = { path = "../futures-core", version = "0.3.0-alpha", default-features = false} | ||
| futures-util = { path = "../futures-util", version = "0.3.0-alpha", default-features = false} | ||
| futures-channel = { path = "../futures-channel", version = "0.3.0-alpha", default-features = false} | ||
| num_cpus = { version = "1.0", optional = true } | ||
| lazy_static = { version = "1.0", optional = true } | ||
|
|
||
| [dev-dependencies] | ||
| futures = { path = "../futures", version = "0.2.0" } | ||
| futures-channel = { path = "../futures-channel", version = "0.2.0" } | ||
| futures = { path = "../futures", version = "0.3.0-alpha" } | ||
| futures-channel = { path = "../futures-channel", version = "0.3.0-alpha" } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -2,21 +2,21 @@ use std::prelude::v1::*; | |
|
|
||
| use std::cell::{RefCell}; | ||
| use std::rc::{Rc, Weak}; | ||
| use std::mem::PinMut; | ||
| use std::marker::Unpin; | ||
|
|
||
| use futures_core::{Future, Poll, Async, Stream}; | ||
| use futures_core::task::{Context, Waker, LocalMap}; | ||
| use futures_core::executor::{Executor, SpawnError}; | ||
| use futures_core::never::Never; | ||
| use futures_core::{Future, CoreFutureExt, Poll, Stream}; | ||
| use futures_core::task::{Context, LocalWaker, TaskObj, local_waker_from_nonlocal}; | ||
| use futures_core::executor::{Executor, SpawnObjError, SpawnErrorKind}; | ||
| use futures_util::stream::FuturesUnordered; | ||
| use futures_util::stream::StreamExt; | ||
|
|
||
| use thread::ThreadNotify; | ||
| use enter; | ||
| use ThreadPool; | ||
|
|
||
| struct Task { | ||
| fut: Box<Future<Item = (), Error = Never>>, | ||
| map: LocalMap, | ||
| struct TaskContainer { | ||
| task: TaskObj, | ||
| } | ||
|
|
||
| /// A single-threaded task pool. | ||
|
|
@@ -31,7 +31,7 @@ struct Task { | |
| /// single-threaded, it supports a special form of task spawning for non-`Send` | ||
| /// futures, via [`spawn_local`](LocalExecutor::spawn_local). | ||
| pub struct LocalPool { | ||
| pool: FuturesUnordered<Task>, | ||
| pool: FuturesUnordered<TaskContainer>, | ||
| incoming: Rc<Incoming>, | ||
| } | ||
|
|
||
|
|
@@ -42,19 +42,19 @@ pub struct LocalExecutor { | |
| incoming: Weak<Incoming>, | ||
| } | ||
|
|
||
| type Incoming = RefCell<Vec<Task>>; | ||
| type Incoming = RefCell<Vec<TaskContainer>>; | ||
|
|
||
| // Set up and run a basic single-threaded executor loop, invocing `f` on each | ||
| // turn. | ||
| fn run_executor<T, F: FnMut(&Waker) -> Async<T>>(mut f: F) -> T { | ||
| fn run_executor<T, F: FnMut(&LocalWaker) -> Poll<T>>(mut f: F) -> T { | ||
| let _enter = enter() | ||
| .expect("cannot execute `LocalPool` executor from within \ | ||
| another executor"); | ||
|
|
||
| ThreadNotify::with_current(|thread| { | ||
| let waker = &Waker::from(thread.clone()); | ||
| let local_waker = local_waker_from_nonlocal(thread.clone()); | ||
| loop { | ||
| if let Async::Ready(t) = f(waker) { | ||
| if let Poll::Ready(t) = f(&local_waker) { | ||
| return t; | ||
| } | ||
| thread.park(); | ||
|
|
@@ -101,8 +101,8 @@ impl LocalPool { | |
| /// | ||
| /// The function will block the calling thread until *all* tasks in the pool | ||
| /// are complete, including any spawned while running existing tasks. | ||
| pub fn run(&mut self, exec: &mut Executor) { | ||
| run_executor(|waker| self.poll_pool(waker, exec)) | ||
| pub fn run<Exec>(&mut self, exec: &mut Exec) where Exec: Executor + Sized { | ||
| run_executor(|local_waker| self.poll_pool(local_waker, exec)) | ||
| } | ||
|
|
||
| /// Runs all the tasks in the pool until the given future completes. | ||
|
|
@@ -132,35 +132,31 @@ impl LocalPool { | |
| /// be inert after the call completes, but can continue with further use of | ||
| /// `run` or `run_until`. While the function is running, however, all tasks | ||
| /// in the pool will try to make progress. | ||
| pub fn run_until<F>(&mut self, mut f: F, exec: &mut Executor) -> Result<F::Item, F::Error> | ||
| where F: Future | ||
| pub fn run_until<F, Exec>(&mut self, mut f: F, exec: &mut Exec) -> F::Output | ||
| where F: Future + Unpin, Exec: Executor + Sized | ||
|
||
| { | ||
| // persistent state for the "main task" | ||
| let mut main_map = LocalMap::new(); | ||
|
|
||
| run_executor(|waker| { | ||
| run_executor(|local_waker| { | ||
| { | ||
| let mut main_cx = Context::new(&mut main_map, waker, exec); | ||
| let mut main_cx = Context::new(local_waker, exec); | ||
|
|
||
| // if our main task is done, so are we | ||
| match f.poll(&mut main_cx) { | ||
| Ok(Async::Ready(v)) => return Async::Ready(Ok(v)), | ||
| Err(err) => return Async::Ready(Err(err)), | ||
| match f.poll_unpin(&mut main_cx) { | ||
| Poll::Ready(output) => return Poll::Ready(output), | ||
| _ => {} | ||
| } | ||
| } | ||
|
|
||
| self.poll_pool(waker, exec); | ||
| Async::Pending | ||
| self.poll_pool(local_waker, exec); | ||
| Poll::Pending | ||
| }) | ||
| } | ||
|
|
||
| // Make maximal progress on the entire pool of spawned task, returning `Ready` | ||
| // if the pool is empty and `Pending` if no further progress can be made. | ||
| fn poll_pool(&mut self, waker: &Waker, exec: &mut Executor) -> Async<()> { | ||
| fn poll_pool<Exec>(&mut self, local_waker: &LocalWaker, exec: &mut Exec) -> Poll<()> | ||
| where Exec: Executor + Sized { | ||
| // state for the FuturesUnordered, which will never be used | ||
| let mut pool_map = LocalMap::new(); | ||
| let mut pool_cx = Context::new(&mut pool_map, waker, exec); | ||
| let mut pool_cx = Context::new(local_waker, exec); | ||
|
|
||
| loop { | ||
| // empty the incoming queue of newly-spawned tasks | ||
|
|
@@ -171,18 +167,17 @@ impl LocalPool { | |
| } | ||
| } | ||
|
|
||
| if let Ok(ret) = self.pool.poll_next(&mut pool_cx) { | ||
| // we queued up some new tasks; add them and poll again | ||
| if !self.incoming.borrow().is_empty() { | ||
| continue; | ||
| } | ||
| let ret = self.pool.poll_next_unpin(&mut pool_cx); | ||
| // we queued up some new tasks; add them and poll again | ||
| if !self.incoming.borrow().is_empty() { | ||
| continue; | ||
| } | ||
|
|
||
| // no queued tasks; we may be done | ||
| match ret { | ||
| Async::Pending => return Async::Pending, | ||
| Async::Ready(None) => return Async::Ready(()), | ||
| _ => {} | ||
| } | ||
| // no queued tasks; we may be done | ||
| match ret { | ||
| Poll::Pending => return Poll::Pending, | ||
| Poll::Ready(None) => return Poll::Ready(()), | ||
| _ => {} | ||
| } | ||
| } | ||
| } | ||
|
|
@@ -202,14 +197,14 @@ lazy_static! { | |
| /// | ||
| /// Use a [`LocalPool`](LocalPool) if you need finer-grained control over | ||
| /// spawned tasks. | ||
| pub fn block_on<F: Future>(f: F) -> Result<F::Item, F::Error> { | ||
| pub fn block_on<F: Future + Unpin>(f: F) -> F::Output { | ||
|
||
| let mut pool = LocalPool::new(); | ||
| pool.run_until(f, &mut GLOBAL_POOL.clone()) | ||
| } | ||
|
|
||
| /// Turn a stream into a blocking iterator. | ||
| /// | ||
| /// Whne `next` is called on the resulting `BlockingStream`, the caller | ||
| /// When `next` is called on the resulting `BlockingStream`, the caller | ||
| /// will be blocked until the next element of the `Stream` becomes available. | ||
| /// The default executor for the future is a global `ThreadPool`. | ||
| pub fn block_on_stream<S: Stream>(s: S) -> BlockingStream<S> { | ||
|
|
@@ -226,62 +221,55 @@ impl<S: Stream> BlockingStream<S> { | |
| } | ||
| } | ||
|
|
||
| impl<S: Stream> Iterator for BlockingStream<S> { | ||
| type Item = Result<S::Item, S::Error>; | ||
| impl<S: Stream + Unpin> Iterator for BlockingStream<S> { | ||
| type Item = S::Item; | ||
| fn next(&mut self) -> Option<Self::Item> { | ||
| let s = self.stream.take().expect("BlockingStream shouldn't be empty"); | ||
| let (item, s) = | ||
| match LocalPool::new().run_until(s.next(), &mut GLOBAL_POOL.clone()) { | ||
| Ok((Some(item), s)) => (Some(Ok(item)), s), | ||
| Ok((None, s)) => (None, s), | ||
| Err((e, s)) => (Some(Err(e)), s), | ||
| }; | ||
| LocalPool::new().run_until(s.next(), &mut GLOBAL_POOL.clone()); | ||
|
|
||
| self.stream = Some(s); | ||
| item | ||
| } | ||
| } | ||
|
|
||
| impl Executor for LocalExecutor { | ||
| fn spawn(&mut self, f: Box<Future<Item = (), Error = Never> + Send>) -> Result<(), SpawnError> { | ||
| self.spawn_task(Task { | ||
| fut: f, | ||
| map: LocalMap::new(), | ||
| }) | ||
| fn spawn_obj(&mut self, task: TaskObj) -> Result<(), SpawnObjError> { | ||
| if let Some(incoming) = self.incoming.upgrade() { | ||
| incoming.borrow_mut().push(TaskContainer { task }); | ||
| Ok(()) | ||
| } else { | ||
| Err(SpawnObjError{ task, kind: SpawnErrorKind::shutdown() }) | ||
| } | ||
| } | ||
|
|
||
| fn status(&self) -> Result<(), SpawnError> { | ||
| fn status(&self) -> Result<(), SpawnErrorKind> { | ||
| if self.incoming.upgrade().is_some() { | ||
| Ok(()) | ||
| } else { | ||
| Err(SpawnError::shutdown()) | ||
| Err(SpawnErrorKind::shutdown()) | ||
| } | ||
| } | ||
| } | ||
|
|
||
| impl LocalExecutor { | ||
| fn spawn_task(&self, task: Task) -> Result<(), SpawnError> { | ||
| let incoming = self.incoming.upgrade().ok_or(SpawnError::shutdown())?; | ||
| incoming.borrow_mut().push(task); | ||
| Ok(()) | ||
| } | ||
|
|
||
| /* | ||
| /// Spawn a non-`Send` future onto the associated [`LocalPool`](LocalPool). | ||
| pub fn spawn_local<F>(&mut self, f: F) -> Result<(), SpawnError> | ||
| pub fn spawn_local<F>(&mut self, f: F) -> Result<(), SpawnObjError> | ||
| where F: Future<Item = (), Error = Never> + 'static | ||
| { | ||
| self.spawn_task(Task { | ||
| fut: Box::new(f), | ||
| map: LocalMap::new(), | ||
| }) | ||
| } | ||
| */ | ||
| } | ||
|
|
||
| impl Future for Task { | ||
| type Item = (); | ||
| type Error = Never; | ||
| impl Future for TaskContainer { | ||
| type Output = (); | ||
|
|
||
| fn poll(&mut self, cx: &mut Context) -> Poll<(), Never> { | ||
| self.fut.poll(&mut cx.with_locals(&mut self.map)) | ||
| fn poll(mut self: PinMut<Self>, cx: &mut Context) -> Poll<()> { | ||
| self.task.poll_unpin(cx) | ||
| } | ||
| } | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you add a comment to this that describes what it's for? Something like "wrapper for TaskObj that implements
Futurewith a unit output" would help-- it's less obvious what its purpose is now that it no longer pairs the task with a LocalMap.N.B. I think that TaskObj should implement
Future-- we should add that impl in std. Opened rust-lang/rust#51667There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Already made TaskObj implement Future
I‘ll add the comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm getting rid of TaskContainer entirely. We can use
TaskObjdirectly. Can't do that inThreadPool, but here it's simple