Futures Concurrency IV: Join Ergonomics
— 2022-09-19

1. async let in swift
2. join in rust
3. join ergonomics
4. other operations
5. fallibility
6. conclusion

On Thursday this week Rust 1.64 will be released, and in it it will include a stabilized version of IntoFuture. Much like IntoIterator is used in the desugaring of for..in loops, IntoFuture will be used in the desugaring of .await.

In this post I want to show some of the ergonomics improvements IntoFuture might enable, inspired by Swift's recent improvements in async/await ergonomics.

async let in swift

Swift has recently added support for async let ¹ in the language. This provides lightweight syntax to create Swift tasks, which before this proposal always had to be constructed explicitly within a task group. In the evolution proposal they show an example like this:

func makeDinner() async -> Meal {
    // Create a task group to scope the lifetime of our three child tasks
    return try await withThrowingTaskGroup(of: CookingTask.self) { group in
    // spawn three cooking tasks and execute them in parallel:
    group.async {
        CookingTask.veggies(try await chopVegetables())
    }
    group.async {
        CookingTask.meat(await marinateMeat())
    }
    group.async {
        CookingTask.oven(await preheatOven(temperature: 350))
    }
    // ... a whole lot more code after this
}

That's quite a bit of code. And async let exists to simplify that. Instead of explicitly creating the task group and manually creating tasks within it, using async let the right scope is inferred for us ², and instead the concurrency is expressed at the await point later on:

func makeDinner() async throws -> Meal {
  async let veggies = chopVegetables()
  async let meat = marinateMeat()
  async let oven = preheatOven(temperature: 350)

  let dish = Dish(ingredients: await [try veggies, meat]) // notice the concurrent await here
  return try await oven.cook(dish, duration: .hours(3))
}

This feels very elegant in comparison, and definitely feels like a step up for Swift. I think the flow of this is so good in fact, that we may want to adopt something similar in Rust.

Join in Rust

In Rust we already have a notion of a "concurrent await" through the join operation. If you do async Rust you've likely seen it before in macro-form as join! or the join_all free-function:

// example of the `join!` macro:
let a = future::ready(1u8);
let b = future::ready("hello");
let c = future::ready(3u16);
assert_eq!(join!(a, b, c).await, (1, "hello", 3));

// example of the `join_all` free-function:
let futs = vec![ready(1u8), foo(2u8), foo(3u8)];
assert_eq!(join_all(futs).await, [1, 2, 3]);

In my Futures Concurrency II ³ post and library I show that instead of using a combination of macros and free functions we can instead use traits to extend container types with the necessary concurrency operations. This results in a smoother experience overall, since arrays, tuples, and vectors all operate in the same way:

// alternative to the `join!` macro:
let a = future::ready(1u8);
let b = future::ready("hello");
let c = future::ready(3u16);
assert_eq!((a, b, c).join().await, (1, "hello", 3));

// alternative to the `join_all` free-function:
let a = future::ready(1);
let b = future::ready(2);
let c = future::ready(3);
assert_eq!(vec![a, b, c].join().await, vec![1, 2, 3]);

This works fine, but it's not quite as good as what swift has using async let. Let's take a look at how we can improve that.

Join Ergonomics

Awaiting a single future is done by calling .await, but awaiting multiple futures requires calling an intermediate method. To get join semantics you have to call the join method, rather than just being able to await a tuple of futures.

This is where IntoFuture can help: we can use it to implement a conversion to a future directly on tuples, arrays, and vectors - and make it so if they contain futures you can just call .await on them directly to await all futures concurrently. With that in place the above example could be rewritten like this:

let a = future::ready(1u8);
let b = future::ready("hello");
let c = future::ready(3u16);
assert_eq!((a, b, c).await, (1, "hello", 3));    // no more `.join()` needed

let a = future::ready(1);
let b = future::ready(2);
let c = future::ready(3);
assert_eq!(vec![a, b, c].await, vec![1, 2, 3]);  // no more `.join()` needed

While being substantially different under the hood, the surface-level experience of this is actually quite similar to Swift's async let. We just don't operate by default on tasks owned by a runtime, but futures which are lazy and compile down to in-line state machines. But this could be made to trivially work with multi-threaded Rust tasks too, if we follow the spawn approach laid out by tasky (blog post ⁴) ⁵:

let a = future::ready(1u8).spawn();
let b = future::ready("hello").spawn();
let c = future::ready(3u16).spawn();
assert_eq!((a, b, c).await, (1, "hello", 3));    // parallelized `await`

let a = future::ready(1).spawn();
let b = future::ready(2).spawn();
let c = future::ready(3).spawn();
assert_eq!(vec![a, b, c].await, vec![1, 2, 3]);  // parallelized `await`

This, to me, feels like a pretty good outcome for concurrent and parallel awaiting of fixed-sized sets of futures. Awaiting sets of futures which change over time is a different problem, and will likely require something akin to Swift's TaskSet to function. But that's not most concurrent workloads, and I think we can take a page out of Swift's book on this.

Other Operations

In a past post ⁶ I've shown the following table of concurrency operations for futures:

	Wait for all outputs	Wait for first output
Continue on error	Future::join	Future::try_race
Return early on error	Future::try_join	Future::race

join is only one of 4 different concurrency operations. When you have different futures you may in fact want to do different things with it. But not all operations are equal: when we have two or more futures, it's usually because we're interested in observing all of their output. join gives us exactly that. We'll get to try_ variants in the next section, but for now take for the following example:

let a = future::ready(1);
let b = future::ready(2);
let a = a.await;
let b = b.await;

This first awaits a, and then it awaits b. Because the two are unrelated, it's often more efficient to await them concurrently rather than sequentially. With (async) IntoIterator we're handed tools required to operate over values sequentially. But with IntoFuture we're handed tools to operate over values concurrently ⁷:

// `IntoIterator` enables sequential operation
let a = 1;
let b = 2;
for num in [a, b] { .. }   // sequential iteration

// `IntoFuture` enables concurrent operation
let a = future::ready(1);
let b = future::ready(2);
let [a, b] = [a, b].await; // concurrent await

The design of IntoFuture intentionally mirrors the design of IntoIterator. For any type T we can only have a single IntoIterator implementation. This leaves us with a few options:

1. don't implement IntoFuture for containers
2. implement join semantics for containers
3. implement race semantics for containers

This post is argues that 2. is a chance to provide better ergonomics than 1.. But what about 3.? Could we meaningfully have race semantics as the default? race takes N futures and yields one result. This would look like this:

let a = future::ready(1u8);
let b = future::ready(2u8);
let c: u8 = [a, b].await; // `race`-semantics

I've said it before, but this seems like a minority use case. We'll almost always want a AND b. Not a XOR b. At least: we may still want to drop the values later on, but it's usually not decided based on which completed first. join also feels like it extends the existing behavior of await for individual futures to sets of futures. race exposes different semantics than a plain .await does, and so if awaiting tuples of futures was different, the resulting experience would feel inconsistent.

It seems good to have join be the default semantics exposed through IntoFuture, but then have operations such as race and merge be explicit forms of concurrency you can instead opt into by calling the right method.

Fallibility

There is one other issue with the proposal: we have no way to expose fallbility of try_join semantics. try_join allows short-circuiting a join operation if any of the fallible futures returns a Result::Error. Right now we don't have a way to paramaterize the await over fallibility, and since we only have a single IntoFuture implementation we're stuck with just join, which is not ideal. Perhaps using keyword generics ⁸ we can find a way out of this. After all: iterators face many of the same challenges today, since we can't just call for ?..in in loops. With keyword generics we might be able to choose try_join semantics for constructs such as:

let a = future::ready(Ok(1));
let b = future::ready(Ok(2));
let (a, b) = (a, b).await?; // try_join await semantics

This would infer that the want the fallible veriant of IntoFuture since we call ? on it after, and we're in a fallible context. But it's early on and hard to say precisely how this would work. It seems encouraging though that wanting fallible semantics for a construct is a shared problem across most of the language, so there is a desire to solve it in a unified way.

Conclusion

In this post I've shown how Swift's async let allows concurrent awaiting, how we can implement it in Rust through IntoFuture, and what some of the challenges with it might be. I think Swift has done a great job at showing how convenient it is to be able to perform lightweight concurrent awaits, and I think we can learn from their approach and make it our own.

Before starting this post I didn't think too deeply about the similarities between IntoIterator and IntoFuture. But the more I look at them, the closer they seem. The fact that we have IntoIterator for most every container might give us a hint for how we may want to think about container types. I'm not saying we should go out and implement IntoFuture for HashMap - but for vec, array, and tuples it definitely seems to make sense.

It seems that in terms of ordering we aren't too far out from being able to write an RFC for all of this either. I'm feeling pretty confident we've got the basic subset done for fixed-length async concurrency operators. And with IntoFuture exposing join semantics for arrays, tuples, and vectors, we can make the main method of concurrency easy to use as well. We'll have to see exactly what we want to do about tuples wrt the design - since they can't yet be variadic. And we'll want to wait on async traits to land first. But after 3 years of writing and experimentation on this topic, we don't seem far out ⁹ from finally being able to propose something concrete for inclusion in the stdlib. And I think that's pretty exciting!