The biggest unresolved question regarding the async/await syntax is the final syntax for the await operator. There’s been an enormous amount of discussion on this question so far; a summary of the present status of that discussion and the positions within the language team is coming soon. Right now I want to separately focus on one question which impacts that decision but hasn’t been considered very much yet: for loops which process streams.

A feature of other languages - like JavaScript for example - and of the macro-based futures-async-await prototype is a syntax for writing for loops that operate over streams instead of iterators. It’s very likely we’ll want to add this feature to Rust someday, and its worthwhile to think a bit about the syntax for it so that we don’t design ourselves into a corner.

The futures-async-await library uses this syntax for the feature:

#[async]
for elem in stream { ... }

Framing this as an “async for loop” and making the syntax something like async for elem in stream makes some intuitive sense, but I think its actually a mistake when you investigate further. async modifies things to make them create a future of something instead of evaluating to that thing directly - this is not what this syntax does. What this does is yield from the surrounding async item when the next item in the stream is not ready yet - exactly the same thing as what the await operator does to futures already. This is why JavaScript went with this syntax, which uses await:

for await (elem of stream) { ... }

(In JavaScript, the equivalent of for elem in stream is for (elem of stream).

Proceding from the basis that we this syntax would use the await keyword, we should think about what our options for the syntax here are, and how they feed back into the decision for the initial await expression operator.

Is the await part of the pattern or the loop syntax?

One of the fundamental decisions we need to make in implementing this syntax is this: is the await operator a part of the pattern component of the syntax, or the for loop operation. If its a part of the pattern, then it would be an irrefutable pattern that would match any future and await it, and could be used in other positions (like the branches of a match statement). If its part of the loop, then its just a modified form of the for loop syntax, that takes both a pattern and an expression.

Making it part of the pattern is somewhat appealing: its more generally useful and makes sense as a feature to have. It also goes hand-in-hand with another appealing feature: allowing the ? operator in patterns to unwrap them irrefutably. This is desirable because this is already a common pattern when dealing with IO iterators, like iterating over the lines in a file:

for line in file.lines() {
    let lines = lines?;
    // ...
}

However, making await a part of the pattern has a big problem: for loops take an iterator, and a stream is not equivalent to an iterator of futures (that is, you could not implement Iterator for S where S: Stream) because the futures returned by a stream are not wholy independent of one another - the Next future that Stream::next returns borrows mutably from the underlying stream until it resolves. In other words, Streams are more analogous to the hypothetical trait StreamingIterator (which can’t be expressed in Rust today) rather than to the Iterators.

I’ll explore a way we could modify for loops to take streams and make the await part of the pattern in the next post, but for the rest of this post let’s assume we will make await a part of the for loop syntax, and what the implications of that are.

Implications of for await constructs

Based on precedent from other languages, the most obvious syntax for this feature would be for await elem in stream. However, if the syntax we use for the await expression syntax is not await future (as it is in other languages), this introduces quite a bit of inconsistency between the two positions in which the await keyword appears.

On the other hand, the majority of the postfix await syntaxes visually conjoin the await operator with its operand - that is, foo.await (for example) is designed so that foo and .await are not intended to be whitespace separated. It seems confusing and unexpected to me that, using a syntax like for elem.await in stream, that .await is not a part of the pattern that the loop takes, but is instead part of the for loop.

It also seems especially challenging for syntaxes which mimic other expression types. For example, some users have argued for foo.await() so that the operator looks like a method and can be thought of like one (though it isn’t a method at all). I think it would be problematic and very inconsistent to allow for elem.await() but not allow “other” methods in this position.

I think this favors space-separated syntaxes, either of these, because they clearly separate the operator from the pattern:

for await elem in stream { }
for elem await in stream { }

However, there is one interesting quirk to consider for both of these syntaxes. The most popular prefix argument favors adding an await? syntax sugar which combines the await operator with the ? operator. It would be most consistent for that to work here as well:

for await? elem in stream { }

As I mentioned earlier, we’ve also long considered adding ? to patterns. Because of the inherent order of operations between the for loop syntax and the pattern syntax, this would make these two syntaxes equivalent:

// Apply ? to the stream item as a part of the for await loop syntax
for await? elem in stream { }

// Run the for await loop syntax, then apply ? to the stream item
// as a part of the pattern
for await elem? in stream { }

The reason that the await? syntactic sugar is being proposed is that the order of operations of await elem? in the expression context is not of this order. Having both orders work the same way in loops introduces its own small inconsistency between for await loops and await expressions.

For postfix-space, it introduces a similar but different problem. In particular, postfix-space does not propose to add a special-case await? combined operation, so it does not follow that for elem await? would work. This means that if we ever add ? in patterns, the behavior of for elem? await would be to apply await before ? (what users probably want), even though visually they are in the opposite order. This seems quite troubling.

Conclusions

I think that if we make await part of the loop syntax, the choice that makes the most sense to me is the prefix-await syntax. However, even that syntax introduces a surprising little inconsistency where the order of operations in await elem? in a for loop header is different from the order of operations in await future? in an expression context. So none of these syntaxes really works perfectly.

That’s why in the next post I’ll look at the changes we’d need to allow making await in patterns integrate well with for loops, and also at how that would impact things.