SourceT improvements

The core SourceT<M> has improvements:

• The | operator which takes the left-hand stream and right-hand stream and merges them into a single stream can now handle errors emitted from either stream and propagate it up through the reducer. Your M type must support Fallible<M> for this to work.
• I have rewritten the internals of StreamT.lift(Channel) to use the new Monad.Recur functionality - this removes any risk of stack-overflows with your bespoke M types (assuming you've implemented Monad.Recur correctly.

StreamT non-termination issue resolved

This question was raised where a StreamT fold was not terminating. The issue was slightly deeper than I initially thought. SourceT took a different approach to terminating a reducer than Source. SourceT used M.Pure as a terminating value whereas Source has a dedicated structure, Reduced, which has a Continue flag, indicating whether any reducer is complete or not.

I thought M.Pure was enough to terminate a stream, but it wasn't sufficient, I made a mistake on that one. So, I've updated all of the SourceT reducers to now return K<M, Reduced<A>> which allows streams like SourceT.forever to spot when a 'downstream' reducer has ended the stream and can therefore terminate, rather than continue forever heating up the processor.

StreamT is still completely without any form of decent test coverage, so still be cautious using it.

There are now four reducing methods on StreamT which allow for different return values in the reducer function:

• S - just the pure reduced value
• K<M, A> - the lifted reduced value
• Reduced<S> - the pure reduced value with the Continue flag (in case you want to use this in other reducers)
• K<M ,Reduced<S>> - the lifted reduced value with the Continue flag (in case you want to use this in other reducers)

public K<M, S> Reduce<S>(S state, Reducer<A, S> reducer) =>
    ReduceM(state, (s, x) => M.Pure(reducer(s, x)));

public K<M, S> FoldReduce<S>(S state, Func<S, A, S> reducer) =>
    ReduceM(state, (s, x) => M.Pure(Reduced.Continue(reducer(s, x))));

public K<M, S> ReduceM<S>(S state, ReducerM<M, A, S> reducer) =>
    ReduceInternalM(state, (s, mx) => mx.Bind(x => reducer(s, x))).Map(r => r.Value);

public K<M, S> FoldReduceM<S>(S state, Func<S, A, K<M, S>> reducer) =>
    ReduceInternalM(state, (s, mx) => mx.Bind(x => reducer(s, x).Map(Reduced.Continue))).Map(r => r.Value);

Support functions added for Monad.Recur

The support for generalised monad tail-recursion release that I did in the early hours of Christmas Day needed some support functions, as also requested here, so I have added the following functions to the Monad module:

Function	Behaviour
Monad.forever	Repeatedly runs a monad's effect
Monad.replicate	Runs a monad's effect n times, collecting the results in an Iterable<A>
Monad.accumWhile	Repeatedly runs a monad's effect while the predicate returns true, collecting the results in an Iterable<A>
Monad.accumWhileM	Repeatedly runs a monad's effect while the monadic predicate returns true, collecting the results in
Monad.accumUntil	Repeatedly runs a monad's effect until the predicate returns true, collecting the results in an Iterable<A>
Monad.accumUntilM	Repeatedly runs a monad's effect until the monadic predicate returns true, collecting the results in an Iterable<A>

The functions themselves are quite good demonstrations of Monad.recur, so I have pasted the source-code below just FYI:

/// <summary>
/// This is equivalent to the infinite loop below without the stack-overflow issues:
///
///     K〈M, A〉go =>
///         ma.Bind(_ => go);
/// 
/// </summary>
/// <param name="ma">Computation to recursively run</param>
/// <typeparam name="M">Monad</typeparam>
/// <typeparam name="A">Bound value type</typeparam>
/// <typeparam name="B">'Result' type, there will never be a result of `B`, but the monad rules may exit the loop
/// with an alternative value; and so `B` is still valid</typeparam>
/// <returns>A looped computation</returns>
[Pure]
public static K<M, A> forever<M, A>(K<M, A> ma)
    where M : Monad<M> =>
    forever<M, A, A>(ma);

/// <summary>
/// This is equivalent to the infinite loop below without the stack-overflow issues:
///
///     K〈M, A〉go =>
///         ma.Bind(_ => go);
/// 
/// </summary>
/// <param name="ma">Monadic computation to recursively run</param>
/// <typeparam name="M">Monad</typeparam>
/// <typeparam name="A">Bound value type</typeparam>
/// <typeparam name="B">'Result' type, there will never be a result of `B`, but the monad rules may exit the loop
/// with an alternative value; and so `B` is still valid</typeparam>
/// <returns>A looped computation</returns>
[Pure]
public static K<M, B> forever<M, A, B>(K<M, A> ma) 
    where M : Monad<M> => 
    recur<M, A, B>(default!, _ => ma.Map(_ => Next<A, B>.UnsafeDefault));

/// <summary>
/// Running the monadic computation `ma` a fixed number of times (`count`) collecting the results
/// </summary>
/// <param name="ma">Monadic computation to repeatedly run</param>
/// <param name="count">Number of times to replicate monadic computation</param>
/// <typeparam name="M">Monad</typeparam>
/// <typeparam name="A">Bound value type</typeparam>
/// <returns>A lifted iterable of values collected</returns>
[Pure]
public static K<M, Iterable<A>> replicate<M, A>(K<M, A> ma, int count)
    where M : Monad<M> =>
    recur<M, (Iterable<A> Items, int Remain), Iterable<A>>(
        ([], count),
        acc => ma * (a => acc.Remain > 0 
                      ? Next.Loop<(Iterable<A> Items, int Remain), Iterable<A>>((acc.Items.Add(a), acc.Remain - 1))
                      : Next.Done<(Iterable<A> Items, int Remain), Iterable<A>>(acc.Items.Add(a))));

/// <summary>
/// Keep running the monadic computation `ma` collecting the result values until a result value
/// yielded triggers a `true` value when passed to the `f` predicate
/// </summary>
/// <param name="ma">Monadic computation to repeatedly run</param>
/// <param name="f">Predicate</param>
/// <typeparam name="M">Monad</typeparam>
/// <typeparam name="A">Bound value type</typeparam>
/// <returns>A lifted iterable of values collected</returns>
[Pure]
public static K<M, Iterable<A>> accumUntil<M, A>(K<M, A> ma, Func<A, bool> f)
    where M : Monad<M> =>
    recur<M, Iterable<A>, Iterable<A>>(
        [],
        acc => ma * (a => f(a) ? Next.Done<Iterable<A>, Iterable<A>>(acc)
                               : Next.Loop<Iterable<A>, Iterable<A>>(acc.Add(a))));

/// <summary>
/// Keep running the monadic computation `ma` collecting the result values until a result value
/// yielded triggers a `true` value when passed to the `f` predicate
/// </summary>
/// <param name="ma">Monadic computation to repeatedly run</param>
/// <param name="f">Predicate</param>
/// <typeparam name="M">Monad</typeparam>
/// <typeparam name="A">Bound value type</typeparam>
/// <returns>A lifted iterable of values collected</returns>
[Pure]
public static K<M, Iterable<A>> accumUntilM<M, A>(K<M, A> ma, Func<A, K<M, bool>> f)
    where M : Monad<M> =>
    recur<M, Iterable<A>, Iterable<A>>(
        [],
        acc => ma >> (a => f(a) * (r => r ? Next.Done<Iterable<A>, Iterable<A>>(acc)
                                          : Next.Loop<Iterable<A>, Iterable<A>>(acc.Add(a)))));

/// <summary>
/// Keep running the monadic computation `ma` collecting the result values until a result value
/// yielded triggers a `true` value when passed to the `f` predicate
/// </summary>
/// <param name="ma">Monadic computation to repeatedly run</param>
/// <param name="f">Predicate</param>
/// <typeparam name="M">Monad</typeparam>
/// <typeparam name="A">Bound value type</typeparam>
/// <returns>A lifted iterable of values collected</returns>
[Pure]
public static K<M, Iterable<A>> accumWhile<M, A>(K<M, A> ma, Func<A, bool> f)
    where M : Monad<M> =>
    recur<M, Iterable<A>, Iterable<A>>(
        [],
        acc => ma * (a => f(a) ? Next.Loop<Iterable<A>, Iterable<A>>(acc.Add(a))
                               : Next.Done<Iterable<A>, Iterable<A>>(acc)));

/// <summary>
/// Keep running the monadic computation `ma` collecting the result values until a result value
/// yielded triggers a `true` value when passed to the `f` predicate
/// </summary>
/// <param name="ma">Monadic computation to repeatedly run</param>
/// <param name="f">Predicate</param>
/// <typeparam name="M">Monad</typeparam>
/// <typeparam name="A">Bound value type</typeparam>
/// <returns>A lifted iterable of values collected</returns>
[Pure]
public static K<M, Iterable<A>> accumWhileM<M, A>(K<M, A> ma, Func<A, K<M, bool>> f)
    where M : Monad<M> =>
    recur<M, Iterable<A>, Iterable<A>>(
        [],
        acc => ma >> (a => f(a) * (r => r ? Next.Loop<Iterable<A>, Iterable<A>>(acc.Add(a))
                                          : Next.Done<Iterable<A>, Iterable<A>>(acc))));

Fix for SourceT Skip and Take issues

Fix for:

• SourceT Skip and Take issues: #1503
• Difference between the behaviours of FoldWhile and FoldWhileIO: #1527

A simple update to transducer-lifting in SourceT which fixes issues seen with Skip, Take, and Fold*

Support for generalised monad tail-recursion

Quite a big release here. It's not had a huge amount of testing, but thought it would be good to get it out before I disappear for Christmas, in case anyone wants to have a play with the new features.

Monad.Recur

As suggested here and at various times over the years, we could support the same tailRecM pattern as seen in Cats and Scalaz libraries from the Scala ecosystem. I have been a little sceptical of the approach, mostly because it forces the user to manually trampoline. I wasn't keen on going all in, but then I woke up Saturday morning thinking about it (that's how sad my life is) and felt inspired to implement it.

150+ file changes later and we now have support. tailRecM in Scala becomes Monad.recur in language-ext. Every monad is expected to implement the Monad.Recur trait method, but you could also use the placeholder function (Monad.unsafeRecur) that still uses regular recursion. This is because:

• Sometimes it's not important to implement it right now
• It's not always trivial to implement and you may struggle to do so for certain monadic-types
• Some monads may already have their own trampoline built-in (like IO<A>), so using regular recursion is fine

There's also two other helper functions: Monad.iterableRecur and Monad.enumerableRecur which are default Monad.Recur implementations for collections.

You're unlikely to need recursion for collections anyway, because it'd be like the most insanely nested set of for-loops ever. But, technically you could have lots of singleton lists that blow a stack in a recursion scenario, so these helper functions save you the hassle of working out how to unfold a recursive bind on collections.

Here's the implementation of Monad.Recur for Option:

static K<Option, B> Monad<Option>.Recur<A, B>(A value, Func<A, K<Option, Next<A, B>>> f)
{
    while (true)
    {
        var mr = +f(value);
        if (mr.IsNone) return Option<B>.None;
        var mnext = (Next<A, B>)mr;
        if(mnext.IsDone) return Some(mnext.Done);
        value = mnext.Loop;
    }
}

If you're wondering what the prefix + operator is, it's the downcast-operator from K<Option, A> to Option<A>. Like calling f(value).As(), just more elegant.

What's really nice about this implementation is how 'light' it is. It's a simple while-loop with a test to see if we're done: if(mnext.IsDone) or a continuation-value (mnext.Loop) to be passed to the 'bind' function f. As well as the standard Option monad-bind logic of earlying-out when in a None state.

There were two reasons I decided to ignore my earlier concerns about this approach and this is one of them: Option is a struct and is designed to be a lightweight data-type. The only other way to support recursion would have been to make it into an interpreter style monad (like the IO monad) which would bring more overhead where we don't want it. So, this manual opt-in to recursion when you need it and the subsequent tight while-loop makes recursion with simple data-monads very effetive.

It's certainly not very elegant in C#, I would mostly advise using maps, folds, and traversals (where possible). But there's always that one time where recursion would be better, so now you can guarantee its safety.

I still want to build support for tail-recursion into the 'control' monads (those that are lazy computations and therefore can be backed by an interpreted DSL). I still believe that's more effective and elegant for the end-user.

Iterable now supports IAsyncEnumerable

So Iterable which started life as a wrapper for IEnumerable now supports both synchronous and asynchronous lazy streams. This is continuing the goal of killing Task and async/await. Only when you use an Iterable do you care how you consume it. If you want asynchronicity then you can either call AsAsyncEnumerable() or use any of the *IO suffixed methods (like CountIO, FoldIO, etc.)

IterableNE is a new non-empty Iterable

Every wanted to work with a lazy stream that must have at least one item in it? Well, now you can: IterableNE. It's impossible for an IterableNE to be constructed without at least one item in it, so you can assume it will always have a Head value.

It also supports synchronous and asynchronous sequences.

Applicative.Actions in the Applicative trait updated

There were previously two methods:

Applicative.Actions(IEnumerable)
Applicative.Actions(IAsyncEnumerable)

Now there's one

Applicative.Actions(IterableNE);

Actions would previously throw exceptions with empty sequences.

Monad bind operator

Previous change to change the monad bind operator from >> to >>>` have been reverted. It just didn't 'feel' right, so we're back to where we were.

Lots of other minor tweaks

There are lots of other minor tweaks, but it's now 1am on Christmas day, so I'm going to stop typing! Happy (belated) Hanukkah, Merry Christmas, and/or Happy Holidays to you all.

Bug fixing release

Fixes for:

• Resources disposition not working? #1521
• Arr.GetHashCode fails for default instances #1522

IO improvements

awaitAny improved

The awaitAny function - that works with any MonadUnliftIO trait-implementing type (IO, Eff, etc.), accepts a number of asynchronous computations, and returns when the first computation completes (a functional version of Task.WhenAny) - will now cancel all computations once the first one has completed. This is probably the most desirable default functionality, rather than letting all computations continue until they're complete.

This example will only write out "C", the A and B processes will be cancelled once the C process completes:

var eff = awaitAny(delayed("A", 9),
                   delayed("B", 7),
                   delayed("C", 5));

ignore(eff.Run(env));
Console.ReadKey();

static Eff<Unit> delayed(string info, int time) =>
    Eff.lift(async e =>
             {
                 await Task.Delay(time * 1000, e.Token);
                 Console.WriteLine(info);
                 return unit;
             });

UninterruptibleIO

The MonadUnliftIO trait now has a new method: UninterruptibleIO:

public static virtual K<M, A> UninterruptibleIO<A>(K<M, A> ma) =>
    M.MapIO(ma, io => io.Uninterruptible());

The default behaviour is to block cancellation-requests that are raised via the EnvIO cancellation-token. It does this by creating a new local-EnvIO context that ignores the parent's cancellation context.

There's an UninterruptibleIO extension method and an uninterruptible function in the Prelude that will work with any MonadUnliftIO trait-implementing type (IO, Eff, etc.).

If you wished to go back to the old awaitAny behaviour then you can wrap each computation with uninterruptible(...). If we do that with the previous example:

var eff = awaitAny(uninterruptible(delayed("A", 9)),
                   uninterruptible(delayed("B", 7)),
                   uninterruptible(delayed("C", 5)));

Then the output will be:

C
B
A

Which was the previous default functionality.

Timeout improvements

• Added timeout support to EnvIO - a TimeSpan can be provided for when CancellationTokenSource is null and it will then use that timeout value when constructing a new one.
• There's also EnvIO.LocalWithTimeout(TimeSpan) and EnvIO.LocalCancelWithTimeout(TimeSpan), which are the most useful functions as they allow for a local context with a timeout attached.
• The IO<A>.Timeout, IO.timeout, and Prelude.timeout functions have been improved to leverage this new functionality. The previous implementation was quite naive and not the most efficient, so this needed doing.

Effects prelude

I've moved the more general (K<M, A>) based IO operators/functions to the root of the Effects folder to show they're general and not just for the IO type. It makes functions slightly more discoverable in the API documentation.

Minor ambiguous method resolution fixed

#1513

Minor bug fix for Case on `Option<T>`

To fix this isssue: #1512

Extension operators supported for more types

Following on from the extension-operators and .NET 10 release yesterday, I have now implemented operators for even more of the core types in language-ext.

The full set so far:

• ChronicleT<Ch, M, A>
• Eff<A>
• Eff<RT, A>
• Either<L, R>
• EitherT<L, M, R>
• Fin<A>
• FinT<M, A>
• IO<A>
• Option<A>
• OptionT<M, A>
• These<A, B>
• Try<A>
• TryT<M, A>
• Validation<F, A>
• ValidationT<F, M, A>

There is no need to do any of these to make the extension operators work for the types, but the generic extensions will all return an abstract K<F, A> type. So, I am in the process of making these bespoke versions return a concrete type. This is purely for usability's sake.

And because each core type can support many different traits, the number of operators they support can be quite large too (see the Try operators for a good example!). So, for each core type I've decided on a new folder structure:

TYPE folder
  | ----- TYPE definition.cs
  | ----- TYPE module.cs
  | ----- TYPE case [1 .. n].cs  (Left, Right, Some, None, etc.)
  | ----- Extensions
  | ----- Operators
  | ----- Prelude

This will keep the many method extensions and operator extensions away from the core functionality for any one type: which hopefully will make the API docs easier to read and the source-code easier to navigate.

Let me know if you see any quirkiness with the new operators. This is a lot of typing, so it would be good to catch issues early!