Question: MS Orleans support?

Reuben Bond (@ReubenBond)

Is there any Aspire support/guidance planned for Microsoft Orleans?

Orleans is a natural fit in the cloud native space - especially for C# devs
OpenTelemetry observability is already built in

If not, any advice / pointers on how to integrate it?

Example scenario

E.g. a scenario that I will be implementing in the coming months is running Orleans in Azure Container Apps (auto-scale) with both REST endpoints (minimal API's) and SignalR endpoints (BFF for WASM browser and native mobile clients, built in C# with Uno Platform)

I'm looking for ways to make end2end observability easier to achieve and maintain:
Rest API -> Orleans grains / streams chain <-> SignalR <-> client
And I'm wondering if Aspire could help with that.

Any insights appreciated! (Reuben Bond (@ReubenBond) ?)

Vincent Hoogendoorn (@VincentH-Net)

Hi Vincent Hoogendoorn (@VincentH-Net)! We've discussed this and we agree that Orleans is a natural fit here. I think we could have two main pieces:

An Aspire AppHost Orleans cluster resource which consists of:
- Default cluster, Grain persistence, & Reminder storage (we can pick Azure Table Storage for clustering & reminders, and Azure Blob Storage for grain state)
- ClusterId / ServiceId values, and any other common configuration
- The ability to add this resource to the service projects (i.e, where the Orleans silos & clients live)
An Aspire Orleans Component which pulls in the requisite packages (Eg, the Azure Storage providers) and uses IConfiguration (specified by the AppHost component in the dev case) to configure them.
- It should also configure Open Telemetry integration, as you mentioned

Benjamin Petit (@benjaminpetit) is looking into this. Your input is greatly appreciated.

Vincent Hoogendoorn (@VincentH-Net)

Hi Vincent Hoogendoorn (@VincentH-Net) ,

Quickly discussed with Reuben Bond (@ReubenBond), and I think for storage we should provision three different storage instances:

one for clustering
one for reminders
one for grain states

For local dev we could start an azurite docker instance. Or even maybe target some real storage if they are available?

Reuben Bond (@ReubenBond)

Thanks Reuben Bond (@ReubenBond), that sounds promising!

I'll follow up with Benjamin Petit (@benjaminpetit) - he already mailed me.

Benjamin Petit (@benjaminpetit)

Benjamin Petit (@benjaminpetit) Why 3?

If I am not mistaken, it doesn't cost more, but you benefit isolation from each other:

reminders can do a lot of requests
same for storage
clustering doesn't, but it's better if it doesn't get throttled because of reminder or storage.

Plus, we could do premium blob storage for grain storage, cosmosdb for reminders, and simple table storage for clustering for example.

Benjamin Petit (@benjaminpetit)

Benjamin Petit (@benjaminpetit) quick questions:

perhaps offer very lightweight and simple option(s) for local dev, e.g. in-memory or SqLite? Devs usually process very little data on local machines
is blob storage also a good option for many grains with small state size, or would table storage be better then? If yes, offer the dev a simple way to choose?
why would you need cosmosdb? It is powerful but also potentially complex & expensive - perhaps not a good fit when trying to make things simpler for devs...

is blob storage also a good option for many grains with small state size, or would table storage be better then? If yes, offer the dev a simple way to choose?

Blobs are a good default, especially because they don't have the 1MB limit which table rows do and they have lower overhead (no splitting / joining columns, less encoding), plus there are more price/performance/durability options for blob storage than with tables. The two most appropriate options for grain storage in Azure are Blobs & Cosmos DB, in my opinion.

microsoft/aspire-samples#20

i'd also love orleans support,
for local dev though i'd prefer to only use in memory storage (at least as an option), but maybe you're talking about the opinionated defaults for deploying?

we do in memory clustering when running locally but also do reminders in the local postgres database, (and dont use grain persistence at all) so being able to configure them individually would be great, even if there is also a kitchen-and-sink option that configures everything

aL3891 (@aL3891)

aL3891 (@aL3891) understood. I imagine we will offer a set of default choices as well as a way to configure providers in a more finer-grained manner.

Benjamin Petit (@benjaminpetit)

Small update, here's the branch for Benjamin Petit (@benjaminpetit)'s WIP: https://github.com/dotnet/aspire/tree/rebond/bpetit/orleans-hosting

Here's a usage example showing adding Azure Table Storage for clustering and Azure Blob Storage as the "default" grain state store:

var builder = DistributedApplication.CreateBuilder(args);

builder.AddAzureProvisioning();

var storage = builder.AddAzureStorage("storage");
var clusteringTable = storage.AddTables("clustering");
var grainStorage = storage.AddBlobs("grainstate");

var orleans = builder.AddOrleans("my-app")
                     .WithClustering(clusteringTable)
                     .WithGrainStorage("default", grainStorage);

builder.AddProject<Projects.OrleansServer>("silo")
       .WithOrleansServer(orleans)
       .WithReplicas(3);

using var app = builder.Build();

await app.RunAsync();

The pertinent line in the corresponding Orleans server project is builder.UseOrleansAspire(), as you can see in the sample, here: https://github.com/dotnet/aspire/blob/aaa5d258286a870bea66b1949572bf944e05f555/samples/orleans/OrleansServer/Program.cs#L7

Reuben Bond (@ReubenBond)

Reuben Bond (@ReubenBond) I succeeded in getting OrleansAppHost from https://github.com/dotnet/aspire/tree/rebond/bpetit/orleans-hosting to run. It took some puzzling effort to get Azure rights correct though.

The only change I had to make was in appsettings.json in the OrleansAppHost project, to update the Azure SubscriptionId.

However, I also had to assign the managed identity of the azure VM I was developing on the Storage Blob Data Contributor and Storage Table Data Contributor roles for the generated storage account.

Without those roles, host.StartAsync() throws:

Orleans.Runtime.OrleansException: 'Exception trying to create or connect to the Azure table OrleansSiloInstances'
RequestFailedException: This request is not authorized to perform this operation using this permission.
Status: 403 (Forbidden)
ErrorCode: AuthorizationPermissionMismatch

-> Perhaps good to add some guidance (readme in the sample source?) on this to help devs getting things working quickly?

Reuben Bond (@ReubenBond)

Reuben Bond (@ReubenBond) I noticed something 'off' in AddKeyedAzureTableService definition versus usage in https://github.com/dotnet/aspire/tree/rebond/bpetit/orleans-hosting:

The Orleans branch adds below overload, which adds the connectionName parameter:

    public static void AddKeyedAzureTableService(
        this IHostApplicationBuilder builder,
        string connectionName,
        string name,
        Action<AzureDataTablesSettings>? configureSettings = null,
        Action<IAzureClientBuilder<TableServiceClient, TableClientOptions>>? configureClientBuilder = null)

However, this overload is not referenced. The code in AspireOrleansServerExtensions invokes AddKeyedAzureTableService like this in 3 placees:

builder.AddKeyedAzureTableService(connectionName);

Which matches to the existing overload:

    public static void AddKeyedAzureTableService(
        this IHostApplicationBuilder builder,
        string name,
        Action<AzureDataTablesSettings>? configureSettings = null,
        Action<IAzureClientBuilder<TableServiceClient, TableClientOptions>>? configureClientBuilder = null)

So the passed in connectionName is used as name - does not seem intentional?

(how / when) can I use Orleans with public NuGet Aspire bits? If not possible today, any ETA? Thanks!

I'm trying to get Aspire for Orleans co-hosted with a minimal web API to work.

Approach 1

I tried to use the new Aspire.Hosting.Orleans and Aspire.Orleans.Server from https://github.com/dotnet/aspire/tree/rebond/bpetit/orleans-hosting with the existing preview 1 NuGets (just replaced the project references in those 2 Aspire projects with the latest NuGet.org package references, and then referenced the project assemblies and the NuGet packages in a regular Aspire preview 1 project that already contained a web api)

I got it all building without warnings, however when I do azd init (version 1.5.0) it fails with:

ERROR: generating bicep from manifest: unsupported resource type: azure.storage.table.v0

Updating azd to latest daily build (1.6.0 beta) like this issue comment suggests gives the same error.

Approach 2

I then tried to upgrade the Orleans example in the Aspire repo to add a WEB API with Rest and SignalR.
This works locally - I can run and debug on localhost, and have 3 silo's / API endpoints that form one Orleans cluster, where each endpoint hosts a minimal API and a SignalR hub. All work as expected.

However, when I try azd init in the orleans example folder I get:

ERROR: generating bicep from manifest:

Is deploying Orleans to ACA supported at this point?

I would like to see the addition of LogConsistencyStorage/JournaledGrain providers too.

In the hosting project, clustering and grain storage resources are tightly coupled to Azure resources. Are there any plans to modify this? What if I want to use CosmosDB or some custom storage provider? Currently, I get around this by adding the projects directly to my solution and modify it to accommodate my defaults.

Updating azd to latest daily build (1.6.0 beta) like this issue comment suggests gives the same error.

I would put that comment on the azd issue.

I would put that comment on the azd issue.

done

Benjamin Petit (@benjaminpetit)

Aspire support for Orleans is now added (requires Orleans nightly feed packages for now), including an example - I tested it, defects were fixed and all works well.

Thanks for this Benjamin Petit (@benjaminpetit) Reuben Bond (@ReubenBond) and all who chimed in!
NJoy all!

Is that sample deployable to azure using azd?

David Fowler (@davidfowl)

Is that sample deployable to azure using azd?

David Fowler (@davidfowl) Yes, if you replace the Aspire project references in the sample with the corresponding NuGet package references, and you comment out this line in the example source as indicated in the comment above that line.

Proof:

Note that if you try to deploy the Orleans playground to ACA without above modifications, AZD does not detect Aspire and fails like below:

Btw none of the other playgrounds are deployable to ACA as-is atm (they fail with different errors though).

Mitch Denny (@mitchdenny)

Mitch Denny (@mitchdenny) I think we need to exclude the manifest dashboard project from the manifest automagically as well so this just works.

https://github.com/dotnet/aspire/blob/104cbd4e414733d989cd62772efb0e500a5c30d6/playground/orleans/OrleansAppHost/Program.cs#L5-L12

You should be able to remove this branch. AzureProvisioning only applies when there's no publisher and UseEmulator isn't effective when publishing the manifest.

David Fowler (@davidfowl)

You should be able to remove this branch. AzureProvisioning only applies when there's no publisher and UseEmulator isn't effective when publishing the manifest.

David Fowler (@davidfowl) As an Orleans developer I never use the emulator. As a dev I code against the Orleans abstraction, and Orleans has providers for azure and in-memory. Locally I am not testing the Orleans providers, just my app code. So it makes more sense to me to use the most lightweight Orleans providers for the local dev loop.

The reason I am bringing this up is that in the common case, an Orleans dev does need to distinguish between local run and azure deployment. For me the most representative example would read like below. Imo this would be more representative of what most Orleans - Aspire devs will want:

var builder = DistributedApplication.CreateBuilder(args);

var orleans = builder.AddOrleans("my-app");

if (!args.Contains("--publisher")) // Local run; see https://github.com/dotnet/aspire/issues/1823
{
    orleans = orleans
        .WithDevelopmentClustering()
        .WithMemoryGrainStorage("Default");
}
else // Azure deployment
{
    var storage = builder.AddAzureStorage("storage");
    var clusteringTable = storage.AddTables("clustering");
    var grainStorage = storage.AddBlobs("grainstate");

    orleans = orleans
        .WithClustering(clusteringTable)
        .WithGrainStorage("Default", grainStorage);
}

builder.AddProject<Projects.OrleansServer>("silo")
       .WithReference(orleans);

using var app = builder.Build();

await app.RunAsync();

This is something we are going to add support for. In this particular sample it's unnecessary.

True. My concern is guidance for devs today, until the new API is there.

I agree it makes sense to either remove the branch in the current example or modify the example to something like above. Since the current way to do it is a bit of an ugly wart, I can understand wanting to avoid that in the example ;-)

If we keep the example as-is (minus the branch) I will point devs to #1823 which shows the current workaround and allows to track the new API

David Fowler (@davidfowl)

David Fowler (@davidfowl) I created a PR for the Orleans playground to address your comments:
#1904

Navigation Menu

Repository navigation

Question: MS Orleans support? #724

Example scenario

3 remaining items

Approach 1

Approach 2

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Question: MS Orleans support? #724

Description

Example scenario

Activity

ReubenBond commented on Nov 7, 2023

benjaminpetit commented on Nov 7, 2023

VincentH-Net commented on Nov 7, 2023

davidfowl commented on Nov 7, 2023

benjaminpetit commented on Nov 7, 2023

VincentH-Net commented on Nov 7, 2023

ReubenBond commented on Nov 7, 2023

danmoseley commented on Nov 13, 2023

aL3891 commented on Nov 17, 2023

ReubenBond commented on Nov 17, 2023

ReubenBond commented on Nov 22, 2023

VincentH-Net commented on Nov 23, 2023

3 remaining items

VincentH-Net commented on Nov 27, 2023

VincentH-Net commented on Nov 27, 2023

Approach 1

Approach 2

zbarrier commented on Nov 27, 2023

davidfowl commented on Nov 28, 2023

VincentH-Net commented on Nov 28, 2023

VincentH-Net commented on Jan 24, 2024

davidfowl commented on Jan 24, 2024

VincentH-Net commented on Jan 25, 2024

davidfowl commented on Jan 25, 2024

VincentH-Net commented on Jan 25, 2024

davidfowl commented on Jan 25, 2024

VincentH-Net commented on Jan 26, 2024

VincentH-Net commented on Jan 27, 2024

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions