Skip to content

Adding Basic anthropic Claude support. #404

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
dluc merged 13 commits into from
May 19, 2024
Merged

Adding Basic anthropic Claude support. #404

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
154c811
Added very basic anthropic support
alkampfergit Apr 17, 2024
8eb69f2
Rebase with latest code from main branch
dluc Apr 17, 2024
1801f7f
Fixes / Refactoring
dluc Apr 17, 2024
1fc10fd
Exclude cl100k_base.tiktoken from typos check
dluc Apr 17, 2024
a07589e
More fix after PR comments
alkampfergit Apr 18, 2024
3a7566f
Fix after rebase
alkampfergit Apr 24, 2024
fe8b8d5
Fix config, refactoring names, csproj setup, nugget details
dluc Apr 26, 2024
092a844
Fix csproj
dluc Apr 27, 2024
5a6b5cc
Remove tiktoken files and use TikToken package. Fix null checks. Fix …
dluc Apr 28, 2024
c0e5c3a
Update .github/_typos.toml
dluc Apr 28, 2024
8641683
Update extensions/Anthropic/Anthropic.csproj
dluc Apr 28, 2024
d780389
Complete client
dluc May 19, 2024
4f8532e
Add Anthropic support in service config
dluc May 19, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions KernelMemory.sln
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -276,6 +276,10 @@ Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "applications", "application
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Evaluation", "applications\evaluation\Evaluation.csproj", "{432AC1B4-8275-4284-9A44-44988A6F0C24}"
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Anthropic", "extensions\Anthropic\Anthropic.csproj", "{A0C81A29-715F-463E-A243-7E45DB8AE53F}"
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "110-dotnet-anthropic", "examples\110-dotnet-anthropic\110-dotnet-anthropic.csproj", "{EE0D8645-2770-4E12-8E18-019B30970FE6}"
EndProject
Global
GlobalSection(SolutionConfigurationPlatforms) = preSolution
Debug|Any CPU = Debug|Any CPU
Expand Down Expand Up @@ -513,6 +517,13 @@ Global
{432AC1B4-8275-4284-9A44-44988A6F0C24}.Debug|Any CPU.Build.0 = Debug|Any CPU
{432AC1B4-8275-4284-9A44-44988A6F0C24}.Release|Any CPU.ActiveCfg = Release|Any CPU
{432AC1B4-8275-4284-9A44-44988A6F0C24}.Release|Any CPU.Build.0 = Release|Any CPU
{A0C81A29-715F-463E-A243-7E45DB8AE53F}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{A0C81A29-715F-463E-A243-7E45DB8AE53F}.Debug|Any CPU.Build.0 = Debug|Any CPU
{A0C81A29-715F-463E-A243-7E45DB8AE53F}.Release|Any CPU.ActiveCfg = Release|Any CPU
{A0C81A29-715F-463E-A243-7E45DB8AE53F}.Release|Any CPU.Build.0 = Release|Any CPU
{EE0D8645-2770-4E12-8E18-019B30970FE6}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{EE0D8645-2770-4E12-8E18-019B30970FE6}.Debug|Any CPU.Build.0 = Debug|Any CPU
{EE0D8645-2770-4E12-8E18-019B30970FE6}.Release|Any CPU.ActiveCfg = Release|Any CPU
EndGlobalSection
GlobalSection(SolutionProperties) = preSolution
HideSolutionNode = FALSE
Expand Down Expand Up @@ -595,6 +606,8 @@ Global
{43877864-6AE8-4B03-BEDA-6B6FA8BB1D8B} = {155DA079-E267-49AF-973A-D1D44681970F}
{FAE4C6B8-38B2-43E7-8881-99693C9CEDC6} = {0A43C65C-6007-4BB4-B3FE-8D439FC91841}
{432AC1B4-8275-4284-9A44-44988A6F0C24} = {DBEA0A6B-474A-4E8C-BCC8-D5D43C063A54}
{A0C81A29-715F-463E-A243-7E45DB8AE53F} = {155DA079-E267-49AF-973A-D1D44681970F}
{EE0D8645-2770-4E12-8E18-019B30970FE6} = {0A43C65C-6007-4BB4-B3FE-8D439FC91841}
EndGlobalSection
GlobalSection(ExtensibilityGlobals) = postSolution
SolutionGuid = {CC136C62-115C-41D1-B414-F9473EFF6EA8}
Expand Down
20 changes: 11 additions & 9 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -175,7 +175,7 @@ Here's comparison table:
| Custom storage schema | some DBs | - |
| Vector DBs with internal embedding | Yes | - |
| Concurrent write to multiple vector DBs | Yes | - |
| LLMs | [Azure OpenAI](https://learn.microsoft.com/azure/ai-services/openai/concepts/models), [OpenAI](https://platform.openai.com/docs/models), [LLamaSharp](https://github.com/SciSharp/LLamaSharp) via [llama.cpp](https://github.com/ggerganov/llama.cpp), [LM Studio](https://lmstudio.ai/), Anthropic, Semantic Kernel connectors | Azure OpenAI, OpenAI, Gemini, Hugging Face, ONNX, custom ones, etc. |
| LLMs | [Azure OpenAI](https://learn.microsoft.com/azure/ai-services/openai/concepts/models), [OpenAI](https://platform.openai.com/docs/models), [Anthropic](https://www.anthropic.com), [LLamaSharp](https://github.com/SciSharp/LLamaSharp) via [llama.cpp](https://github.com/ggerganov/llama.cpp), [LM Studio](https://lmstudio.ai/), Semantic Kernel connectors | Azure OpenAI, OpenAI, Gemini, Hugging Face, ONNX, custom ones, etc. |
| LLMs with dedicated tokenization | Yes | No |
| Cloud deployment | Yes | - |
| Web service with OpenAPI | Yes | - |
Expand Down Expand Up @@ -307,14 +307,16 @@ running the service locally with OpenAPI enabled.
14. [Using Semantic Kernel LLM connectors](examples/107-dotnet-SemanticKernel-TextCompletion)
15. [Using custom content decoders](examples/108-dotnet-custom-content-decoders)
16. [Using a custom web scraper to fetch web pages](examples/109-dotnet-custom-webscraper)
17. [Writing and using a custom ingestion handler](examples/201-dotnet-serverless-custom-handler)
18. [Running a single asynchronous pipeline handler as a standalone service](examples/202-dotnet-custom-handler-as-a-service)
19. [Test project using KM package from nuget.org](examples/203-dotnet-using-core-nuget)
20. [Integrating Memory with ASP.NET applications and controllers](examples/204-dotnet-ASP.NET-MVC-integration)
21. [Sample code showing how to extract text from files](examples/205-dotnet-extract-text-from-docs)
22. [.NET configuration and logging](examples/206-dotnet-configuration-and-logging)
23. [Expanding chunks retrieving adjacent partitions](examples/207-dotnet-expanding-chunks-on-retrieval)
24. [Using local models via LM Studio](examples/208-dotnet-lmstudio)
17. [Generating answers with Anthropic LLMs](examples/110-dotnet-anthropic)
18. [Hybrid Search with Azure AI Search](examples/111-dotnet-azure-ai-hybrid-search)
19. [Writing and using a custom ingestion handler](examples/201-dotnet-serverless-custom-handler)
20. [Running a single asynchronous pipeline handler as a standalone service](examples/202-dotnet-custom-handler-as-a-service)
21. [Test project using KM package from nuget.org](examples/203-dotnet-using-core-nuget)
22. [Integrating Memory with ASP.NET applications and controllers](examples/204-dotnet-ASP.NET-MVC-integration)
23. [Sample code showing how to extract text from files](examples/205-dotnet-extract-text-from-docs)
24. [.NET configuration and logging](examples/206-dotnet-configuration-and-logging)
25. [Expanding chunks retrieving adjacent partitions](examples/207-dotnet-expanding-chunks-on-retrieval)
26. [Using local models via LM Studio](examples/208-dotnet-lmstudio)

## Tools

Expand Down
2 changes: 1 addition & 1 deletion docs/index.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,7 @@ Here's comparison table:
| Custom storage schema | some DBs | - |
| Vector DBs with internal embedding | Yes | - |
| Concurrent write to multiple vector DBs | Yes | - |
| LLMs | [Azure OpenAI](https://learn.microsoft.com/azure/ai-services/openai/concepts/models), [OpenAI](https://platform.openai.com/docs/models), [LLamaSharp](https://github.com/SciSharp/LLamaSharp) via [llama.cpp](https://github.com/ggerganov/llama.cpp), [LM Studio](https://lmstudio.ai/), Anthropic, Semantic Kernel connectors | Azure OpenAI, OpenAI, Gemini, Hugging Face, ONNX, custom ones, etc.
| LLMs | [Azure OpenAI](https://learn.microsoft.com/azure/ai-services/openai/concepts/models), [OpenAI](https://platform.openai.com/docs/models), [Anthropic](https://www.anthropic.com), [LLamaSharp](https://github.com/SciSharp/LLamaSharp) via [llama.cpp](https://github.com/ggerganov/llama.cpp), [LM Studio](https://lmstudio.ai/), Semantic Kernel connectors | Azure OpenAI, OpenAI, Gemini, Hugging Face, ONNX, custom ones, etc.
| LLMs with dedicated tokenization | Yes | No |
| Cloud deployment | Yes | - |
| Web service with OpenAPI | Yes | - |
Expand Down
23 changes: 23 additions & 0 deletions examples/110-dotnet-anthropic/110-dotnet-anthropic.csproj
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
<Project Sdk="Microsoft.NET.Sdk.Web">

<PropertyGroup>
<TargetFramework>net8.0</TargetFramework>
<ImplicitUsings>enable</ImplicitUsings>
<NoWarn>$(NoWarn);KMEXP00;</NoWarn>
</PropertyGroup>

<ItemGroup>
<ProjectReference Include="..\..\service\Core\Core.csproj" />
</ItemGroup>

<ItemGroup>
<None Remove="file5-NASA-news.pdf" />
</ItemGroup>

<ItemGroup>
<Content Include="file5-NASA-news.pdf">
<CopyToOutputDirectory>Always</CopyToOutputDirectory>
</Content>
</ItemGroup>

</Project>
38 changes: 38 additions & 0 deletions examples/110-dotnet-anthropic/Program.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
// Copyright (c) Microsoft. All rights reserved.

using Microsoft.KernelMemory;
using Microsoft.KernelMemory.AI.Anthropic;
using Microsoft.KernelMemory.DocumentStorage.DevTools;
using Microsoft.KernelMemory.MemoryStorage.DevTools;

var anthropicConfig = new AnthropicConfiguration();
var azureOpenAIEmbeddingConfig = new AzureOpenAIConfig();

new ConfigurationBuilder()
.AddJsonFile("appsettings.json")
.AddJsonFile("appsettings.Development.json", optional: true)
.AddEnvironmentVariables()
.Build()
.BindSection("KernelMemory:Services:Anthropic", anthropicConfig)
.BindSection("KernelMemory:Services:AzureOpenAIEmbedding", azureOpenAIEmbeddingConfig);

var memory = new KernelMemoryBuilder()
// Generate answers using Anthropic
.WithAnthropicTextGeneration(anthropicConfig)
// Generate embeddings using Azure OpenAI
.WithAzureOpenAITextEmbeddingGeneration(azureOpenAIEmbeddingConfig)
// Persist memory on disk
.WithSimpleFileStorage(SimpleFileStorageConfig.Persistent)
.WithSimpleVectorDb(SimpleVectorDbConfig.Persistent)
.Build<MemoryServerless>();

// Import a document in memory
await memory.ImportDocumentAsync("file5-NASA-news.pdf", "file5-NASA-news.pdf");
Console.WriteLine("Document imported");

// now ask a question
var question = "What is orion?";
var answer = await memory.AskAsync(question);

Console.WriteLine($"Question: {question}");
Console.WriteLine($"Answer: {answer.Result}");
43 changes: 43 additions & 0 deletions examples/110-dotnet-anthropic/appsettings.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
{
"Logging": {
"LogLevel": {
"Default": "Warning",
// Examples: how to handle logs differently by class
// "Microsoft.KernelMemory.Handlers.TextExtractionHandler": "Information",
// "Microsoft.KernelMemory.Handlers.TextPartitioningHandler": "Information",
// "Microsoft.KernelMemory.Handlers.GenerateEmbeddingsHandler": "Information",
// "Microsoft.KernelMemory.Handlers.SaveEmbeddingsHandler": "Information",
// "Microsoft.KernelMemory.DocumentStorage.AzureBlobs": "Information",
// "Microsoft.KernelMemory.Pipeline.Queue.AzureQueues": "Information",
"Microsoft.AspNetCore": "Warning"
}
},
"KernelMemory": {
"Services": {
"Anthropic": {
"Endpoint": "https://api.anthropic.com",
"EndpointVersion": "2023-06-01",
// Note: this can be set also via "KernelMemory__Services__Anthropic__ApiKey" env var
"ApiKey": "",
// See https://docs.anthropic.com/claude/docs/models-overview for list of models and details
"TextModelName": "claude-3-haiku-20240307",
// How many tokens the model can receive in input and generate in output
// See https://docs.anthropic.com/claude/docs/models-overview
"MaxTokenIn": 200000,
"MaxTokenOut": 4096
},
"AzureOpenAIEmbedding": {
// "ApiKey" or "AzureIdentity"
// AzureIdentity: use automatic AAD authentication mechanism. You can test locally
// using the env vars AZURE_TENANT_ID, AZURE_CLIENT_ID, AZURE_CLIENT_SECRET.
"Auth": "AzureIdentity",
"Endpoint": "https://<...>.openai.azure.com/",
"APIKey": "",
"Deployment": "",
// The max number of tokens supported by model deployed
// See https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/models
"MaxTokenTotal": 8191
}
}
}
}
Binary file added examples/110-dotnet-anthropic/file5-NASA-news.pdf
View file
Open in desktop
Binary file not shown.
Loading