KernelConfig.CompletionBackends is closed, cannot add custom ai backend to the kernel. #146

Activity

1. EvanChaki_MSFT (evchaki)

   assigned

   Devis Lucato (dluc)

   on Mar 23, 2023
2. dluc commented on Mar 24, 2023

   

   Devis Lucato (dluc)

   on Mar 24, 2023

   Contributor

   More actions

   Rodrigo Matiazo (@matiazo) without disclosing private info, could you provide the technical changes you would make? we can try adding support for more scenarios
3. Devis Lucato (dluc)

   added

   extensibility

   on Mar 24, 2023
4. Rodrigo Matiazo (matiazo)

   changed the title [-]KernelConfig.CompletionBackends is closed, cannot add custom ai backs to the kernel.[/-] [+]KernelConfig.CompletionBackends is closed, cannot add custom ai backend to the kernel.[/+] on Mar 24, 2023
5. matiazo commented on Mar 24, 2023

   

   Rodrigo Matiazo (matiazo)

   on Mar 24, 2023

   Author

   More actions

   Goal is not add more implementation but open Kernel to accept new implementations of IBackendConfig and ITextCompletionClient. We also need to review usage of IEmbeddingGenerator implementations.

   I'm working in a PR but the overall changes so far:

   • add KernelConfig.AddCompletionBackend(IBackendConfig)
   • add public ITextCompletionClient CreateAIBackend() to IBackendConfig
   • Kernel.CreateSemanticFunction - replace a swtich on concrete implementation of IBackendConfig
6. dluc commented on Mar 24, 2023

   

   Devis Lucato (dluc)

   on Mar 24, 2023

   Last edited by dluc

   Contributor

   More actions

   right, that's in line with current work in progress:

   add KernelConfig.AddCompletionBackend(IBackendConfig)
   

   spot on. Currently the list is hardcoded, we're removing that and simply work with interfaces 👍

   add public ITextCompletionClient CreateAIBackend() to IBackendConfig
   

   that could be a temporary stop gap, but we're removing this hard coded assumptions about what kind of AI is used.

   Kernel.CreateSemanticFunction - replace a swtich on concrete implementation of IBackendConfig
   

   yup makes sense, in fact I would suggest limiting the changes to a minimum and only in this area, without worrying too much about hardcoding a new option. In other words it's ok following the current approach and hardcoding a new option, because we're taking care of that and it's a rather big change
7. dluc commented on Mar 25, 2023

   

   Devis Lucato (dluc)

   on Mar 25, 2023

   Last edited by dluc

   Contributor

   More actions

   Rodrigo Matiazo (@matiazo) I have a PR almost ready addressing this, thanks for raising this issue
8. dluc commented on Mar 26, 2023

   

   Devis Lucato (dluc)

   on Mar 26, 2023

   Contributor

   More actions

   Rodrigo Matiazo (@matiazo) this should help #161 - Here's an example showing how to plug in custom backends: https://github.com/dluc/semantic-kernel/blob/dluc99-backends/samples/dotnet/kernel-syntax-examples/Example16_CustomLLM.cs
9. awharrison-28 commented on Mar 29, 2023

   

   Abby Harrison (awharrison-28)

   on Mar 29, 2023

   Contributor

   More actions

   Updating thread with the link that is now in main https://github.com/microsoft/semantic-kernel/blob/main/samples/dotnet/kernel-syntax-examples/Example16_CustomLLM.cs
10. dluc commented on Mar 31, 2023

    

    Devis Lucato (dluc)

    on Mar 31, 2023

    Contributor

    More actions

    Shipped
11. Devis Lucato (dluc)

    closed this as completedon Mar 31, 2023