[MS] Track Your Token Usage and Costs with Semantic Kernel - devamazonaws.blogspot.com

Image skpatternlarge This is a follow up blog post to Unlock the Power of Telemetry in Semantic Kernel SDK. We are excited to share we have added logging and metering for prompt, completion, and total tokens for each request to Azure OpenAI/Open AI using Semantic Kernel. This feature will help you monitor and optimize your API usage and costs, as well as troubleshoot any issues that may arise. Example of logging in Azure Application Insights Image Token logging Example of metering in Azure Application Insights Image Token metering With this update, Microsoft.SemanticKernel.Connectors.AI.OpenAI package captures the metrics listed below.
  • SK.Connectors.OpenAI.PromptTokens - number of prompt tokens used.
  • SK.Connectors.OpenAI.CompletionTokens - number of completion tokens used.
  • SK.Connectors.OpenAI.TotalTokens - total number of tokens used.

Benefits of logging and metering with Semantic Kernel

Logging and metering your requests with Semantic Kernel have several key benefits:
  • You can easily keep track of your API usage and costs, as each token corresponds to a unit of billing. You can also compare the token usage across different models and parameters to find the optimal settings for your use case.
  • You can troubleshoot any issues or errors that may occur during your requests, as each request is logged with its prompt, completion, and token count. You can also use the logs to analyze the performance and quality of your completions.
  • You can leverage the power and flexibility of Semantic Kernel to orchestrate your requests with other AI services or plugins, while still having full visibility and control over your token usage.

Next Steps

  • Try out Semantic Kernel by visiting our Github repo. Don’t forget to support our project by star’ing our repo.
  • You can also check out our documentation for more examples and tutorials on how to use Semantic Kernel.
  • Join our community to contribute or if you have question.
We hope you enjoy using Semantic Kernel to create amazing AI apps.
Post Updated on August 30, 2023 at 05:52PM
Thanks for reading
from devamazonaws.blogspot.com

Comments

Post a Comment

Popular posts from this blog

Scenarios capability now generally available for Amazon Q in QuickSight - devamazonaws.blogspot.com

Today, AWS announces the general availability of the scenarios capability of Amazon Q in QuickSight. Amazon Q guides you through data analysis by uncovering hidden trends, making recommendations for your business, and intelligently suggesting next steps for deeper exploration—all in response to natural language interactions. Now anyone can explore past trends, forecast future scenarios, and model solutions without needing specialized skill, analyst support, or manual manipulation of data in spreadsheets. With its intuitive interface and step-by-step guidance, the scenarios capability of Amazon Q in QuickSight helps users perform complex data analysis up to 10x faster than spreadsheets. Whether you're optimizing marketing budgets, streamlining supply chains, or analyzing investments, Amazon Q makes advanced data analysis accessible so you can make data-driven decisions across your organization. This capability is accessible from any Amazon QuickSight dashboard, so you can move seam...
Read more

Research and Engineering Studio on AWS Version 2024.08 now available - devamazonaws.blogspot.com

Today we’re excited to announce Research and Engineering Studio (RES) on AWS Version 2024.08. This release adds new features such as Amazon S3 bucket mountpoints for Linux, allows creation of custom user roles and permission profiles, and offers the ability to adjust the list of Amazon EC2 instances available to launch as virtual desktops. Amazon S3 mountpoints allow Linux desktops to access S3 buckets as a file system. Amazon S3 buckets are created using the AWS Console or AWS CLI and onboarded to RES by administrators using the S3 Buckets page in the web portal. You can mount buckets in either Read Only or Read/Write mode. Read/Write buckets have an optional setting to restrict data access by project, or both project and user. RES can also mount S3 buckets from other AWS accounts with the proper permissions. Custom permission profiles allow administrators to create unique permissions and assign them to users or groups. Start by modifying the default Project Member and Project Ow...
Read more

Amazon EC2 C6id instances are now available in AWS Europe (Paris) region - devamazonaws.blogspot.com

Starting today, Amazon Elastic Compute Cloud (Amazon EC2) C6id instances are available in Europe (Paris) Region. These instances are powered by 3rd generation Intel Xeon Scalable Ice Lake processors with an all-core turbo frequency of 3.5 GHz and up to 7.6 TB of local NVMe-based SSD block-level storage. C6id instances are built on AWS Nitro System , a combination of dedicated hardware and lightweight hypervisor, which delivers practically all of the compute and memory resources of the host hardware to your instances for better overall performance and security. Customers can take advantage of access to high-speed, low-latency local storage to scale performance of applications such data logging, distributed web-scale in-memory caches, in-memory databases, and real-time big data analytics. These instances are generally available today in the US West (Oregon), US East (Ohio, N. Virginia), Canada (Central), Canada West (Calgary), AWS GovCloud (US-West), Mexico (Central), South America (...
Read more