AWS Neuron adds support for PyTorch 2.1 and Llama-2-70b model inference - devamazonaws.blogspot.com

AWS Neuron is the SDK for Amazon EC2 Inferentia and Trainium based instances purpose-built for generative AI. Today, with Neuron 2.16 release, we are announcing support for Llama-2 70b model inference on Inf2 instances. 

Post Updated on December 22, 2023 at 06:22PM

Comments

Post a Comment

Popular posts from this blog

Scenarios capability now generally available for Amazon Q in QuickSight - devamazonaws.blogspot.com

Today, AWS announces the general availability of the scenarios capability of Amazon Q in QuickSight. Amazon Q guides you through data analysis by uncovering hidden trends, making recommendations for your business, and intelligently suggesting next steps for deeper exploration—all in response to natural language interactions. Now anyone can explore past trends, forecast future scenarios, and model solutions without needing specialized skill, analyst support, or manual manipulation of data in spreadsheets. With its intuitive interface and step-by-step guidance, the scenarios capability of Amazon Q in QuickSight helps users perform complex data analysis up to 10x faster than spreadsheets. Whether you're optimizing marketing budgets, streamlining supply chains, or analyzing investments, Amazon Q makes advanced data analysis accessible so you can make data-driven decisions across your organization. This capability is accessible from any Amazon QuickSight dashboard, so you can move seam...
Read more

Research and Engineering Studio on AWS Version 2024.08 now available - devamazonaws.blogspot.com

Today we’re excited to announce Research and Engineering Studio (RES) on AWS Version 2024.08. This release adds new features such as Amazon S3 bucket mountpoints for Linux, allows creation of custom user roles and permission profiles, and offers the ability to adjust the list of Amazon EC2 instances available to launch as virtual desktops. Amazon S3 mountpoints allow Linux desktops to access S3 buckets as a file system. Amazon S3 buckets are created using the AWS Console or AWS CLI and onboarded to RES by administrators using the S3 Buckets page in the web portal. You can mount buckets in either Read Only or Read/Write mode. Read/Write buckets have an optional setting to restrict data access by project, or both project and user. RES can also mount S3 buckets from other AWS accounts with the proper permissions. Custom permission profiles allow administrators to create unique permissions and assign them to users or groups. Start by modifying the default Project Member and Project Ow...
Read more

Amazon EC2 C6id instances are now available in AWS Europe (Paris) region - devamazonaws.blogspot.com

Starting today, Amazon Elastic Compute Cloud (Amazon EC2) C6id instances are available in Europe (Paris) Region. These instances are powered by 3rd generation Intel Xeon Scalable Ice Lake processors with an all-core turbo frequency of 3.5 GHz and up to 7.6 TB of local NVMe-based SSD block-level storage. C6id instances are built on AWS Nitro System , a combination of dedicated hardware and lightweight hypervisor, which delivers practically all of the compute and memory resources of the host hardware to your instances for better overall performance and security. Customers can take advantage of access to high-speed, low-latency local storage to scale performance of applications such data logging, distributed web-scale in-memory caches, in-memory databases, and real-time big data analytics. These instances are generally available today in the US West (Oregon), US East (Ohio, N. Virginia), Canada (Central), Canada West (Calgary), AWS GovCloud (US-West), Mexico (Central), South America (...
Read more