Comprehensive observability for Amazon SageMaker AI LLM inference: From GPU utilization to LLM quality

TutoSartup excerpt from this article:
LLM output quality can change over time as input distributions shift, and quality monitoring helps detect these changes early... A comprehensive observability approach for LLM inference must address two distinct but complementary dimensions: model serving infrastructure (quantity) and LLM quality... The next stage adds LLM quality through sampling and evaluation, which surface issues such as mod...

Why and how to migrate to a Transit Gateway-attached AWS Network Firewall

TutoSartup excerpt from this article:
Customers commonly use Transit Gateway to route traffic from Amazon Virtual Private Cloud (Amazon VPC) networks to a centralized inspection VPC (a VPC dedicated to hosting firewall endpoints for traffic inspection) where their network firewall endpoints are deployed... Customers deploying Network Firewall in a centralized deployment model using Transit Gateway have traditionally set up a dedicat...

Training Azerbaijani language models on Amazon SageMaker AI

TutoSartup excerpt from this article:
The custom monolingual tokenizer achieved the strongest results, halving the tokens per word compared to the baseline... Fine-tuning used approximately 2,000 single-turn Azerbaijani question-answer pairs using Hugging Face’s SFTTrainer with a learning rate of 1e-4—higher than CPT’s learning rates because LoRA adapters are randomly initialized and benefit from stronger gradient updates... He...

Build a custom portal with embedded Amazon SageMaker AI MLflow Apps

TutoSartup excerpt from this article:
As ML teams grow, embedding Amazon SageMaker AI MLflow Apps into a custom portal requires a scalable approach to access management... Teams who rely on SSO-integrated internal portals need their MLflow experiment tracking accessible alongside other internal applications through a single bookmarkable URL... With this solution, you give your machine learning (ML) teams a persistent, bookmarkable UR...

Streamline external access to Amazon SageMaker MLflow using a REST API proxy

TutoSartup excerpt from this article:
Machine learning (ML) teams use MLflow to manage their ML lifecycle effectively... Amazon SageMaker MLflow provides comprehensive ML experiment tracking and model management capabilities... Many organizations need to integrate Amazon SageMaker MLflow with their established systems while maintaining their security and infrastructure patterns... In this post, we demonstrate how to build a secure F...

Introducing the next generation of AWS Resilience Hub for generative AI-based SRE resilience journey

TutoSartup excerpt from this article:
Rather than choosing a single rigid policy type, you construct policies by selecting the requirements that matter to your application, such as service level objective (SLO), multi-AZ and multi-Region disaster recovery, and data recovery requirements... Systems represent a business application, user journeys describe critical business paths, and services are the deployable units comprising AWS re...

Simplifying policy management with URL and Domain Category filtering on AWS Network Firewall

TutoSartup excerpt from this article:
With URL and domain category filtering, security teams can use predefined categories to control access instead of managing individual domains... Instead of manually tracking every new AI service, you can control access to the entire Artificial Intelligence and Machine Learning category while creating exceptions for approved services... In this post, we walk through URL and domain category filter...