Looking back on Q1 2023 - what has changed in Amazon SageMaker over the last few months?

Not much it seems.

I initially planned to host a quarterly review as a part of this blog. Unfortunately, the first quarter of 2023 was slightly underwhelming in terms of new SageMaker announcements. Perhaps Amazon and AWS is busy building their own LLM in secret …or completely ignores the revolution (or fad, depending who you ask).

EDIT: On 13 April, Amazon indeed revealed their LLM and LLM-as-a-Service. I’ll cover that in my next post!

Let’s hope for the former and jump right into the quarter summary.

ml.p4de.24xlarge instances are in Preview

These monsters hold eight A100 GPUs with 80GB high bandwitdth memory each. Compared to ml.p4d.24xlarge, this is twice as much memory in your GPU (320GB vs 640GB). Juicy for any LLM-related workloads. These are available only in two regions (us-east-1 and us-west-2 to be precise) and you need to apply them via AWS Support.

HuggingFace partnership

Not SageMaker per-se but AWS just expanded their partnership with HuggingFace. However, it is not clear what it actually means for us, the customers. Probably HF containers and models will easily run on Trainium and Inferentia, purpose-built chips created by AWS.

Well, let’s hope that the HF AWS Deep Learning Containers that’s over one year old will get some updates, as part of this partnership.

Many tweaks in SageMaker Canvas

Probably the most actively developed service was SageMaker Canvas. This no-code ML tool targeted to business analysts now:

• trains their regular models up to 3 times faster

• supports NLP and CV cases

• allows you to choose from ready-to-use models (I wonder if they’ll add native integration with HuggingFace hundreds of thousands of models?)

There’s even more, but technically in Q2 (early April):

• it now allows you to pull data from 45+ sources (via Amazon AppFlow). These sources include Facebook Ads, Github, Asana, Google Ads, Google Analytics, Jira, LinkedIn Ads, Salesforce, Snowflake, Stripe, Zendesk and more!

• it integrates with Amazon QuickSight, meaning you can use Canvas-trained models in QuickSight

SageMaker is available in 5 more regions

For people starting out with AWS, this might be a shock, but Amazon SageMaker is not available in all AWS Regions. Of course, if you’re working in us-east-1, us-west-2, eu-west-1 or similar established regions, you might not even realise that.

Q1 opened the service for customers in GovCloud US-East, Middle East, Zurich and Hyderabad.

Generative AI Accelerator

Startups building on Generative AI might get up to $300,000 in AWS credits to build their products on AWS. I wonder if this programme verifies whether a startup actually builds something or only hosts a tiny overlay on top of GPT-4 with LangChain, like most of them do. Not sure what LangChain is? I’ve got you covered:

LangChain - the Guide of LLMs

AWS also started hosting a weekly “Build on Generative AI” Twitch streams, where they showcase various Generative AI related news, projects and prototypes. If you wish, you can contact them and present your project too - live on Twitch TV!

Minor changes in Automatic Model Tuning

You can now specify environment variables for your Training Jobs managed by AMT and stop the jobs with new criteria such as “maximum total runtime”.

The rest

• you can use SageMaker Training with a private Docker registry

• you can choose which algorithms will SageMaker Autopilot use

• you can launch SageMaker Model Monitor directly from SageMaker Model Dashboard

• SageMaker Data Wrangler now supports EMR Hive as query engine and can use OAuth to access Snowflake

• SageMaker Feature Store supports hard deletion in online store

---

Summary

So, WDYT? Were any of the changes actually meaningful to you. I’m personally waiting until AWS/Amazon releases their own LLM, to add some fuel to the fire. A vector database (or a pg_vector in RDS) would be great to have too.