What SageMaker JumpStart is and should you even care

SageMaker JumpStart is getting a lot of traction lately. Is it a separate service? An API? A GUI? A SDK?

There's been a lot of buzz around SageMaker JumpStart - and it turns out, there's a good reason for this.

What is SageMaker JumpStart?

SageMaker JumpStart was introduced at re:Invent 2020. The reason for its creation was simple - Amazon SageMaker struck people as a complex platform. To train a simple model and then deploy it, you had to run at least several different SageMaker technologies. This was challenging if you just began your journey. To make things worse, the quality of tutorials on the internet varied and many of them were outdated due to rapid development of SageMaker itself.

To overcome that mental burden, SageMaker JumpStart was introduced. It provides with a simple (“one-click”) UI that allows you to deploy sample machine learning cases on SageMaker. Underneath, usual services such as SageMaker Inference, SageMaker Training, SageMaker Processing are doing the heavy-lifting.

After selecting a particular use case or a model, you are presented with a simple UI that lets you train or straight up deploy the pretrained model:

Additionally, JumpStart also contains a detailed end-to-end notebooks with the selected solution. This means you also receive a tutorial regarding each use case, that showcases how it was built, leveraging the usual SageMaker SDK.

Essentially, this is a built-in collection of ready to use solutions/models that also include AWS-tailored (and AWS-managed) tutorials with more details regarding each of them. JumpStart does not provide anything but that high-level UI to get you started with ML on AWS. As I said, the solutions are deployed using the regular SageMaker services such as SageMaker Inference or SageMaker Training.

Of course, the devil was in the details - rarely had the provided solution actually solve your problem end-to-end on the first try. You often needed to collect your own dataset, analyse it thoroughly, prepare it for the algorithm, tweak and retrain the model many times and so on. Nevertheless, it gave you a great head start, especially when you just began your SageMaker journey.

However, with the growing popularity of large language models, things seem to have changed a little - and it looks like JumpStart is benefiting from this shift.

A major shift in machine learning in 2022 and beyond

In many cases, foundation models, especially these with the most hype around them (LLMs) often do not require any fine-tuning at all. The models have seen so much data and have been trained in such a way, that you don’t need to do anything around them to let them do their magic. All you need to do is pick a suitable model, deploy it and use it as an API in your application. This hardly requires any machine learning skills.

Therefore, a tool such as JumpStart seems just perfectly suited for this fundamental shift. You have a hub of ready to use open source models in your application. One click, some waiting - et voila! The model is deployed and ready to use.

If open source foundation models keep growing in popularity and strength (think what Falcon 40B can do), JumpStart makes it plain simple to deploy them on AWS.

Note - foundation models are not a silver bullet. There’s still a lot of “traditional” machine learning in predictive or prescriptive machine learning or forecasting at hundreds of thousands of companies. We have not thrown everything out the window. However, for some NLP-related cases, it seems like (at least for now) all you need to do is to deploy a foundation model and then prompt it accordingly.

JumpStart could be the easiest way to deliver one of the building blocks (a model).

New SageMaker JumpStart additions of 2023

Oh, if you’ve read my previous piece linked below, you might remember that there are two ways to interact with SageMaker. The API, giving you more control but more boilerplate and the SDK, which hides a lot of complexity from you and gives you less control:

Well, turns out that there was a slight addition to the SageMaker SDK too. If using the regular SageMaker SDK was too much …you can now use JumpStart package to deploy models available in JumpStart on AWS with literally one line of code:

from sagemaker.jumpstart.model import JumpStartModel

my_model = JumpStartModel(model_id="huggingface-llm-falcon-40b-bf16")
predictor = my_model.deploy()

This will automatically do everything on your behalf. Pick a right set of compute, a right container, a right version of a model and so on.

Additionally, SageMaker JumpStart has been made partially available in the regular AWS Console. Previously, to access JumpStart you had to venture into SageMaker Studio and this required you to set up SageMaker Domain. All these steps were simple, but might have put off some people.

Now, you can browse the JumpStart available models straight from the AWS Console in Amazon SageMaker:

This is only a catalog of models though - if you wish to actually deploy them, the UI will open Studio with a Notebook in it that lets you to perform necessary steps.

I wonder if a magic link that launches a CloudFormation template with the model wouldn’t have been a better choice. 🤔

Is it worth a shot? Who is the service targeted at?

I like this direction and believe that SageMaker needed such a thing. There are a lot of AWS users that will inevitably benefit from using SageMaker JumpStart. If you’re a seasoned ML-on-AWS developer, you might not use it. But if you have just started or you only need a quick foundation model up and running, then JumpStart gets you up to speed rapidly.

It is (slightly) similar to AWS ElasticBeanstalk. AWS-savvy people absolutely despise it. People not caring too much about AWS at all, that only want their app in the cloud love it.

Do note however, that most of the provided solutions shall be used only as starting points. Deploy them, play with them, but don’t expect that to be the end of your work. You might sooner than later need to build your own solutions, perhaps based on the foundations or ideas shown in JumpStart.

When it comes to the models in JumpStart, then the idea is generally speaking all right, but one thing comes in mind. The lack of Infrastructure of Code tooling. The models you deploy in SageMaker are often treated more as an infrastructure and handed over to the infrastructure people, who love IaC.

There are some AWS blog posts showcasing how you can deploy JumpStart using CDK, but if you look at the code, they completely ignore JumpStart and just deploy usual SageMaker Inference endpoints. This is understandable - JumpStart is not a real service, but only a UI after all. Still, some of us might get confused.

If you want to deploy a SageMaker Jumpstart model with an IaC tool of choice, you need to just look up how to deploy a SageMaker Inference Endpoint. Then, you need to somehow look up the exact combination of the container, the model and the compute used by SageMaker Jumpstart’s magic one-click solution and you’re good to go.

This means that a person that only played with JumpStart, needs to eventually learn some internals of the SageMaker services underneath, in order to deploy JumpStart goodies in a safe, DevOps friendly way.

---

Also - in some ways, SageMaker JumpStart is similar to SageMaker Projects, a service that only adds an overlay UI to other SageMaker services. See my previous article here with more details:

On a completely unrelated note - I have been just named an AWS Machine Learning Hero. This will enable me to deliver even more juicy useful content. 😊