Zachary Bennett

Building Async Python Services with Starlette

Zachary Bennett

Aug 5, 2020
5 Min read
9,167 Views

Aug 5, 2020
5 Min read
9,167 Views

Data

Data Analytics

Machine Learning

Python

Introduction

Python and the WSGI web server frameworks within its ecosystem, such as Flask, are often enough to get the job done as far as building simple, synchronous APIs. But what if you need to scale the backend of your app? As an example, what if you have a machine learning model that can scale its computational predictions? Well, you would also need to ensure that the API layer sitting on top of your model can scale as well. In these instances, synchronous WSGI frameworks may not be enough... Starlette to the rescue!

Starlette is an ASGI web server framework that can run completely asynchronously. As such, Starlette can handle requests at scale, solving the problem broached above! In the sections to come, you will learn the following:

How to get up and running with Starlette

How to add routes and serve static files

How to make scale your service asynchronously

Let's get started!

Installation

Starlette requires Python version 3.6 or greater and is available to download using pip.

To install Starlette using pip, you can run the following command:

1pip install starlette

bash

If you are inside of a virtual environment and wish to install via a requirements.txt file, you can do so like this:

1# requirements.txt
2starlette==0.13.6

python

1pip install -r requirements.txt

bash

Once the library is downloaded, you can begin using the framework. As you will see, Starlette has a number of imports to use. Here are a few of the really important ones:

1# app.py
2
3from starlette.applications import Starlette
4from starlette.routing import Route

python

Creating a Starlette Service

It is very straightforward to create a Starlette service or API. The below code demonstrates how to instantiate a Starlette app and then create routes for it.

1from starlette.applications import Starlette
2from starlette.routing import Route
3from starlette.responses import PlainTextResponse
4from starlette.responses import JSONResponse
5import uvicorn
6
7from my_model import predict
8
9
10def index(request):
11    return PlainTextResponse("My Index Page!")
12
13def model_stats(request):
14    return JSONResponse({'stats': [1, 0 , 2, 3]})
15
16def model_predict(request):
17    prediction_req = request.json()
18    prediction = predict(prediction_req)
19    return JSONResponse(prediction)
20
21
22routes = [
23    Route('/', index),
24    Route('/stats', model_stats),
25    Route('/predict', model_predict, methods=['POST'])
26]
27
28app = Starlette(debug=True, routes=routes)
29
30if __name__ == "__main__":
31    uvicorn.run(app, host='0.0.0.0', port=8000)

python

To create a route that services static files, just use Mount like this:

1from starlette.routing import Mount
2from starlette.staticfiles import StaticFiles
3
4routes = [
5    Route('/', index),
6    Route('/stats', model_stats),
7    Route('/predict', model_predict, methods=['POST'])
8    Mount('/media', app=StaticFiles(directory='media'), name='media')
9]

python

Making Your Service Asynchronous

This is cool, but can you scale your prediction route by making it run asynchronously? Python 3.5 brought the async/await syntax to the language via the new, asyncio portion of the Python standard library. Using this syntax, you can create completely non-blocking API routes.

Note: Careful--asynchronous code is notoriously hard to debug. To turn on debug mode for asyncio, ensure that the PYTHONASYNCIODEBUG environment variable is set.
Ex: PYTHONASYNCIODEBUG=1 python app.py

Here is the new, asynchronous code:

1from starlette.applications import Starlette
2from starlette.routing import Route
3from starlette.responses import PlainTextResponse
4from starlette.responses import JSONResponse
5import uvicorn
6import asyncio
7
8from my_model import predict
9
10
11async def index(request):
12    return PlainTextResponse("My Index Page!")
13
14async def model_stats(request):
15    return JSONResponse({'stats': [1, 0 , 2, 3]})
16
17async def model_predict(request):
18    prediction_req = request.json()
19    prediction = await predict(prediction_req)
20    return JSONResponse(prediction)
21
22
23routes = [
24    Route('/', index),
25    Route('/stats', model_stats),
26    Route('/predict', model_predict, methods=['POST'])
27]
28
29app = Starlette(debug=True, routes=routes)
30
31if __name__ == "__main__":
32    uvicorn.run(app, host='0.0.0.0', port=8000)

python

In the above example, the asyncio library was first imported so that you could use async/await. Next, all of the routes were designated as async routes. Note the model_predict route which is used to wrap the machine learning model's prediction capability. Now the await keyword is being used in order to take full advantage of the asynchronous capabilities of the model behind the scenes. Now the API will scale alongside the model!

Conclusion

In this guide, you have learned how to use Starlette to create a Python HTTP service built on the ASGI framework. More importantly, you have discovered how to make your service completely asynchronous so that it can scale with the number of requests coming in.

This guide has only been a brief intro into the world of Starlette and ASGI. There are many more capabilities of this framework in addition to what was discussed here. For more information, and for advanced usage, please check out the Starlette documentation.

LEARN MORE