Python and the WSGI web server frameworks within its ecosystem, such as Flask, are often enough to get the job done as far as building simple, synchronous APIs. But what if you need to scale the backend of your app? As an example, what if you have a machine learning model that can scale its computational predictions? Well, you would also need to ensure that the API layer sitting on top of your model can scale as well. In these instances, synchronous WSGI frameworks may not be enough... Starlette to the rescue!
Starlette is an ASGI web server framework that can run completely asynchronously. As such, Starlette can handle requests at scale, solving the problem broached above! In the sections to come, you will learn the following:
Let's get started!
Starlette requires Python version 3.6 or greater and is available to download using pip
.
To install Starlette using pip
, you can run the following command:
1pip install starlette
If you are inside of a virtual environment and wish to install via a requirements.txt
file, you can do so like this:
1# requirements.txt
2starlette==0.13.6
1pip install -r requirements.txt
Once the library is downloaded, you can begin using the framework. As you will see, Starlette has a number of imports to use. Here are a few of the really important ones:
1# app.py
2
3from starlette.applications import Starlette
4from starlette.routing import Route
It is very straightforward to create a Starlette service or API. The below code demonstrates how to instantiate a Starlette app and then create routes for it.
1from starlette.applications import Starlette
2from starlette.routing import Route
3from starlette.responses import PlainTextResponse
4from starlette.responses import JSONResponse
5import uvicorn
6
7from my_model import predict
8
9
10def index(request):
11 return PlainTextResponse("My Index Page!")
12
13def model_stats(request):
14 return JSONResponse({'stats': [1, 0 , 2, 3]})
15
16def model_predict(request):
17 prediction_req = request.json()
18 prediction = predict(prediction_req)
19 return JSONResponse(prediction)
20
21
22routes = [
23 Route('/', index),
24 Route('/stats', model_stats),
25 Route('/predict', model_predict, methods=['POST'])
26]
27
28app = Starlette(debug=True, routes=routes)
29
30if __name__ == "__main__":
31 uvicorn.run(app, host='0.0.0.0', port=8000)
To create a route that services static files, just use Mount
like this:
1from starlette.routing import Mount
2from starlette.staticfiles import StaticFiles
3
4routes = [
5 Route('/', index),
6 Route('/stats', model_stats),
7 Route('/predict', model_predict, methods=['POST'])
8 Mount('/media', app=StaticFiles(directory='media'), name='media')
9]
This is cool, but can you scale your prediction route by making it run asynchronously? Python 3.5 brought the async/await syntax to the language via the new, asyncio
portion of the Python standard library. Using this syntax, you can create completely non-blocking API routes.
Note: Careful--asynchronous code is notoriously hard to debug. To turn on debug mode for asyncio, ensure that the
PYTHONASYNCIODEBUG
environment variable is set.Ex:
PYTHONASYNCIODEBUG=1 python app.py
Here is the new, asynchronous code:
1from starlette.applications import Starlette
2from starlette.routing import Route
3from starlette.responses import PlainTextResponse
4from starlette.responses import JSONResponse
5import uvicorn
6import asyncio
7
8from my_model import predict
9
10
11async def index(request):
12 return PlainTextResponse("My Index Page!")
13
14async def model_stats(request):
15 return JSONResponse({'stats': [1, 0 , 2, 3]})
16
17async def model_predict(request):
18 prediction_req = request.json()
19 prediction = await predict(prediction_req)
20 return JSONResponse(prediction)
21
22
23routes = [
24 Route('/', index),
25 Route('/stats', model_stats),
26 Route('/predict', model_predict, methods=['POST'])
27]
28
29app = Starlette(debug=True, routes=routes)
30
31if __name__ == "__main__":
32 uvicorn.run(app, host='0.0.0.0', port=8000)
In the above example, the asyncio
library was first imported so that you could use async/await. Next, all of the routes were designated as async
routes. Note the model_predict
route which is used to wrap the machine learning model's prediction capability. Now the await
keyword is being used in order to take full advantage of the asynchronous capabilities of the model behind the scenes. Now the API will scale alongside the model!
In this guide, you have learned how to use Starlette to create a Python HTTP service built on the ASGI framework. More importantly, you have discovered how to make your service completely asynchronous so that it can scale with the number of requests coming in.
This guide has only been a brief intro into the world of Starlette and ASGI. There are many more capabilities of this framework in addition to what was discussed here. For more information, and for advanced usage, please check out the Starlette documentation.