Using GRPC for SDK & microservice API with Python

I am designing a micro-service based system that will be accessed by a python SDK, from an application.

The SDK will be used to access machine learning models hosted as micro-services in a backend system.

I understands that GRPC relies on protobuf, which is lightweight and support streams. We do need to send data vectors to the hosted models, so it is appealing to use protobuf.
The question is more about the GRPC, as I understand it is using HTTP/2.

GRPC seems useful for accessing an API hosted on the backend, however there is also a need to keep an open live connection to receive general update events from the server.
For example I would like to have a “context” where incoming events can arrive, mainly for asynchronous communication, for example after several model invocations are performed, the system might use aggregated data on the backend and send an event when a prediction crosses some threshold.

Hypothetical usage example:

ctx = BackendSystemSDK.connect(APIGatewayHost,port)

ctx.registerCallback(SomeAlertCallbackFunc)

ctx.applyModel(‘model1’,SomeVector1)

ctx.applyModel(‘model2’,SomeVector2)

When program terminates

ctx.close()

I played with GRPC and tested a client and server successfully in python.
However I am not sure about implementing the “context” open connection idea with GRPC.

It is more like a pub/sub concept, however I could not find any examples for local testing such architecture. I did see examples for Google cloud, however it is not relevant as I want to host everything on-premise using kubernetes for scale.

The system should support heavy load of requests, for example processing multiple incoming video streams, per frame, so 24 requests per minute for 20-50 cameras (or more) could easily be the case.

Is GRPC good fit for such scenario?, not only for inter-microservice communication but also for the main API access gateway?

And for the “context”, Maybe it should be implemented as live connection part with websockets or other protocol? I really want to simplify the development and use a single technology.

I could not fully understand from the documentation whether HTTP/2 and GRPC supports a long running bi-directional open connection.

Read more here: Source link