We have successfully trained our model and deployed with model server. However, we would like to add some authentication proxy.
We tried with AWS API Gateway but seems to add an unnecessary overhead to the response time. Making a request to the model server via AWS ALB takes about 300ms for a response while with our API Gateway implementation, it takes 6seconds. This is not good enough.
Anyone with ideas on the best way to approach this?
Thanks in advance