Hiram Software Blog

Day Two with the Amazon API Gateway

This post is an update to a lengthy "trip report" from my experience of porting the retail SheetsDB API over to the new Amazon API Gateway on the product's first day.

(I also turned off retail signups for SheetsDB since I was getting an unwanted influx of product tourists. Ironically, if I had been able to port the SheetsDB API over it would not have been an issue. If you would like to try out SheetsDB, send me an email, since the enterprise servers are obviously still running).

In the previous post I identified two functional issues that prevented me from migrating the legacy API over to the new platform, and within 10 days the product team has fixed both issues in production. My hope in documenting these issues was in part to provide feedback in the best way I knew how and also to help save a fellow developer from losing 5 or so hours of his or her time. I must thank the team for hearing the feedback and then acting on it.

The Amazon API Gateway may not be perfect, or it may have other issues, but I still have optimism for the product's direction and team's commitment to succeed. Maybe AWS releases too early (is this even valid criticism?), and maybe the Amazon API Gateway was not ready, but at this pace I would recommend kicking the tires now since it seems this product is getting serious investment and can become what many of us want it to.

Last, I would recommend checking out Stefano's answers on the AWS Developer Forum. He has been great in providing timely feedback and support to me and others. Maybe with better docs his involvement wouldn't be necessary, but my take is the Amazon API Gateway is behaving like a small startup, and I like the romantic idea that even products from large corporations can have souls.

Read below the fold for the specific changes.

Specific changes

The two blocking issues for me were related to mapping input values from the HTTP request to the Lambda event, and rationalizing IAM policies between the principal that invokes the API and the principal that executes the Lambda function. There also was a third clarification that we can chalk up to me misunderstanding (or misreading) the documentation around schema validation.

New json() method

For the first issue, the Amazon API Gateway team released a fix on the next day that added a new function, $input.json(). This function returns the JSON from the input object, selected using the Amazon API Gateway selector syntax. $ seems to mean root object. So, to get the whole input document you would use $input.json('$').

See below for an example using the new method:

API Gateway with new JSON method

And the resulting test:

API Gateway test results with new JSON method

This is a great improvement. See the previous post how worked only 10 days ago.

It still means you will have to map each API method to your Lambda function using this syntax, but I imagine in the future there will be some helpful autocomplete generators or useful defaults. This fix changes the scenario of "How do I map the POST body alongside the path parameters?" from "it's not possible" to "it's possible, and you may not like it for large APIs, but just call $input.json('$')."

New credentials option on context object

For the second issue, I think it's fair to say that it may not be an oversight so much as calling out a seam between two teams. I get the sense that the API Gateway product is getting added to the Lambda product, and so some of these details did not have time to iron out. My first reaction was grounded in the marketing messaging that one could use IAM to lock down Lambda functions and API calls. When it became clear there was an elevation of privilege attack hiding in this seam, my alarms bells went off, and honestly, that's what prompted me to write the post.

The team is releasing a new feature to add the originating IAM principal to the Lambda context object. Ideally I would love to just use the same IAM principal for both automatically, but again this change unlocks some inline authorization checks within one's Lambda function. The fix changes the scenario of "How do I use IAM to restrict access within the Lambda function?" from "it's not possible" to "it's possible, but you have to write code to do it."

Below is an example of the new option highlighted in the input mapping screen:

API Gateway with new JSON method

When I went to test, however, it looks like the functionality may not be there in my region because the context object was unchanged. I have enough confidence that it will be there to put this in the "fixed" column.

Schema not required

In some of the documentation for Lambda there was language around needing to provide schema along side the input mapping. I now understand that I was mistaken. You do not need to define the schema if all you do is map inputs to the Lambda event object.

For those of you who saw a response from Jeff Barr on Hacker News, this is item #3 to which he refers. I was wrong, and I'm sorry for that.

Instead, if you would like the Amazon API Gateway product to automatically generate client code for you, then yes, you will need to define a schema. This is reminiscent of the old WSDL days for those of you who remember it.

I don't plan to adopt WSDL any time soon, but I appreciate why the product team chose to add it. I also appreciate that schemas are optional.


The product team heard the feedback and acted on it within 10 days. They should be proud of this, and it is because of their execution and commitment that I will try porting SheetsDB over to Amazon API again when I next have downtime.

Thank you to them for this product and for their continued hard work.