Since we couldn't manually reproduce that accessing our system, we started trying to fetch the static files with some unexpected headers using Postman.
We finally could quickly reproduce the issue by sending huge Cookies.
Our hypothesis is that, probably, Amazon S3's static website has some configuration that blocks requests with some specific long headers/cookies.
Once we couldn't find a way to configure that on S3, we've added a plugin to our API Gateway that when it was a static file request, it would remove the cookie before redirecting it to S3 (it wasn't required).
After that change, we've monitored our error tracking tool, and the occurrences of http 400 on static website dropped to 0.
与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…