Skip to content
This repository was archived by the owner on Sep 14, 2025. It is now read-only.

Website Routing Rules#490

Merged
kherock merged 15 commits into
jamhall:masterfrom
jagregory:website-routing-rules
Jul 31, 2019
Merged

Website Routing Rules#490
kherock merged 15 commits into
jamhall:masterfrom
jagregory:website-routing-rules

Conversation

@jagregory

@jagregory jagregory commented Jul 17, 2019

Copy link
Copy Markdown
Contributor

Hey, I need support for routing rules for my setup so I figured I'd give it a shot at implementing them. This is a work in progress, but I figured I'd open the PR early.

The example I have implemented so far is what I personally need, but it isn't the simplest scenario. I wanted to prove out that I could get this working for my case before implementing anything else. I only have one test at the moment, but I'll expand them as I go.

Any feedback is welcome. I'll update this PR as I progress.

TODO

  • Rules with no Condition
  • KeyPrefixEquals only conditional
  • HttpErrorCodeReturnedEquals only conditional
  • KeyPrefixEquals and HttpErrorCodeReturnedEquals conditonal
  • Redirect with default hostname
  • Redirect with custom hostname from HostName value
  • Redirect with default protocol
  • Redirect with protocol from Protocol value
  • Redirect with key prefix replaced by ReplaceKeyPrefixWith value
  • Redirect with key replaced by ReplaceKeyWith value
  • Redirect with custom response code specified with HttpRedirectCode
  • Redirect with default status code 302? test what S3 does
  • Validation of config
  • HttpErrorCodeReturnedEquals for a non-404 status code

@jagregory

Copy link
Copy Markdown
Contributor Author

I think this is good to go now. I've implemented all the variations of Conditions and Redirect rules I can see in the docs, and they're thoroughly tested along with the config validation.

The only thing I haven't added support for is HttpErrorCodeReturnedEquals matching error codes other than 404. I think this might be too hard to do/need some refactoring first, as it'll require the routing code to be able to handle all errors first before they reach the proper error handler. I'm calling this too hard for me right now.

If you have time, a review would be great.

@kherock

kherock commented Jul 18, 2019

Copy link
Copy Markdown
Collaborator

Wow, thanks so much for this PR! I love the approach, these changes look like they'll fit right in.

I'll have to take a closer look this weekend. It looks like you were diligent with all the test cases, so as long as they look right to me I think we should be able to merge this quickly.

@jagregory

jagregory commented Jul 20, 2019

Copy link
Copy Markdown
Contributor Author

Not a problem! Thanks for the project :)

I've just pushed another commit because I realised elements that contain XML entities weren't being decoded, which messes things up if you're rewriting URLs to query strings. I added a new test case to cover this.

@jagregory jagregory force-pushed the website-routing-rules branch from dbb3a31 to 2a3b792 Compare July 20, 2019 00:23
Needed for WebsiteConfiguration where you're doing a
ReplaceKeyPrefixWith to something that puts the Key into a QueryString.

e.g. `/thumbnail/image.png` is transformed to `/resize?size=thumbnail&key=image.png`

The & in the query strings have to be encoded in the website config,
and without this change they stay encoded and break things.
@jagregory jagregory force-pushed the website-routing-rules branch from 2a3b792 to 633a9e4 Compare July 20, 2019 00:26

@kherock kherock left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jagregory Thanks again for the quality PR. I feel like everything is actually already in place to check off the last subtask for HttpErrorCodeReturnedEquals, but let me know if I'm missing something.

Comment thread lib/middleware/website.js

if (config.routingRules) {
for (const routingRule of config.routingRules) {
if (routingRule.shouldRedirect(key, 404)) {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add a comment here describing that we're deviating from actual S3 behavior by only handling 404s?

Also, is there any reason you can't attempt to read err.status || 500 here? The existing implementation was initially written to only add special handling for the NoSuchKey error, but in this case, I think it should be possible to perform this routing rule check before the NoSuchKey check on L81, grabbing whatever error code is set.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I attempted this before, because I agree it should be possible; however, I couldn't get a testcase working to simulate the behaviour. I can give it another shot, but if you have any tips on how to get the server into a state where it'll respond with a different status code that'd be very helpful.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After a bit more experimenting with S3, I still can’t find an actual scenario where I can get it to redirect with anything other than a 404. S3 doesn’t do blanket redirections, I assume depending on where the error occurs will decide whether the routing rules kick in.

  • S3 responds with a 403 when a file isn’t public accessible = No redirect
  • S3 responds with a 405 when you try accessing a key with an unexpected method (e.g. a PUT) = No redirect

I’m a bit stumped. I can’t find any examples of people doing anything other than 404.

So I'm a bit wary of allowing any error to trigger a redirect in s3rver, as that's not the same behaviour as S3, but it's kinda hard to determine what actually is the expected behaviour! I'll add a comment for now.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

More experimenting yielded: The provided HTTP error code (200) is not valid. Valid codes are 4XX or 5XX, so I'll add this as an extra validation check too.

Comment thread lib/models/routing-rule.js Outdated
Comment thread lib/models/routing-rule.js Outdated
Comment thread test/test.js Outdated
@jagregory

Copy link
Copy Markdown
Contributor Author

@kherock I think this is probably as good as it's going to get right now, is there anything else outstanding to get this merged? Thanks!

@kherock

kherock commented Jul 31, 2019

Copy link
Copy Markdown
Collaborator

apologies, I was on vacation last week and forgot to take care of this. The changes look great, I want to take care of a couple other issues first, but expect a release soon!

@kherock kherock merged commit 6d7c1d1 into jamhall:master Jul 31, 2019
@jagregory

Copy link
Copy Markdown
Contributor Author

Thanks!

@kherock

kherock commented Aug 5, 2019

Copy link
Copy Markdown
Collaborator

v3.3.0 is released. Thank you for contributing!

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants