-
Notifications
You must be signed in to change notification settings - Fork 238
IPIP-280: App Conventions for HTTP Gateway Detection #280
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
markg85
wants to merge
32
commits into
ipfs:main
Choose a base branch
from
markg85:master
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from 1 commit
Commits
Show all changes
32 commits
Select commit
Hold shift + click to select a range
d5673ca
Add INTEGRATION document
markg85 9630240
Update INTEGRATION.md
markg85 b23caf5
Merge branch 'ipfs:main' into master
markg85 f956141
rewrote integration document
markg85 8a65c2c
Remove drawio files, we're using mermaid now
markg85 0e8fa0d
follow the template
markg85 81adfd0
linter fixes
markg85 ed0fe22
a few more linter fixes
markg85 0d6b8d6
wip image instead
markg85 0dd13f2
review feedback and updated to new concepts
markg85 d362c62
Refer to the new file as gateways (plural).
markg85 7ce0432
Move integrations document to:
markg85 9e5cd95
template fixes
markg85 4d80040
280. Not 180.
markg85 50b70cf
Add global configuration details
markg85 c6699f7
Little less bold
markg85 32b1cfd
Reword the compatibility section and fix some review feedback.
markg85 3990b3d
Merge branch 'main' into master
markg85 72cdde2
Merge origin/main into markg85-master
lidel 6ee4e19
ipip-280: update and split into specs and ipip
lidel 6290ead
fix: typos and clarify file encoding
lidel 85702ce
fix: clarify single vs multiple URL handling in gateway file
lidel 9d0024f
Merge remote-tracking branch 'origin/main' into pr-280
lidel e657c46
docs: document out-of-scope alternatives
lidel a5334ca
fix: remove stray word in curl section
lidel c65d0a6
fix: simplify and fix OS-specific path handling
lidel 65e6099
chore: update metadata and relax MUST to SHOULD
lidel 7acc773
fix: clarify gateway URL requirements
lidel e8581ce
fix: capitalize SHOULD for RFC 2119 consistency
lidel 81e97eb
chore: add libp2p-gateway to integratingHttpGateways
lidel 22779d6
chore: mark IPIP as ratified (implemented since 2022)
lidel b4e73a2
docs: add official ffmpeg docs reference
lidel File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,46 @@ | ||
| #  Integration <!-- omit in toc --> | ||
|
|
||
| **Author(s)**: | ||
| - [Mark Gaiser](https://github.com/markg85/) | ||
|
|
||
| * * * | ||
|
|
||
| **Abstract** | ||
| This integration spec defines the recommended way for a third party applications to integrate IPFS support in their application and thereby gaining easy access to resources stored on the IPFS platform. | ||
markg85 marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
|
||
| Integration here has 2 different meanings. | ||
|
|
||
| 1. The implementing application can handle IPFS resources with the `ipfs` and `ipns` protocols. As an example, the implementing application should be able to handle a url in this format `ipfs://QmbGtJg23skhvFmu9mJiePVByhfzu5rwo74MEkVDYAmF5T` which in this case would give you the big buck bunny video. Likewise a url in the `ipns` should be handled too. Making these protocols usable is left up to the specific application implementing this support. | ||
markg85 marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
|
||
| 2. One way to handle the protocols is to use the [http gateway](https://docs.ipfs.io/concepts/ipfs-gateway/). This document describes how to determine that gateway to use. | ||
markg85 marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
|
||
| ## Decision tree | ||
| The below decision tree defines the decisions to be made in order to choose the proper gateway. Do note that the tree, when implementing this logic, is more elaborate then this tree makes you think it is. Validation here is left out of the tree but should be added in a local implementation. | ||
|
|
||
| <img src="img/gateway_decision_tree.drawio.png" width="503" height="716" /> | ||
markg85 marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
|
||
| ### Gateway from command argument | ||
| An application can opt to support a command line option to provide a gateway. A user should not _need_ to provide this and should therefore be considered optional. However, if a user does provide this option then it should overrule any other gateway detection and be used as the gateway to use. An example implementation that is doing this is ffmpeg with the ffplay utility. It allows the `-gateway` argument which by default is empty but can be set like: `-gateway http://127.0.0.1:8080` and would then be used to handle `ipfs://<cid>` or `ipns://<cid>`. | ||
|
|
||
| ### Gateway from IPFS_GATEWAY environment variable | ||
| When there is no command line argument, the `IPFS_GATEWAY` environment is next. If it contains a value, it should be used as gateway. | ||
|
|
||
| ### Gateway file from IPFS_PATH environment variable or home folder ipfs data | ||
markg85 marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| If the `IPFS_PATH` environment variable is defined, it should point to the ipfs data folder. If this environment variable isn't defined then an attempt should be made to see if `$USER/.ipfs` exists and consider that to be the IPFS data folder when it does. | ||
|
|
||
| If this turns out to be an existing path the existence of the `gateway` file in that path should be checked for. If gateway file exists then the first line should be considered to be the full url to the local IPFS gateway. | ||
|
|
||
| Do note that this gateway file logic relies on the future implementation of having the gateway file be auto-generated. It's specced [here](https://github.com/ipfs/go-ipfs/issues/8847) and is approved to be implemented. Implementers of this spec should act as if that gateway file already exists! | ||
|
|
||
| ## Validation and fallback gateway | ||
| All of the above describes what should ideally be done. Validation of any kind is up to the application implementing IPFS support. To provide an "always working" feeling, implementers are recommended to use a fallback gateway and they should pick `dweb.link`. That gateway is maintained by Protocol Labs and is allowed to be used for this purpose. | ||
markg85 marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
|
||
| ## Current implementations | ||
| ### ffmpeg | ||
| As of ffmpeg 5.1, it implements this logic. The source for it's implementation can be found [here](https://github.com/FFmpeg/FFmpeg/blob/master/libavformat/ipfsgateway.c). | ||
|
|
||
| ### mpv | ||
| mpv itself isn't doing any of this, it relies on the ffmpeg implementation. | ||
|
|
||
| ### curl | ||
| The implementation for curl is currently a work in progress but it too follows the steps as outlined in the decision tree. | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| <mxfile host="app.diagrams.net" modified="2022-05-12T19:53:07.883Z" agent="5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36" etag="P1leR3BdtQYfv4BkFvw_" version="18.0.3" type="device"><diagram name="Page-1" id="10a91c8b-09ff-31b1-d368-03940ed4cc9e">7VrbcqM4EP0aHpPiDnm0PblM1c5udjO1s5mXLdkI0EZGHiHH9n79SEZchLDj2M7gSlzlKtONaKD7NOdIYDij6fKWgln6hUQQG7YZLQ3nk2HblmvbhviZ0arwBL5TOBKKIjmodjyg/6F0mtI7RxHMlYGMEMzQTHVOSJbBCVN8gFKyUIfFBKtnnYEEao6HCcC69xuKWCq9lmnWO+4gSlJ56tCTO8Zg8pRQMs/k+TKSwWLPFJRh5NA8BRFZNFzOteGMKCGs2JouRxCLtJYZK4672bC3umQKM7bLAb4dXjlWGE7MGASOb1/ICM8Az2UaUpCLYgAGF2C1Tm0yn4rwxQ2wVZmvRYoYfJiBibAXHBOGM0zZFHPL4pvrhMBIWtWNCwODMcTDKmsjggnlu9Z540MZJU9VCcT4mGTsBkwRFsj6G9IIZEC6JYwsnskhwCjJuDHhVwt5wKGenfJ2IWVw2XDJbN1CMoWMituWe90SnRLUrjQXDYSUxU0b4PClD0hQJlXkujp8QxZox2I5WrEM28dM5GwGMr6diO3P9zcP/94Ovl5/GzyKdoQxyngd5MgxLceVHn4ZzcM/aJ19tc5W0GehXa3QvxOtLnXiTbUiL6UdUDYQz8uW7waJC1wHW9sy5R63YRa1juCexnhuNUb/BxlbSRvMGeEuQllKEpIB/BshsxIr+wFgU7FzMqcTuP05x+8rgWzDGNleMFJ4YiNkLsxLxzRtFTYyBoUYMPSsMksXRGT0e4LEE7YaQuI4h0zDUHUR+8HK02D1yAn3jKvywcIRRVf/iHNfeqX5WF0KNz4tFWslraNBkvnJ3fj7n4PPz/Fo8N1Nfvx14ZXjXgNL70pB5YVlnTIsS4pt4LKWHzElU/6nktq+DGUewlB2vwzlBCpDOXafDGXpwrGuGRIKMkY0F9jBSKhhE2VrH4Yfs3ptHemHvVbPPzPBsRWGs4PCKEHwime57Vtu62F+0hrDCs7a9XQ0RgcoOzWG/WpcWvaVisuy8CcBy21CapvMOHyxo2eS0hipAzMby2oHPU6Ct+FylxWP+8HXu4+03HFQpT2/RzHZWWl9beusSBTeWCJW0IbtSfNRRhDbNWsIY2U0KGRvsmmxwYvss62DX2Qg/5SEUD1Gnr997krOlzGKPMijtgRqN55ntgIVidICHYsDzyuLO3ZZ8C67zNvaZbzJQitUJYD963rMC47TY4HTb4/py6wgz7msNAShund/fLnmYS/RLNb5jUdHs1y0wQsy5AgawHfVNLm2t5sGCN9KA+irEl1qj///mIsXpsNKvpeOaonJ5A2as7xcd5qJ97BnRbh1gSpsNY3TtyIMO9BwJqo2Ue3DUkFfNOUdTQyGYfsVh38YUa06D9iFtlqN4+9LW76pBgranfXGtHWl9dt5BvYeZmC+3nQbXuec7PTLK5/9h/aY66vyNvjF0rDjVed5/vUO5l8dPdZd/1NaV+9YnPCORGWtGZjfnjK8dZvpS/0xwFh8qGk0v21cv6iOFnB8iVH2pPXhh1j099r65Q3fTHOz/ta1KHb9LbFz/RM=</diagram></mxfile> |
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.