Consider a packed archive format for content addressable content

[stevvooe]

With docker save, we typically have a complete image that can be transported independent of a registry. It includes configuration, layers and pointers to the actual content.

For OCI, we should define a similar tree-structured format focused on distributing OCI manifests and constituent content addressable content. The resulting archive can be used to transport complete OCI images based on manifests.

[philips]

I am certain that this is the same spirit of @vbatts issue on #23

[stevvooe]

@philips No, it is quite different. I understood his issue is identifying the actual artifacts involved in an image, whereas this is a format for packing them into a single file.

Using multimedia encoding as an example, imagine #23 as being the video encoding format and this issue as defining the container format, such as mkv or mp4.

[vbatts]

I'm confused. This sounds like what I was (poorly) attempting to arrive at.
With a user story like "As an opensource project, I want to produce an OCI image for the world to consume" what do I do to get this packed archive?

[stevvooe]

@vbatts Perhaps, with the original title. The new title is now "Outline the transportable objects". I saw that as defining the inner components (manifests, layers, image config, etc.).

I was imagining it like this:

[runtime bundle] -> [image spec components] --+----> [pack] --> [transports]
                                              |
                                              +----> [transports]

Are separate these concerns?

[crimeminister]

I am curious as to whether or not OCI is committed to tar as the only export format for images. Is there any value in looking at other options, e.g. 7z, SquashFS, etc.?

[wking]

On Wed, Apr 20, 2016 at 02:28:56PM -0700, Robert Medeiros wrote:

I am curious as to whether or not OCI is committed to tar as the
only export format for images. Is there any value in looking at
other options, e.g. 7z, SquashFS, etc.?

I think the OCI specifying a particular family of serializations and
distribution APIs is independent of whether a discovery registry
requires those serializations and distribution APIs 1. In that
case, OCI work like this issue doesn't exclude alternative
serializations from the discovery-registry ecosystem (although a
particular fetcher may not end up supporting all protocols).

 See the “registered protocol” stuff.

[kamalmarhubi]

@crimeminister I'd be interested in having SquashFS usable for this purpose on Linux. It would allow building the runtime layout via mount which is a nice property.

[vbatts]

@crimeminister I do like the use-cases using squashfs. They enforce read only, the kernel supports them nicely, and they compress well for transport.
Tar archives are also surprisingly widely support and are the current primary packaging format for containers. While I hope to see more innovation in this area, I expect things may stay partially favoring to tar archives.

[stevvooe]

While SquashFS may be an appropriate target, "the kernel" assumes linux.

However, I am not sure that this is relevant to this issue. In this issue, we are more focused on how the target artifacts are shipped. Whether or not they are SquashFS is irrelevant. This may make more sense to discuss in #23, where a SquashFS artifact could be a possible element of an image.

[philips]

I am still not seeing how this discussion is different than #23. Can someone give me a concrete straw man example? @stevvooe? Or maybe we can chat on the call tomorrow about it?

[wking]

On Tue, May 24, 2016 at 01:52:25PM -0700, Brandon Philips wrote:

I am still not seeing how this discussion is different than #23.

Based on 1, my guess is that @stevvooe expected #23 to be about
“what are our fundamental objects?” (e.g. layer blobs, configs,
manifests, tags, …) while this issue would be about “how do we pack
those objects into an archive format?” (e.g. make a tarball with the
blobs at blobs/{algo}:{digest}, the manifests at manifests/{name}, …).
Since #23 is attempting to address both of those issues at once, I
expect this issue can be closed.

On the other hand, if #23 gets refocused on what is getting
distributed (ignoring how that gets packed into a file), I think that
would let it settle down faster. And once it settles down, figuring
out how to pack it into a file will be less contentious (since that's
just about performance, instead of also being about the modeling).