Skip to content

Add erofs-util API support for overlaybd #332

Merged
BigVan merged 1 commit intocontainerd:mainfrom
salvete:erofs-io-incremental
Jun 27, 2024
Merged

Add erofs-util API support for overlaybd #332
BigVan merged 1 commit intocontainerd:mainfrom
salvete:erofs-io-incremental

Conversation

@salvete
Copy link
Member

@salvete salvete commented Jun 24, 2024

What this PR does / why we need it:

Previously, erofs-utils was integrated into overlaybd by using the mkfs.erofs executable file. It's not optimial since raw data needs to be dumped first and output data needs to be write into overlaybd then.

This adds EROFS API support for overlaybd.

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #

Please check the following list:

  • Does the affected code have corresponding tests, e.g. unit test, E2E test?
  • Does this change require a documentation update?
  • Does this introduce breaking changes that would require an announcement or bumping the major version?
  • Do all new files have an appropriate license header?

@hsiangkao
Copy link
Member


erofs_inode_manager_init();

root = erofs_mkfs_alloc_root(sbi);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's rename this as erofs_rebuild_make_root(sbi);

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.


#define EROFS_UNIMPLEMENTED 1

struct erofs_sector {
Copy link
Member

@hsiangkao hsiangkao Jun 25, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we could just use a std::map<addr, struct liberofs_inmem_sector> and stl::set for dirty bitmap?

struct liberofs_inmem_sector {
     char data[SECTOR_SIZE];
};

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

public:
LibErofs(photon::fs::IFile *target);
~LibErofs();
int extract_tar(photon::fs::IFile *source, uint64_t blksize, bool meta_only, bool first_layer);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

uint64_t blksize should be moved to the constructor.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

@salvete salvete force-pushed the erofs-io-incremental branch from afeb875 to 528d634 Compare June 25, 2024 09:22
Previously, erofs-utils was integrated into overlaybd by
using the `mkfs.erofs` executable file. It's not optimial
since raw data needs to be dumped first and output data
needs to be write into overlaybd then.

This commit introduces the I/O API in tarerofs, that the data
stream of the:
- output image
- input tar file
can be controlled by the I/O manager.

This also adds block read/write cache support for IFile to
improve the efficiency of making file system images.

Signed-off-by: Hongzhen Luo <hongzhen@linux.alibaba.com>
@salvete salvete force-pushed the erofs-io-incremental branch from 528d634 to 652bc76 Compare June 26, 2024 07:02
Copy link
Member

@hsiangkao hsiangkao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PTAL thanks @BigVan

@BigVan
Copy link
Member

BigVan commented Jun 27, 2024

LGTM

@BigVan BigVan merged commit 6c1bf35 into containerd:main Jun 27, 2024
@BigVan
Copy link
Member

BigVan commented Jun 27, 2024

will build a new release after improving CI test

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants