-
Notifications
You must be signed in to change notification settings - Fork 60
Add System.Formats.Tar test assets #238
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
93c4ef9
d647bf6
1919299
ec0aa32
fd224c2
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Large diffs are not rendered by default.
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,91 @@ | ||
| # System.Formats.Tar.TestData | ||
|
|
||
| This package contains test assets for the `System.Formats.Tar` assembly. | ||
|
|
||
| ## Script | ||
|
|
||
| The files can be autogenerated with the `GenerateTarFiles.sh` script. This script needs to be executed in a Linux machine (preferably Ubuntu, since that's where it was tested). The OS should have the Unix `tar` tool available. | ||
|
|
||
| The user executing this script must be part of the sudo group (but should _not_ execute the script with `sudo`). | ||
|
|
||
| The script is executed without arguments. Make sure to mark the script as executable before executing it: | ||
|
|
||
| ```bash | ||
| $ chmod +x GenerateTarFiles.sh | ||
| $ ./GenerateTarFiles.sh | ||
| ``` | ||
|
|
||
| ## Output | ||
|
|
||
| The script generates uncompressed (`*.tar`) and compressed (`*.tar.gz`) files in all the supported formats: V7, Ustar, PAX and GNU. | ||
|
|
||
| In the case of GNU, there are two variants supported by the Unix `tar` tool. We generate both, inside the `gnu` and `old_gnu` folders. | ||
|
|
||
| In the case of PAX, the Unix `tar` tool can generate archives without and with a Global Extended Attributes entry at the beginning. For that, we generate files inside the `pax` and `pax_gea` folders, respectively. | ||
|
|
||
| ## Excluded unarchived files | ||
|
|
||
| The final `*.nupkg` also includes the `unarchived` folder, which contains all the files that are expected to be found inside the `*.tar` and `*.tar.gz` files. But there are some file categories that are excluded because either NuGet or Visual Studio do not support them: | ||
|
|
||
| - Files and directories with very long paths. | ||
| - Hard links. They get packged by `*.nupkg` as regular files (this might be expected, since there is no way to differentiate between regular files or hard links). | ||
| - Special files: Block device, character device, fifo. | ||
|
|
||
| These files are generated by the script on the fly, archived with the `tar` tool, and then deleted at the end of the script execution to prevent getting them packed. | ||
|
|
||
| ## Expected metadata | ||
|
|
||
| The script makes sure all the unarchived files are marked with the following metadata information before archiving them: | ||
|
|
||
| - `uname`: `dotnet` | ||
| - `gname`: `devdiv` | ||
| - `uid`: `7913` | ||
| - `gid`: `3579` | ||
| - `mode`: `744` - This translates to u+rwx, g+r, o+r | ||
|
|
||
| Device files get the following numbers: | ||
|
|
||
| - Character devices: | ||
| - `devmajor`: `49` | ||
| - `devminor`: `86` | ||
| - Block devices: | ||
| - `devmajor`: `71` | ||
| - `devminor`: `53` | ||
|
|
||
| PAX files with a Global Extended Attribute entry at the beginning of the archive, get the following custom attribute added: | ||
|
|
||
| `key`: `globexthdr.MyGlobalExtendedAttribute` | ||
| `value`: `hello` | ||
|
|
||
| ## Consumption in dotnet/runtime | ||
|
|
||
| When running tests, the "current directory" is the location where all these test assets can be found. | ||
|
|
||
| To access an unarchived asset file: | ||
|
|
||
| ```cs | ||
| string path = Path.Join(Directory.GetCurrentDirectory(), "unarchived", "file_hardlink", "file.txt"); | ||
| ``` | ||
|
|
||
| To access a tar file: | ||
|
|
||
| ```cs | ||
| string path = Path.Join(Directory.GetCurrentDirectory(), "tar", "pax_gea", "file_hardlink.tar"); | ||
| ``` | ||
|
|
||
| To access a `tar.gz` file: | ||
|
|
||
| ```cs | ||
| string path = Path.Join(Directory.GetCurrentDirectory(), "targz", "pax_gea", "file_hardlink.tar.gz"); | ||
| ``` | ||
|
|
||
| ## Test cases | ||
|
|
||
| - `file`: Contains a regular file named `file.txt` | ||
| - `file_hardlink`: Contains a regular file named `file.txt` and a hard link named `hardlink.txt` whose target is the `file.txt` inode. | ||
| - `file_symlink`: Contains a regular file named `file.txt` and a symboliclink named `link.txt` whose target is `file.txt`. | ||
| - `folder_file`: Contains a directory named `folder`, which contains a regular file named `file.txt`. | ||
| - `folder_file_utf8`: Contains a directory named `földër` which contains a regular file named `áöñ.txt`. | ||
| - `folder_subfolder_file`: Contains a directory named `parent`, which contains a directory named `child`, which contains a regular file named `file.txt`. | ||
| - `foldersymlink_folder_subfolder_file`: Contains at the root a directory named `parent`, which contains a directory named `child`, which contains a regular file named `file.txt`, and at the root, it also contains a symbolic link named `childlink` that points to the `child` directory. | ||
| - `many_small_files`: Contains 10 directories, named from `0` to `9`, and each directory contains 10 regular files, each file named `0.txt` to `9.txt`. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,9 @@ | ||
| <Project Sdk="Microsoft.Build.NoTargets"> | ||
| <PropertyGroup> | ||
| <NoWarn>$(NoWarn);NU5123</NoWarn> | ||
| </PropertyGroup> | ||
| <ItemGroup> | ||
| <None Update="README.md" Pack="False" /> | ||
| <None Update="GenerateTarFiles.sh" Pack="False" /> | ||
| </ItemGroup> | ||
| </Project> | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| Hello file | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Instead of "Hello" in each file, could this have a single-line description of what scenario the file represents?
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The reason why I did it like this was so I could verify the expected file contents with a simple enough message that applied to all tests. What do you think about putting a description of each test case in the readme instead? |
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| Hello file_hardlink |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| Hello file_hardlink |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| Hello file_symlink |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| file.txt |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| Hello folder_file |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| Hello folder_file_utf8 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| Hello folder_subfolder_file |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| parent/child |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| Hello foldersymlink_folder_subfolder_file |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| many_small_files 0/0 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| many_small_files 0/1 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| many_small_files 0/2 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| many_small_files 0/3 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| many_small_files 0/4 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| many_small_files 0/5 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| many_small_files 0/6 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| many_small_files 0/7 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| many_small_files 0/8 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| many_small_files 0/9 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| many_small_files 1/0 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| many_small_files 1/1 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| many_small_files 1/2 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| many_small_files 1/3 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| many_small_files 1/4 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| many_small_files 1/5 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| many_small_files 1/6 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| many_small_files 1/7 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| many_small_files 1/8 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| many_small_files 1/9 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| many_small_files 2/0 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| many_small_files 2/1 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| many_small_files 2/2 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| many_small_files 2/3 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| many_small_files 2/4 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| many_small_files 2/5 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| many_small_files 2/6 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| many_small_files 2/7 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| many_small_files 2/8 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| many_small_files 2/9 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| many_small_files 3/0 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| many_small_files 3/1 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| many_small_files 3/2 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| many_small_files 3/3 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| many_small_files 3/4 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| many_small_files 3/5 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| many_small_files 3/6 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| many_small_files 3/7 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| many_small_files 3/8 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| many_small_files 3/9 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| many_small_files 4/0 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| many_small_files 4/1 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| many_small_files 4/2 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| many_small_files 4/3 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| many_small_files 4/4 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| many_small_files 4/5 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| many_small_files 4/6 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| many_small_files 4/7 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| many_small_files 4/8 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| many_small_files 4/9 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| many_small_files 5/0 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| many_small_files 5/1 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| many_small_files 5/2 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| many_small_files 5/3 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| many_small_files 5/4 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| many_small_files 5/5 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| many_small_files 5/6 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| many_small_files 5/7 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| many_small_files 5/8 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| many_small_files 5/9 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| many_small_files 6/0 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| many_small_files 6/1 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| many_small_files 6/2 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| many_small_files 6/3 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| many_small_files 6/4 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| many_small_files 6/5 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| many_small_files 6/6 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| many_small_files 6/7 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| many_small_files 6/8 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| many_small_files 6/9 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| many_small_files 7/0 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| many_small_files 7/1 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| many_small_files 7/2 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| many_small_files 7/3 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| many_small_files 7/4 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| many_small_files 7/5 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| many_small_files 7/6 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| many_small_files 7/7 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| many_small_files 7/8 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| many_small_files 7/9 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| many_small_files 8/0 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| many_small_files 8/1 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| many_small_files 8/2 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| many_small_files 8/3 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| many_small_files 8/4 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| many_small_files 8/5 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| many_small_files 8/6 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| many_small_files 8/7 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| many_small_files 8/8 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| many_small_files 8/9 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| many_small_files 9/0 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| many_small_files 9/1 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| many_small_files 9/2 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| many_small_files 9/3 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| many_small_files 9/4 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| many_small_files 9/5 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| many_small_files 9/6 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| many_small_files 9/7 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| many_small_files 9/8 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| many_small_files 9/9 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1 +1,5 @@ | ||
| <Project Sdk="Microsoft.Build.NoTargets" /> | ||
| <Project Sdk="Microsoft.Build.NoTargets"> | ||
| <PropertyGroup> | ||
| <NoWarn>$(NoWarn);NU5123</NoWarn> | ||
| </PropertyGroup> | ||
| </Project> |
Uh oh!
There was an error while loading. Please reload this page.