Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
472 changes: 472 additions & 0 deletions src/System.Formats.Tar.TestData/GenerateTarFiles.sh

Large diffs are not rendered by default.

91 changes: 91 additions & 0 deletions src/System.Formats.Tar.TestData/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
# System.Formats.Tar.TestData

This package contains test assets for the `System.Formats.Tar` assembly.

## Script

The files can be autogenerated with the `GenerateTarFiles.sh` script. This script needs to be executed in a Linux machine (preferably Ubuntu, since that's where it was tested). The OS should have the Unix `tar` tool available.

The user executing this script must be part of the sudo group (but should _not_ execute the script with `sudo`).

The script is executed without arguments. Make sure to mark the script as executable before executing it:

```bash
$ chmod +x GenerateTarFiles.sh
$ ./GenerateTarFiles.sh
```

## Output

The script generates uncompressed (`*.tar`) and compressed (`*.tar.gz`) files in all the supported formats: V7, Ustar, PAX and GNU.

In the case of GNU, there are two variants supported by the Unix `tar` tool. We generate both, inside the `gnu` and `old_gnu` folders.

In the case of PAX, the Unix `tar` tool can generate archives without and with a Global Extended Attributes entry at the beginning. For that, we generate files inside the `pax` and `pax_gea` folders, respectively.

## Excluded unarchived files

The final `*.nupkg` also includes the `unarchived` folder, which contains all the files that are expected to be found inside the `*.tar` and `*.tar.gz` files. But there are some file categories that are excluded because either NuGet or Visual Studio do not support them:

- Files and directories with very long paths.
- Hard links. They get packged by `*.nupkg` as regular files (this might be expected, since there is no way to differentiate between regular files or hard links).
- Special files: Block device, character device, fifo.

These files are generated by the script on the fly, archived with the `tar` tool, and then deleted at the end of the script execution to prevent getting them packed.

## Expected metadata

The script makes sure all the unarchived files are marked with the following metadata information before archiving them:

- `uname`: `dotnet`
- `gname`: `devdiv`
- `uid`: `7913`
- `gid`: `3579`
- `mode`: `744` - This translates to u+rwx, g+r, o+r

Device files get the following numbers:

- Character devices:
- `devmajor`: `49`
- `devminor`: `86`
- Block devices:
- `devmajor`: `71`
- `devminor`: `53`

PAX files with a Global Extended Attribute entry at the beginning of the archive, get the following custom attribute added:

`key`: `globexthdr.MyGlobalExtendedAttribute`
`value`: `hello`

## Consumption in dotnet/runtime

When running tests, the "current directory" is the location where all these test assets can be found.

To access an unarchived asset file:

```cs
string path = Path.Join(Directory.GetCurrentDirectory(), "unarchived", "file_hardlink", "file.txt");
```

To access a tar file:

```cs
string path = Path.Join(Directory.GetCurrentDirectory(), "tar", "pax_gea", "file_hardlink.tar");
```

To access a `tar.gz` file:

```cs
string path = Path.Join(Directory.GetCurrentDirectory(), "targz", "pax_gea", "file_hardlink.tar.gz");
```

## Test cases

- `file`: Contains a regular file named `file.txt`
- `file_hardlink`: Contains a regular file named `file.txt` and a hard link named `hardlink.txt` whose target is the `file.txt` inode.
- `file_symlink`: Contains a regular file named `file.txt` and a symboliclink named `link.txt` whose target is `file.txt`.
- `folder_file`: Contains a directory named `folder`, which contains a regular file named `file.txt`.
- `folder_file_utf8`: Contains a directory named `földër` which contains a regular file named `áöñ.txt`.
- `folder_subfolder_file`: Contains a directory named `parent`, which contains a directory named `child`, which contains a regular file named `file.txt`.
- `foldersymlink_folder_subfolder_file`: Contains at the root a directory named `parent`, which contains a directory named `child`, which contains a regular file named `file.txt`, and at the root, it also contains a symbolic link named `childlink` that points to the `child` directory.
- `many_small_files`: Contains 10 directories, named from `0` to `9`, and each directory contains 10 regular files, each file named `0.txt` to `9.txt`.
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
<Project Sdk="Microsoft.Build.NoTargets">
<PropertyGroup>
<NoWarn>$(NoWarn);NU5123</NoWarn>
</PropertyGroup>
<ItemGroup>
<None Update="README.md" Pack="False" />
<None Update="GenerateTarFiles.sh" Pack="False" />
</ItemGroup>
</Project>
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file added src/System.Formats.Tar.TestData/targz/v7/file.tar.gz
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
1 change: 1 addition & 0 deletions src/System.Formats.Tar.TestData/unarchived/file/file.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Hello file
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of "Hello" in each file, could this have a single-line description of what scenario the file represents?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The reason why I did it like this was so I could verify the expected file contents with a simple enough message that applied to all tests.

What do you think about putting a description of each test case in the readme instead?

Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Hello file_hardlink
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Hello file_hardlink
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Hello file_symlink
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Hello folder_file
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Hello folder_file_utf8
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Hello folder_subfolder_file
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Hello foldersymlink_folder_subfolder_file
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
many_small_files 0/0
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
many_small_files 0/1
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
many_small_files 0/2
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
many_small_files 0/3
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
many_small_files 0/4
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
many_small_files 0/5
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
many_small_files 0/6
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
many_small_files 0/7
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
many_small_files 0/8
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
many_small_files 0/9
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
many_small_files 1/0
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
many_small_files 1/1
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
many_small_files 1/2
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
many_small_files 1/3
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
many_small_files 1/4
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
many_small_files 1/5
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
many_small_files 1/6
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
many_small_files 1/7
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
many_small_files 1/8
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
many_small_files 1/9
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
many_small_files 2/0
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
many_small_files 2/1
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
many_small_files 2/2
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
many_small_files 2/3
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
many_small_files 2/4
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
many_small_files 2/5
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
many_small_files 2/6
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
many_small_files 2/7
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
many_small_files 2/8
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
many_small_files 2/9
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
many_small_files 3/0
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
many_small_files 3/1
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
many_small_files 3/2
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
many_small_files 3/3
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
many_small_files 3/4
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
many_small_files 3/5
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
many_small_files 3/6
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
many_small_files 3/7
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
many_small_files 3/8
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
many_small_files 3/9
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
many_small_files 4/0
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
many_small_files 4/1
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
many_small_files 4/2
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
many_small_files 4/3
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
many_small_files 4/4
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
many_small_files 4/5
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
many_small_files 4/6
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
many_small_files 4/7
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
many_small_files 4/8
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
many_small_files 4/9
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
many_small_files 5/0
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
many_small_files 5/1
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
many_small_files 5/2
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
many_small_files 5/3
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
many_small_files 5/4
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
many_small_files 5/5
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
many_small_files 5/6
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
many_small_files 5/7
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
many_small_files 5/8
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
many_small_files 5/9
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
many_small_files 6/0
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
many_small_files 6/1
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
many_small_files 6/2
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
many_small_files 6/3
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
many_small_files 6/4
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
many_small_files 6/5
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
many_small_files 6/6
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
many_small_files 6/7
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
many_small_files 6/8
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
many_small_files 6/9
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
many_small_files 7/0
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
many_small_files 7/1
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
many_small_files 7/2
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
many_small_files 7/3
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
many_small_files 7/4
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
many_small_files 7/5
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
many_small_files 7/6
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
many_small_files 7/7
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
many_small_files 7/8
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
many_small_files 7/9
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
many_small_files 8/0
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
many_small_files 8/1
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
many_small_files 8/2
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
many_small_files 8/3
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
many_small_files 8/4
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
many_small_files 8/5
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
many_small_files 8/6
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
many_small_files 8/7
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
many_small_files 8/8
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
many_small_files 8/9
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
many_small_files 9/0
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
many_small_files 9/1
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
many_small_files 9/2
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
many_small_files 9/3
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
many_small_files 9/4
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
many_small_files 9/5
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
many_small_files 9/6
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
many_small_files 9/7
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
many_small_files 9/8
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
many_small_files 9/9
Original file line number Diff line number Diff line change
@@ -1 +1,5 @@
<Project Sdk="Microsoft.Build.NoTargets" />
<Project Sdk="Microsoft.Build.NoTargets">
<PropertyGroup>
<NoWarn>$(NoWarn);NU5123</NoWarn>
</PropertyGroup>
</Project>