Skip to content
This repository was archived by the owner on Feb 25, 2025. It is now read-only.

Started identifying fuchsia sdk rom files as binary files in license checker #41900

Closed
gaaclarke wants to merge 3 commits into from
Closed

Started identifying fuchsia sdk rom files as binary files in license checker #41900

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
f1ea39d
Started identifying fuchsia sdk rom files as binary files in license …
gaaclarke May 10, 2023
516b9ad
switchs to extension test
gaaclarke May 10, 2023
5e47198
updated signature
gaaclarke May 10, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion ci/licenses_golden/tool_signature
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
Signature: cb8556d451f3040815fe5ee661d5dbad
Signature: e2ee4730163d78d2bccaf0b4ceb5cbcf

22 changes: 12 additions & 10 deletions tools/licenses/lib/filesystem.dart
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -70,20 +70,21 @@ bool isMultiLicenseNotice(Reader reader) {
}

FileType identifyFile(String name, Reader reader) {
final List<String> pathComponents = path.split(name);
List<int>? bytes;
if ((path.split(name).reversed.take(6).toList().reversed.join('/') == 'third_party/icu/source/extra/uconv/README') || // This specific ICU README isn't in UTF-8.
(path.split(name).reversed.take(6).toList().reversed.join('/') == 'third_party/icu/source/samples/uresb/sr.txt') || // This specific sample contains non-UTF-8 data (unlike other sr.txt files).
(path.split(name).reversed.take(2).toList().reversed.join('/') == 'builds/detect.mk') || // This specific freetype sample contains non-UTF-8 data (unlike other .mk files).
(path.split(name).reversed.take(3).toList().reversed.join('/') == 'third_party/cares/cares.rc')) {
if ((pathComponents.reversed.take(6).toList().reversed.join('/') == 'third_party/icu/source/extra/uconv/README') || // This specific ICU README isn't in UTF-8.
(pathComponents.reversed.take(6).toList().reversed.join('/') == 'third_party/icu/source/samples/uresb/sr.txt') || // This specific sample contains non-UTF-8 data (unlike other sr.txt files).
(pathComponents.reversed.take(2).toList().reversed.join('/') == 'builds/detect.mk') || // This specific freetype sample contains non-UTF-8 data (unlike other .mk files).
(pathComponents.reversed.take(3).toList().reversed.join('/') == 'third_party/cares/cares.rc')) {
return FileType.latin1Text;
}
if (path.split(name).reversed.take(6).toList().reversed.join('/') == 'dart/runtime/tests/vm/dart/bad_snapshot') { // Not any particular format
if (pathComponents.reversed.take(6).toList().reversed.join('/') == 'dart/runtime/tests/vm/dart/bad_snapshot') { // Not any particular format
return FileType.binary;
}
if (path.split(name).reversed.take(9).toList().reversed.join('/') == 'fuchsia/sdk/linux/dart/zircon/lib/src/fakes/handle_disposition.dart' || // has bogus but benign "authors" reference, reported to jamesr@
path.split(name).reversed.take(6).toList().reversed.join('/') == 'third_party/angle/src/common/fuchsia_egl/fuchsia_egl.c' || // has bogus but benign "authors" reference, reported to author and legal team
path.split(name).reversed.take(6).toList().reversed.join('/') == 'third_party/angle/src/common/fuchsia_egl/fuchsia_egl.h' || // has bogus but benign "authors" reference, reported to author and legal team
path.split(name).reversed.take(6).toList().reversed.join('/') == 'third_party/angle/src/common/fuchsia_egl/fuchsia_egl_backend.h') { // has bogus but benign "authors" reference, reported to author and legal team
if (pathComponents.reversed.take(9).toList().reversed.join('/') == 'fuchsia/sdk/linux/dart/zircon/lib/src/fakes/handle_disposition.dart' || // has bogus but benign "authors" reference, reported to jamesr@
pathComponents.reversed.take(6).toList().reversed.join('/') == 'third_party/angle/src/common/fuchsia_egl/fuchsia_egl.c' || // has bogus but benign "authors" reference, reported to author and legal team
pathComponents.reversed.take(6).toList().reversed.join('/') == 'third_party/angle/src/common/fuchsia_egl/fuchsia_egl.h' || // has bogus but benign "authors" reference, reported to author and legal team
pathComponents.reversed.take(6).toList().reversed.join('/') == 'third_party/angle/src/common/fuchsia_egl/fuchsia_egl_backend.h') { // has bogus but benign "authors" reference, reported to author and legal team
return FileType.binary;
}
final String base = path.basename(name);
Expand All @@ -93,7 +94,7 @@ FileType identifyFile(String name, Reader reader) {
return FileType.notPartOfBuild;
} // The ._* files in Mac OS X archives that gives icons and stuff
}
if (path.split(name).contains('cairo')) {
if (pathComponents.contains('cairo')) {
bytes ??= reader();
// "Copyright <latin1 copyright symbol> "
if (hasSubsequence(bytes, <int>[0x43, 0x6f, 0x70, 0x79, 0x72, 0x69, 0x67, 0x68, 0x74, 0x20, 0xA9, 0x20], kMaxSize)) {
Expand Down Expand Up @@ -241,6 +242,7 @@ FileType identifyFile(String name, Reader reader) {
case '.icc': return FileType.binary; // Color profile
case '.swp': return FileType.binary; // Vim swap file
case '.bfbs': return FileType.binary; // Flatbuffers Binary Schema
case '.rom': return FileType.binary;
Copy link
Member

@aam aam May 10, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

.rom-files that are part of fuchsia/sdk/linux/tools should be already skipped by https://github.com/flutter/engine/blob/main/tools/licenses/lib/paths.dart#L40.
Were you able to confirm where other .rom-files are coming?

Copy link
Member Author

@gaaclarke gaaclarke May 10, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea, it's in the linked issue: ../../../fuchsia/sdk/core.tar.gz!core.tar!/tools/arm64/aemu_internal/lib/pc-bios/efi-e1000.rom.

Copy link
Member

@aam aam May 10, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Were you able to confirm where core.tar.gz came from? I don't think it's part of fuchsia sdk.

Copy link
Member Author

@gaaclarke gaaclarke May 10, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nope, I didn't look deeply into it since this is a blocker and it's there somehow, and in no case do we want to do license checks on rom files. I assume its presence is justified somewhere else since the engine code isn't doing it.

Copy link
Member

@aam aam May 10, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would be wary of this being another symptom of some issue with newly introduced license builder. Similar to how 'gclient' was missing, for example.

Copy link
Member Author

@gaaclarke gaaclarke May 10, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(I just verified that srujans PR still needs this fix)

Copy link
Member Author

@gaaclarke gaaclarke May 10, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

actually, every PR needs this

Copy link
Contributor

@Hixie Hixie May 10, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if we're not expecting the core.tar.gz file we should figure out where it is coming from.

Copy link
Contributor

@Hixie Hixie May 10, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(especially a .tar.gz file, which is probably adding more time to the already slow license script process)

Copy link
Member Author

@gaaclarke gaaclarke May 10, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I filed an issue: flutter/flutter#126475

// Archives
case '.zip': return FileType.zip; // ZIP
case '.tar': return FileType.tar; // Tar
Expand Down