[Feature]: Flexible filters for sourceDependencies

[tommyk-gears]

Brief description

Allow a more flexible inclusion of sourceDependencies, similar to what maven-dependency-plugin provides with exclude* and include* parameters. Specifically I would like to be able to do e.g.;

<configuration>
    <sourceDependencies>
        <includeGroupIds>com.mycompany</includeGroupIds>
    </sourceDependencies>
</configuration>

And as a result all .proto files in all direct/transitive project dependencies with groupId that starts with com.mycompany (note that GroupIdFilter matches on prefix).

I imagine maven-common-artifact-filters can/should be used to implement this kind of feature in this plugin to avoid duplication and also make sure that the functionality here is aligned with what users are familiar with from the maven-dependency-plugin.

Aims and goals

At our company we have many artifacts with .proto-files and also generated java classes but we aim to stop using the generated classes inside dependencies and instead have each service generate its own classes for all internal protocol buffers. The reason for this is that it is currently extremely painful to upgrade protoc over versions that introduces incompatible changes in the generated sources. If each service generate its own java classes, then each service can use whatever protoc version it wants without breaking anything else.

Because of the sheer amount of transitive dependencies that contains .proto-files it is not really feasible to list them all explicitly in the services pom files.

Workarounds

One workaround may be to use maven-dependency-plugin:unpack-dependencies first to put the .proto files in a local directory and then adding that directory to sourceDirectories in the config of this plugin.

Would you like to contribute to this feature?

• Yes, I'd like to contribute this change myself!

[ascopes]

👍

Out of curiosity, does the use of the includes and excludes parameters on source paths provide a workaround temporarily for you? That assumes all your proto files are following a directory hierarchy similar to how Java packages are laid out, which I appreciate may not be the case in your use case.

I will see what I can come up with when I have some free time and assess the further feasibility of this.

[tommyk-gears]

If I use maven-dependency-plugin to unpack .proto files from dependencies I get away with this;

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-dependency-plugin</artifactId>
    <executions>
        <execution>
            <id>unpack-proto-dependencies</id>
            <goals>
                <goal>unpack-dependencies</goal>
            </goals>
            <phase>generate-sources</phase>
            <configuration>
                <includeGroupIds>com.mycompany</includeGroupIds>
                <includes>**/*.proto</includes>
                <outputDirectory>${project.build.directory}/dependency/proto</outputDirectory>
            </configuration>
        </execution>
    </executions>
</plugin>

And then I just include the sourceDirectory, no includes/excludes needed;

<plugin>
    <groupId>io.github.ascopes</groupId>
    <artifactId>protobuf-maven-plugin</artifactId>
    <version>3.3.0</version>
    <configuration>
        <protocVersion>3.25.5</protocVersion>
        <sourceDirectories>
            <sourceDirectory>${project.basedir}/src/main/proto</sourceDirectory>
            <sourceDirectory>${project.build.directory}/dependency/proto</sourceDirectory>
        </sourceDirectories>
    </configuration>

    <executions>
        <execution>
            <goals>
                <goal>generate</goal>
            </goals>
        </execution>
    </executions>
</plugin>

[ascopes]

In your case, do you inherit the dependencies from a parent POM usually?

Just want to be able to weigh out the pros and cons of supporting this where possible, as it may prove to be easier and more robust to follow the mechanism that you are following as a workaround in this case rather than us reimplementing this range of protocols in this plugin as well.

My main concern with supporting this will be that it adds complexity to maintain on our side which has to be kept aligned with how the Maven Dependency Plugin is handling things (which operates via a different set of mechanisms on the interface level at the moment).

[ascopes]

Just looking into this on my side...

Regarding this part:

Because of the sheer amount of transitive dependencies that contains .proto-files it is not really feasible to list them all explicitly in the services pom files.

Transitive dependencies should be able to be pulled in implicitly already by tuning the dependency resolution depth settings. Is this the behaviour you are looking for or is there a different requirement behind this part?

[tommyk-gears]

In your case, do you inherit the dependencies from a parent POM usually?

No, the dependencies that (transitively) contains the .proto files are declared in each service. BUT, we do have common parent pom with plugins and plugin management, where we can put the dependency-plugin config and protobuf-maven-plugin config.

Transitive dependencies should be able to be pulled in implicitly already by tuning the dependency resolution depth settings. Is this the behaviour you are looking for or is there a different requirement behind this part?

Yes, transitive dependencies are not a problem, it is just the dependency filtering by groupId (and scope) that is not supported by this plugin afaiu.

Though, as you said - using the dependency-plugin to unpack proto files is an ok workaround from my point of view. Feel free to close this issue if you don't think the feature belongs in the plugin.

[ascopes]

Cool, okay. Thanks for all the info.

If the workaround works for you for now and isn't causing problems/hassle in your use case, then I won't close this, but I will convert it to a discussion. If anyone else comes along and believes they would also benefit from the same feature within the plugin, we can always revisit this!

If another case comes along for you, feel free to let me know and likewise, I can reopen this for consideration!