feat(python_pyi): add generated pyi to py_proto_library

[josephrubin]

Since protoc supports pyi_out, allow py_proto_library to generate the pyi files for editor support (otherwise, most editors cannot autocomplete with proto python output).

References: #1293

Testing:

There are no tests specifically for py_proto_library, but when I run:

~/protobuf/examples$ bazel build //:addressbook_py_pb2

I get:

INFO: Analyzed target //:addressbook_py_pb2 (0 packages loaded, 0 targets configured).
INFO: Found 1 target...
Target //:addressbook_py_pb2 up-to-date:
  bazel-bin/external/protobuf~/src/google/protobuf/_virtual_imports/timestamp_proto/google/protobuf/timestamp_pb2.py
  bazel-bin/external/protobuf~/src/google/protobuf/_virtual_imports/timestamp_proto/google/protobuf/timestamp_pb2.pyi
  bazel-bin/addressbook_pb2.py
  bazel-bin/addressbook_pb2.pyi

[josephrubin]

I'm not entirely sure the best way to do this. I would prefer to just add the --pyi_out to python's proto_lang_toolchain instantiation but the command_line = "--python_out=%s", can only contain one %s string.

This is my first time in this code base so I would appreciate any suggestions.

[haberman]

I think we should have you define another toolchain here for pyi:

protobuf/python/build_targets.bzl

Lines 553 to 561 in 73a5afe

	proto_lang_toolchain(
	name = "python_toolchain",
	command_line = "--python_out=%s",
	progress_message = "Generating Python proto_library %{label}",
	runtime = ":protobuf_python",
	# NOTE: This isn't *actually* public. It's an implicit dependency of py_proto_library,
	# so must be public so user usages of the rule can reference it.
	visibility = ["//visibility:public"],
	)

Then it can be another dependency to the rule, and we can invoke proto_common.compile() twice, once for each toolchain.

[haberman]

This PR should not require changing proto_common.bzl. That is common code used by all proto rules, and shouldn't know anything about .pyi.

[shaod2]

Why relying on the output of py_proto_library is required? I would think that it should be consumed by other bazel rules. Could you just run protoc to get the desired result?

[josephrubin]

I think that bazel build should be the canonical way to generate build artifacts in a bazel repo. You could run protoc directly but bazel in theory has a better understanding of the packaging, build directory, and so forth for a Python project (as well as uses the hermetic protoc).

The resulting pyi files could be used as input to, for example, another rule that generates documentation based on build output.

But I also understand where you are coming from. These pyi files are not needed at runtime.

[haberman]

Sorry for the slow review. I've just added some suggestions for how to do this without changing proto_common.bzl.

[mkruskal-google]

#21567 seems to do this the way Josh was suggesting, closing this to follow up on that PR