SDK should provide a general resource filtering by GVK

[yuwenma]

Description

Related to #2971 (comment)

SDK should provide a general solution which can be easily used to all functions to filter resources by GVK. Currently, kpt provides the selector and exclude syntax in Kptfile and as flags to filter resources before passing into each function.

Proposal

• kpt shall annotate Kptfile and inline ConfigMap with an internal annotation, propose internal.config.kubernetes.io/kpt-meta, the SDK should provide functions to filter resources by annotation (and this should be called inside a function)
• FunctionConfig defines a general interface to filter resources by GVK. SDK provides the actual filtering logic (maybe a default behavior under the SDK resourceList Process so it's applied to all functions automatically). It would be like

apiVersion: config.kubernetes.io/v1
kind: Resourcelist
items:
- apiVersion: v1
  kind: ConfigMap
  metadata:
       name: current-function-config
       annotations:
         internal.config.kubernetes.io/kpt-meta: true # marked as meta resource, exclude by default
- apiVersion: v1
  kind: ConfigMap
  metadata:
       name: other-function-config
  annotations:
       internal.config.kubernetes.io/kpt-meta: true # marked as meta resource, exclude by default
- apiVersion: kpt.dev/v1
  kind: Kptfile   # Kptfile, exclude by default
  ...
functionConfig:
  apiVersion: v1
  kind: ConfigMap
  metadata:
      name: current-function-config
      annotations:
        internal.config.kubernetes.io/exclude-meta: true
        internal.config.kubernetes.io/exclude: |     # function level exclude list.
           - kind:
           - kind:

[yuwenma]

+cc @natasha41575 @droot @bgrant0607

[natasha41575]

kpt shall annotate Kptfile and inline ConfigMap with an internal annotation, propose internal.config.kubernetes.io/kpt-meta, the SDK should provide functions to filter resources by annotation (and this should be called inside a function)

Quick clarification about this for @yuwenma and @droot, according to the kpt variant constructor pattern doc, we have changed the definition of "meta resources" to no longer include functionConfigs. So Kptfile is now the only meta resource. My understanding is that means only the Kptfile should receive this new annotation.

   internal.config.kubernetes.io/kpt-meta: true # marked as meta resource, exclude by default

Similar question here. I think we can provide something in the SDK that helps function authors exclude meta resources it they want to. But in the variant constructor doc, we decided meta resources should be included by default, so I'm not sure if we should automatically call it in Process.

[droot]

So Kptfile is now the only meta resource.

Yes, that is correct.

My opinion about this task is that SDK should provide a nice API for filtering resources that is aligned with selectors and exclusion support in kpt fn. So a nice API for filtering on GVK, labelSelector, annotationSelector etc. (indexing by filePath, or subpackages is another use-case)

Annotating meta-resources annotation seems to be a low priority to me given that there is only one meta resource currently. SDK can provide a syntax sugar around filtering by meta-resources and internally apply the kind filter for Kptfile kind without adding the annotation.