Skip to content

need to expose Device Type and Data Type configuration for users in fluid python API #6141

@QiJune

Description

@QiJune

We now have ProgramDesc to store neural network topology. However, one operator may have many kernels. ProgramDesc only contains operator. It's not enough to decide which kernels to run.
At now, DataType and DeviceType are the keys to choose which kernel to run. We have to expose these two options to users.

  • Low-precision is efficient in both training and inference. Usually, we would have both fp32 and fp16 in a model. We have to set the reasonable data type for operators.
  • In inference, we may have CPU/GPU/FPGA, and device type has to be set for operators.

I am not sure if our framework can set both data type and device type reasonably. But exposing them to users is flexible for both research and online/mobile inference.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions