Feature request: option to deploy human-readable format in parallel with binary deployment for datasets

[ntentes]

A common workflow for me is to deploy a pin containing a dataset to Connect in both a binary and human-readable format. This usually happens when R/python users want a fast, consistently typed binary (like qs or arrow) and other business users want to be able to download from Connect in csv format to pull into a spreadsheet.

Currently, I have to deploy both of these separately with unique names (and GUIDs). When distributing the pin or a when a user searches for the pin in Connect, I either have to inform them of both deployments/names or give them the correct link/name based on their tooling.

For example, my pin_write() calls usually look like this:

board %>%
  pin_write(account_information, 
            name = "account_information", 
            type = "arrow"
            )

board %>%
  pin_write(account_information,
            name = "account_information_csv", 
            type = "csv"
            )

I'm not sure of the best way to implement this in terms of interface, but in my mind's eye I see something like an option to include_human_readable:

board %>%
  pin_write(account_information, 
            name = "account_information", 
            type = "arrow",
            include_human_readable = TRUE
            )

or even an option to include multiple types:

board %>%
  pin_write(account_information, 
            name = "account_information", 
            type = c("arrow", "csv")
            )

Another benefit of "coupling" multiple deployment types of the same deployment could be having import redundancy in the case that the pin reader does not have the required package to download a special type like arrow or qs.

Thanks!

[juliasilge]

I like this idea a lot!

board |>
  pin_write(account_information, 
            name = "account_information", 
            type = c("arrow", "csv"))

We do already have the ability to store multiple files together in a pin in pin_upload() so this might not be too tough.

[lbm364dl]

I would also be interested in having this. Before finding this issue my initial idea was also writing to two different pins. The main reason was trying to save some space when downloading. If I have larger files, ideally I wouldn't want users to download a CSV they're not even going to use, because they would just use the other compressed/fast format. The proposed alternative with pin_upload seems to always get both files when doing pin_download. I think adding a type option to pin_download should also be useful. If we do this, then we would also extend to pin_read, and I would like to be able to do something like:

board |>
  pin_read(
    account_information,
    name = "account_information",
    type = "arrow"
  )

Does this make sense? I'm willing to contribute to this at some point.

[juliasilge]

This is now available in the dev version of pins thanks to @lbm364dl! You can try it out by installing from GitHub:

pak::pak("rstudio/pins-r")

[github-actions]

This issue has been automatically locked. If you believe you have found a related problem, please file a new issue (with a reprex: https://reprex.tidyverse.org) and link to this issue.