Implement Dataset add_column #2145
Conversation
lhoestq
left a comment
lhoestq
left a comment
There was a problem hiding this comment.
This feature has been asked many times :)
This is awesome to finally have it ! Thanks for adding it !
I added some suggestions:
src/datasets/arrow_dataset.py
Outdated
| """ | ||
| column_table = InMemoryTable.from_pydict(column) | ||
| # Concatenate tables horizontally | ||
| self._data = ConcatenationTable.from_blocks([[self._data, column_table]]) |
There was a problem hiding this comment.
self._data may not be a valid table block (i.e. either a InMemoryTable or a MemoryMappedTable object).
For example if self._data is a ConcatenationTable, this won't work.
Maybe we can use another ConcatenationTable constructor for this ?
Like for example a version of ConcatenationTable.from_tables but for axis=1 ?
Under the hood this uses from_blocks anyway, but it allows any kind of tables as input.
lhoestq
left a comment
lhoestq
left a comment
There was a problem hiding this comment.
Cool ! Added a few comments :)
|
#2274 has been merged. You can now merge master into this branch and use |
3 participants
Implement
Dataset.add_column.Close #1954.