Bump max range of dill to 0.3.8

[ringohoffman]

Release on Jan 27, 2024: https://pypi.org/project/dill/0.3.8/#history

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[ringohoffman]

Hmm these errors look pretty weird... can they be retried?

[mariosasko]

Hi, thanks for working on this! To fix the errors, you also need to update this file (by adding version.parse("0.3.8").release to the lists)

[mariosasko]

Thanks!

[albertvillanova]

Thank you.

[github-actions]

Show benchmarks

PyArrow==8.0.0

Show updated benchmarks!

Benchmark: benchmark_array_xd.json

metric	read_batch_formatted_as_numpy after write_array2d	read_batch_formatted_as_numpy after write_flattened_sequence	read_batch_formatted_as_numpy after write_nested_sequence	read_batch_unformated after write_array2d	read_batch_unformated after write_flattened_sequence	read_batch_unformated after write_nested_sequence	read_col_formatted_as_numpy after write_array2d	read_col_formatted_as_numpy after write_flattened_sequence	read_col_formatted_as_numpy after write_nested_sequence	read_col_unformated after write_array2d	read_col_unformated after write_flattened_sequence	read_col_unformated after write_nested_sequence	read_formatted_as_numpy after write_array2d	read_formatted_as_numpy after write_flattened_sequence	read_formatted_as_numpy after write_nested_sequence	read_unformated after write_array2d	read_unformated after write_flattened_sequence	read_unformated after write_nested_sequence	write_array2d	write_flattened_sequence	write_nested_sequence
new / old (diff)	0.005068 / 0.011353 (-0.006285)	0.003657 / 0.011008 (-0.007351)	0.062914 / 0.038508 (0.024406)	0.027965 / 0.023109 (0.004855)	0.241804 / 0.275898 (-0.034094)	0.268069 / 0.323480 (-0.055411)	0.004066 / 0.007986 (-0.003920)	0.002704 / 0.004328 (-0.001624)	0.048745 / 0.004250 (0.044495)	0.042158 / 0.037052 (0.005106)	0.257670 / 0.258489 (-0.000819)	0.279419 / 0.293841 (-0.014422)	0.027193 / 0.128546 (-0.101353)	0.010379 / 0.075646 (-0.065267)	0.207009 / 0.419271 (-0.212262)	0.035494 / 0.043533 (-0.008039)	0.246025 / 0.255139 (-0.009114)	0.265906 / 0.283200 (-0.017294)	0.017335 / 0.141683 (-0.124348)	1.134052 / 1.452155 (-0.318103)	1.184668 / 1.492716 (-0.308049)

Benchmark: benchmark_getitem_100B.json

metric	get_batch_of_1024_random_rows	get_batch_of_1024_rows	get_first_row	get_last_row
new / old (diff)	0.093137 / 0.018006 (0.075130)	0.302279 / 0.000490 (0.301789)	0.000210 / 0.000200 (0.000010)	0.000047 / 0.000054 (-0.000008)

Benchmark: benchmark_indices_mapping.json

metric	select	shard	shuffle	sort	train_test_split
new / old (diff)	0.018190 / 0.037411 (-0.019221)	0.061436 / 0.014526 (0.046910)	0.073102 / 0.176557 (-0.103454)	0.119782 / 0.737135 (-0.617354)	0.074292 / 0.296338 (-0.222046)

Benchmark: benchmark_iterating.json

metric	read 5000	read 50000	read_batch 50000 10	read_batch 50000 100	read_batch 50000 1000	read_formatted numpy 5000	read_formatted pandas 5000	read_formatted tensorflow 5000	read_formatted torch 5000	read_formatted_batch numpy 5000 10	read_formatted_batch numpy 5000 1000	shuffled read 5000	shuffled read 50000	shuffled read_batch 50000 10	shuffled read_batch 50000 100	shuffled read_batch 50000 1000	shuffled read_formatted numpy 5000	shuffled read_formatted_batch numpy 5000 10	shuffled read_formatted_batch numpy 5000 1000
new / old (diff)	0.285905 / 0.215209 (0.070696)	2.809051 / 2.077655 (0.731397)	1.470305 / 1.504120 (-0.033814)	1.350457 / 1.541195 (-0.190738)	1.349111 / 1.468490 (-0.119379)	0.568277 / 4.584777 (-4.016500)	2.353046 / 3.745712 (-1.392666)	2.805862 / 5.269862 (-2.463999)	1.750275 / 4.565676 (-2.815401)	0.062370 / 0.424275 (-0.361905)	0.004954 / 0.007607 (-0.002653)	0.335609 / 0.226044 (0.109564)	3.367200 / 2.268929 (1.098271)	1.829431 / 55.444624 (-53.615193)	1.545093 / 6.876477 (-5.331384)	1.571107 / 2.142072 (-0.570966)	0.640279 / 4.805227 (-4.164949)	0.116209 / 6.500664 (-6.384455)	0.042308 / 0.075469 (-0.033161)

Benchmark: benchmark_map_filter.json

metric	filter	map fast-tokenizer batched	map identity	map identity batched	map no-op batched	map no-op batched numpy	map no-op batched pandas	map no-op batched pytorch	map no-op batched tensorflow
new / old (diff)	0.982972 / 1.841788 (-0.858816)	11.424370 / 8.074308 (3.350062)	10.427111 / 10.191392 (0.235719)	0.129477 / 0.680424 (-0.550946)	0.014166 / 0.534201 (-0.520035)	0.287597 / 0.579283 (-0.291686)	0.265588 / 0.434364 (-0.168776)	0.324007 / 0.540337 (-0.216330)	0.430766 / 1.386936 (-0.956170)

PyArrow==latest

Show updated benchmarks!

Benchmark: benchmark_array_xd.json

metric	read_batch_formatted_as_numpy after write_array2d	read_batch_formatted_as_numpy after write_flattened_sequence	read_batch_formatted_as_numpy after write_nested_sequence	read_batch_unformated after write_array2d	read_batch_unformated after write_flattened_sequence	read_batch_unformated after write_nested_sequence	read_col_formatted_as_numpy after write_array2d	read_col_formatted_as_numpy after write_flattened_sequence	read_col_formatted_as_numpy after write_nested_sequence	read_col_unformated after write_array2d	read_col_unformated after write_flattened_sequence	read_col_unformated after write_nested_sequence	read_formatted_as_numpy after write_array2d	read_formatted_as_numpy after write_flattened_sequence	read_formatted_as_numpy after write_nested_sequence	read_unformated after write_array2d	read_unformated after write_flattened_sequence	read_unformated after write_nested_sequence	write_array2d	write_flattened_sequence	write_nested_sequence
new / old (diff)	0.005347 / 0.011353 (-0.006005)	0.003733 / 0.011008 (-0.007275)	0.049520 / 0.038508 (0.011011)	0.031177 / 0.023109 (0.008068)	0.281854 / 0.275898 (0.005956)	0.300937 / 0.323480 (-0.022543)	0.004385 / 0.007986 (-0.003601)	0.002841 / 0.004328 (-0.001488)	0.048661 / 0.004250 (0.044411)	0.044258 / 0.037052 (0.007205)	0.295651 / 0.258489 (0.037162)	0.322872 / 0.293841 (0.029031)	0.048924 / 0.128546 (-0.079622)	0.010742 / 0.075646 (-0.064905)	0.059327 / 0.419271 (-0.359944)	0.033938 / 0.043533 (-0.009595)	0.282235 / 0.255139 (0.027096)	0.297432 / 0.283200 (0.014233)	0.018295 / 0.141683 (-0.123388)	1.164459 / 1.452155 (-0.287696)	1.214511 / 1.492716 (-0.278205)

Benchmark: benchmark_getitem_100B.json

metric	get_batch_of_1024_random_rows	get_batch_of_1024_rows	get_first_row	get_last_row
new / old (diff)	0.091441 / 0.018006 (0.073435)	0.303023 / 0.000490 (0.302533)	0.000211 / 0.000200 (0.000011)	0.000051 / 0.000054 (-0.000004)

Benchmark: benchmark_indices_mapping.json

metric	select	shard	shuffle	sort	train_test_split
new / old (diff)	0.022024 / 0.037411 (-0.015388)	0.075570 / 0.014526 (0.061044)	0.086761 / 0.176557 (-0.089796)	0.126437 / 0.737135 (-0.610698)	0.088354 / 0.296338 (-0.207984)

Benchmark: benchmark_iterating.json

metric	read 5000	read 50000	read_batch 50000 10	read_batch 50000 100	read_batch 50000 1000	read_formatted numpy 5000	read_formatted pandas 5000	read_formatted tensorflow 5000	read_formatted torch 5000	read_formatted_batch numpy 5000 10	read_formatted_batch numpy 5000 1000	shuffled read 5000	shuffled read 50000	shuffled read_batch 50000 10	shuffled read_batch 50000 100	shuffled read_batch 50000 1000	shuffled read_formatted numpy 5000	shuffled read_formatted_batch numpy 5000 10	shuffled read_formatted_batch numpy 5000 1000
new / old (diff)	0.289360 / 0.215209 (0.074151)	2.816433 / 2.077655 (0.738779)	1.561442 / 1.504120 (0.057322)	1.438168 / 1.541195 (-0.103027)	1.453398 / 1.468490 (-0.015092)	0.579474 / 4.584777 (-4.005303)	2.458640 / 3.745712 (-1.287072)	2.638572 / 5.269862 (-2.631290)	1.725218 / 4.565676 (-2.840458)	0.063550 / 0.424275 (-0.360725)	0.005220 / 0.007607 (-0.002387)	0.338883 / 0.226044 (0.112838)	3.353585 / 2.268929 (1.084656)	1.913186 / 55.444624 (-53.531438)	1.667445 / 6.876477 (-5.209032)	1.740085 / 2.142072 (-0.401987)	0.646369 / 4.805227 (-4.158859)	0.116737 / 6.500664 (-6.383927)	0.041052 / 0.075469 (-0.034417)

Benchmark: benchmark_map_filter.json

metric	filter	map fast-tokenizer batched	map identity	map identity batched	map no-op batched	map no-op batched numpy	map no-op batched pandas	map no-op batched pytorch	map no-op batched tensorflow
new / old (diff)	1.023180 / 1.841788 (-0.818608)	12.078398 / 8.074308 (4.004090)	10.952012 / 10.191392 (0.760620)	0.131335 / 0.680424 (-0.549089)	0.015701 / 0.534201 (-0.518499)	0.289709 / 0.579283 (-0.289574)	0.270495 / 0.434364 (-0.163869)	0.331773 / 0.540337 (-0.208565)	0.417660 / 1.386936 (-0.969276)