Segfault with CSV, SQLite, tables (M1)

[danvinci]

Hi,

the title can be improved but right now I don't know exactly what the issue is about - seems related to #1722

Context:

• Julia 1.7.1 on M1
• I have a bunch of CSV files
• I load these files to a newly-created SQLite file
• I create new tables resulting from the union of these tables
• weird stuff happens (Pluto disconnects, some tasks/processes go uncompleted, DB gets corrupted)
• I start from scratch (delete the DB)

>  From worker 2:	signal (11): Segmentation fault: 11
>       From worker 2:	in expression starting at none:1
>       From worker 2:	getvalue at /Users/dan/.julia/packages/SQLite/5DdLp/src/tables.jl:0
>       From worker 2:	getcolumn at /Users/dan/.julia/packages/SQLite/5DdLp/src/tables.jl:75
>       From worker 2:	unknown function (ip: 0x11aacfd1f)
>       From worker 2:	jl_apply_generic at /Applications/Julia.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.7.dylib (unknown line)
>       From worker 2:	getcolumn at /Users/dan/.julia/packages/SQLite/5DdLp/src/tables.jl:77
>       From worker 2:	getcolumn at /Users/dan/.julia/packages/SQLite/5DdLp/src/tables.jl:78 [inlined]
>       From worker 2:	eachcolumns at /Users/dan/.julia/packages/Tables/M26tI/src/utils.jl:127 [inlined]
>       From worker 2:	_buildcolumns at /Users/dan/.julia/packages/Tables/M26tI/src/fallbacks.jl:187
>       From worker 2:	unknown function (ip: 0x11aacf66b)
>       From worker 2:	jl_apply_generic at /Applications/Julia.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.7.dylib (unknown line)
>       From worker 2:	buildcolumns at /Users/dan/.julia/packages/Tables/M26tI/src/fallbacks.jl:217 [inlined]
>       From worker 2:	columns at /Users/dan/.julia/packages/Tables/M26tI/src/fallbacks.jl:253 [inlined]
>       From worker 2:	columntable at /Users/dan/.julia/packages/Tables/M26tI/src/namedtuples.jl:171
>       From worker 2:	execute at /Users/dan/.julia/packages/DBInterface/1Gmxx/src/DBInterface.jl:135
>       From worker 2:	jl_apply_generic at /Applications/Julia.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.7.dylib (unknown line)
>       From worker 2:	execute at /Users/dan/.julia/packages/DBInterface/1Gmxx/src/DBInterface.jl:144
>       From worker 2:	#execute#3 at /Users/dan/.julia/packages/DBInterface/1Gmxx/src/DBInterface.jl:153 [inlined]
>       From worker 2:	execute at /Users/dan/.julia/packages/DBInterface/1Gmxx/src/DBInterface.jl:153 [inlined]
>       From worker 2:	tables at /Users/dan/.julia/packages/SQLite/5DdLp/src/SQLite.jl:665 [inlined]
>       From worker 2:	tables at /Users/dan/.julia/packages/SQLite/5DdLp/src/SQLite.jl:665
>       From worker 2:	jl_apply_generic at /Applications/Julia.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.7.dylib (unknown line)
>       From worker 2:	##function_wrapped_cell#343 at /Users/dan/Documents/pluto_nb/VAERS/v1.jl#==#dba7c5d3-8fb7-49df-827a-a51af3ed6f07:5 [inlined]
>       From worker 2:	##function_wrapped_cell#343 at ./none:0
>       From worker 2:	unknown function (ip: 0x11aace843)
>       From worker 2:	jl_apply_generic at /Applications/Julia.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.7.dylib (unknown line)
>       From worker 2:	jl_f__call_latest at /Applications/Julia.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.7.dylib (unknown line)
>       From worker 2:	jl_apply_generic at /Applications/Julia.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.7.dylib (unknown line)
>       From worker 2:	do_apply at /Applications/Julia.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.7.dylib (unknown line)
>       From worker 2:	#invokelatest#2 at ./essentials.jl:716
>       From worker 2:	jl_apply_generic at /Applications/Julia.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.7.dylib (unknown line)
>       From worker 2:	do_apply at /Applications/Julia.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.7.dylib (unknown line)
>       From worker 2:	invokelatest at ./essentials.jl:714
>       From worker 2:	jl_apply_generic at /Applications/Julia.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.7.dylib (unknown line)
>       From worker 2:	do_apply at /Applications/Julia.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.7.dylib (unknown line)
>       From worker 2:	compute at /Users/dan/.julia/packages/Pluto/F4y3q/src/runner/PlutoRunner.jl:385
>       From worker 2:	#27 at /Users/dan/.julia/packages/Pluto/F4y3q/src/runner/PlutoRunner.jl:540
>       From worker 2:	run_inside_trycatch at /Users/dan/.julia/packages/Pluto/F4y3q/src/runner/PlutoRunner.jl:425
>       From worker 2:	#run_expression#25 at /Users/dan/.julia/packages/Pluto/F4y3q/src/runner/PlutoRunner.jl:540
>       From worker 2:	run_expression##kw at /Users/dan/.julia/packages/Pluto/F4y3q/src/runner/PlutoRunner.jl:455
>       From worker 2:	unknown function (ip: 0x11a922937)
>       From worker 2:	jl_apply_generic at /Applications/Julia.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.7.dylib (unknown line)
>       From worker 2:	do_call at /Applications/Julia.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.7.dylib (unknown line)
>       From worker 2:	eval_body at /Applications/Julia.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.7.dylib (unknown line)
>       From worker 2:	jl_interpret_toplevel_thunk at /Applications/Julia.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.7.dylib (unknown line)
>       From worker 2:	jl_toplevel_eval_flex at /Applications/Julia.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.7.dylib (unknown line)
>       From worker 2:	jl_toplevel_eval_in at /Applications/Julia.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.7.dylib (unknown line)
>       From worker 2:	eval at ./boot.jl:373
>       From worker 2:	jl_apply_generic at /Applications/Julia.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.7.dylib (unknown line)
>       From worker 2:	do_apply at /Applications/Julia.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.7.dylib (unknown line)
>       From worker 2:	#103 at /Users/sabae/src/julia/usr/share/julia/stdlib/v1.7/Distributed/src/process_messages.jl:274
>       From worker 2:	run_work_thunk at /Users/sabae/src/julia/usr/share/julia/stdlib/v1.7/Distributed/src/process_messages.jl:63
>       From worker 2:	unknown function (ip: 0x11a910327)
>       From worker 2:	jl_apply_generic at /Applications/Julia.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.7.dylib (unknown line)
>       From worker 2:	run_work_thunk at /Users/sabae/src/julia/usr/share/julia/stdlib/v1.7/Distributed/src/process_messages.jl:72
>       From worker 2:	#96 at ./task.jl:423
>       From worker 2:	unknown function (ip: 0x11a91008b)
>       From worker 2:	jl_apply_generic at /Applications/Julia.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.7.dylib (unknown line)
>       From worker 2:	start_task at /Applications/Julia.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.7.dylib (unknown line)
>       From worker 2:	Allocations: 650141462 (Pool: 649895122; Big: 246340); GC: 124
> Worker 2 terminated.
> Unhandled Task ERROR: IOError: read: connection reset by peer (ECONNRESET)
> Stacktrace:
>   [1] wait_readnb(x::Sockets.TCPSocket, nb::Int64)
>     @ Base ./stream.jl:408
>   [2] (::Base.var"#wait_locked#645")(s::Sockets.TCPSocket, buf::IOBuffer, nb::Int64)
>     @ Base ./stream.jl:894
>   [3] unsafe_read(s::Sockets.TCPSocket, p::Ptr{UInt8}, nb::UInt64)
>     @ Base ./stream.jl:900
>   [4] unsafe_read
>     @ ./io.jl:724 [inlined]
>   [5] unsafe_read(s::Sockets.TCPSocket, p::Base.RefValue{NTuple{4, Int64}}, n::Int64)
>     @ Base ./io.jl:723
>   [6] read!
>     @ ./io.jl:725 [inlined]
> 

Zooming out of the specific task, I'm trying to come up with my own workflow for working with reasonable-sized data without having to rely too much on in-memory DataFrames.

Video here (available soon): https://youtu.be/lkeXCOHWfBc

Notebook: https://github.com/fonsp/Pluto.jl/files/7918391/v1.jl.zip

All files used can be retrieved here: https://vaers.hhs.gov/eSubDownload/index.jsp?fn=AllVAERSDataCSVS.zip

[danvinci]

I tried using Julia 1.7.1 (x86) and none of the above takes place, runs smoothly 👍

[fonsp]

Thanks for the issue! Segfaults are a bug of Julia itself, so we should report it to Julia's issues.

The smaller the example can get, the better. For example, we might be able to take Pluto out of the equation by running the notebook in Julia directly, without Pluto. Does that still produce the segfault?

[fonsp]

This might be fixed by JuliaLang/julia#43664 . Can you try building Julia from master and seeing if that fixes it?

[danvinci]

An update on this:

• running the script alone, Julia 1.7.1 (ARM) works
• running the script within Pluto, Julia 1.7.1 (ARM) -> Segfault (this issue)
• running the script within Pluto, Julia 1.7.1 (emulated x86) works
• running the script within Pluto, Julia master (ARM) works

Tried each x2.

As for building Julia from master, I just used 'make' and I assume it has been compiled for ARM.

[fonsp]

It looks like this will be fixed by Julia 1.8, which will be released in a couple of months. Until then, we recommend using Julia x86! The M1 build is still too experimental.

Thanks for opening the issue and for the useful testing!! Let us know if something comes up again!