error when running 'ms4alg.create_label_map'

[vjtiznado]

Hi, first I have to say that I really appreciate your work with mountainsort, it is very valuable

So now my problem.
I'm using the new version of mountainsort in Ubuntu 16.04, and I want to perform the 'curation' step after clustering. As soon as I understood properly the functions' inputs and outputs, after getting the 'firings.mda' file I should run the following functions, in this order:

ephys.compute_cluster_metrics
which gives the metrics.json file as output, then
ms4alg.p_create_label_map
and finally
ms4alg.p_apply_label_map

However, when running ms4alg.p_create_label_map I get the following error:

$ ml-run-process ms4alg.create_label_map --inputs metrics:CM18_reg06_
tt8_cluster_metrics.json --outputs label_map_out:CM18_reg06_tt8_label_map2.mda --parameters firing_rate_thresh:.05 noise_overlap_thresh:.03 peak_snr_thresh:1.5[ Getting processor spec... ]
[ Checking inputs and substituting prvs ... ]
[ Computing process signature ... ]
[ Checking outputs... ]
[ Checking process cache ... ]
[ Creating temporary directory ... ]
[ Preparing temporary outputs... ]
[ Initializing process ... ]
[ Running ... ] /home/labpf/conda/envs/ml-dev/bin/python3 /home/labpf/conda/envs/ml-dev/etc/mountainlab/packages/ml_ms4alg/curation_spec.py.mp ms4alg.create_label_map --metrics=/home/labpf/Documents/VT/mountainlab/data/Registros_para_comparar/cm18_reg06_TTR/CM18_reg06_tt8_cluster_metrics.json --label_map_out=/home/labpf/Documents/VT/mountainlab/data/Registros_para_comparar/cm18_reg06_TTR/tempdir_6fc4b94a0c_dkx9l0/output_label_map_out.mda --firing_rate_thresh=.05 --noise_overlap_thresh=.03 --peak_snr_thresh=1.5 --_tempdir=/home/labpf/Documents/VT/mountainlab/data/Registros_para_comparar/cm18_reg06_TTR/tempdir_6fc4b94a0c_dkx9l0
Traceback (most recent call last):
  File "/home/labpf/conda/envs/ml-dev/etc/mountainlab/packages/ml_ms4alg/curation_spec.py.mp", line 15, in <module>
    if not PM.run(sys.argv):
  File "/home/labpf/conda/envs/ml-dev/lib/python3.6/site-packages/mountainlab_pytools/processormanager/processormanager_impl.py", line 37, in run
    return P(**args)
  File "/home/labpf/conda/envs/ml-dev/lib/python3.6/site-packages/ml_ms4alg/p_create_label_map.py", line 100, in create_label_map
    metrics_data['clusters'][idx]['metrics']['isolation'] < isolation_thresh or \
KeyError: 'isolation'

[ Removing temporary directory ... ]
Process returned with non-zero exit code.

I guess that the error is because the metrics.json file has not the 'isolation' metric, but I'm not sure.
Can you help me, please? If that's the case, how can I make curation function give me the 'isolation' metric?

Thanks in advance,
Vicente

[magland]

Thanks for the question. You need to also use ms3.isolation_metrics, and then combine the two. See the cluster_metrics function in this file:

https://github.com/flatironinstitute/mountainsort_examples/blob/master/mountainsort4_1_0.py

(which may be a broken link sometime in the future)

[vjtiznado]

Thanks a lot for the answer. But I got stuck in a very simple command
I'm far for being an expert with this commands language, so I'm running the spike sorting line by line (but together in a .sh file), so
I successfully run the ms3.isolation_metrics command by running

ml-run-process ms3.isolation_metrics \
	--inputs timeseries:CM18_reg06_tt8_wfilt.mda firings:CM18_reg06_tt8_firings.mda \
	--outputs metrics_out:CM18_reg06_tt8_imetrics.json \
	--parameters compute_bursting_parents:true

But I can't add properly the input for ms3.combine_cluster_metrics function

I naively tried

ml-run-process ms3.combine_cluster_metrics --inputs metrics_list:[CM18_reg06_tt8_cluster_metrics.json,CM18_reg06_tt8_imetrics.json] --outputs metrics_out:CM18_reg06_tt8_fmetrics.json
[ Getting processor spec... ]
[ Checking inputs and substituting prvs ... ]
[ Computing process signature ... ]
Unable to stat file (metrics_list): /home/labpf/Documents/VT/mountainlab/data/Registros_para_comparar/cm18_reg06_TTR/[CM18_reg06_tt8_cluster_metrics.json,CM18_reg06_tt8_imetrics.json]

But it did not work. Could you explain me please how to enter the command line properly?

Thanks in advance

[magland]

oh, sorry, for a list do...

--inputs metrics_list:CM18_reg06_tt8_cluster_metrics.json metrics_list:CM18_reg06_tt8_imetrics.json

[vjtiznado]

Thanks a lot! Now it runs perfectly

[magland]

Great! The documentation is slowly catching up with the code, so thanks for your patience.

[MarcoUniPr]

Hi, I am trying to visualize on qt-mountainview all the cluster metrics of neuronal sorting.
I tried several time to run the following code on my own data:

#!/bin/bash
set -e
#compute cluster metrics
ml-run-process ephys.compute_cluster_metrics
-- inputs timeseries:output/pre.mda.prv firings:output/firings.mda
--outputs metrics_out:output/c_metrics.json
--parameters samplerate:30000
ml-run-process ms3.isolation_metrics
-- inputs timeseries:output/filt.mda.prv firings:output/firings.mda
--outputs metrics_out:output/i_metrics.json
--parameters compute_bursting_parents:true
ml-run-process ms3.combine_cluster_metrics
-- inputs metric_list:output/c_metrics.json metric_list:output/i_metrics.json
--outputs metrics_out:output/m_metrics.json \

Unfortunately the error message is the following:
ZeroDivisionError: float division by zero

Could you help me?

[Tousin1]

@MarcoUniPr I had this error when I attempted to run ephys.compute_cluster_metrics. So try this instead and finger crossed;

#!/bin/bash
set -e
#compute cluster metrics
ml-run-process ephys.compute_cluster_metrics
-- inputs timeseries:output/pre.mda.prv firings:output/firings.mda
--outputs metrics_out:output/c_metrics.json
--parameters samplerate:30000 clip_size:150 refrac_msec:1
ml-run-process ms3.isolation_metrics
-- inputs timeseries:output/filt.mda.prv firings:output/firings.mda
--outputs metrics_out:output/i_metrics.json pair_metrics_out:output/pair_metrics_out
--parameters compute_bursting_parents:true
ml-run-process ms3.combine_cluster_metrics
-- inputs metric_list:output/c_metrics.json metric_list:output/i_metrics.json
--outputs metrics_out:output/m_metrics.json