error in data partition?

[XiaoqianYan]

hello, I have a dataset missing one trial in one of my five experimental conditions, so instead of having 12trial x 5 cons = 60 trials, I have 59 trials in total for that subject. When I ran LDA classification with the multiclass mode, I got an error.

[blairkan]

Hi Xiaoqian, can you please check to see whether the corresponding label for the excluded trial was also removed from the labels vector? The length of the labels vector should match the number of trials in the data matrix.

[XiaoqianYan]

hello blair, yes the label vector is at the same length with my trial number. my Xdata is in 128x164x59, and Y label is in 59x1.

[berneezy3]

Hi Xiaoqian,

Would it be possible to send me to dataset and the code snippet? You could email me at "bernardcwang@gmail.com".

I can take a look at what is causing the issue.

[berneezy3]

Hi Xiaoqian,

I just tried your code on the current version of the dev2 branch, the code was able to get past the error that you mentioned.

Could I ask when you downloaded the dev2 repository? The previous push was 26 days ago - if you downloaded the repository prior to that, then re-pulling from the dev2 branch should fix this issue. However, please let me know if this is not the case.

However, the code did encounter an additional error after getting past the line you specified. This new error is due to the high dimensionality of your dataset. In your current dataset, you have 128 * 164 = 20992 features, and MATLAB's LDA function creates a 20992 * 20992 * 10 matrix (32.8GB), which will likely cause your computer to run out of memory. To bypass this error, I would suggest running your analysis with the 'PCA' parameter set to .99 as such:

C_LDA = RSA.Classification.crossValidateMulti(X_norm,Y_shuf,'PCA', 0.99, 'classifier','LDA');

Making these two fixes allows your script, 'subj13LDA.m', to run through on my system. Please let me know if there are still errors after these two fixes.

[XiaoqianYan]

Hi Bernard,

Thanks for helping. I updated the repo. It works now! However, the outputs with or without PCA first are very different. I have much lower dissimilarity measures along the diagonal without PCA, e.g 0.18 vs 0.68 for faces. Do you know why?

[berneezy3]

Hi Xiaoqian,

Would it be possible to upload screenshots of the two matrices w/ and w/o PCA, the code used to produce both classification, and finally a description of the categories? Blair and I will take a closer look.