hicPlotMatrix has shown error when I took input in .cool format. It works when I take input in .h5 format.

[ajaykumarsaw]

Hi,
I face trouble in hicPlotMatrix command. It show an error when I took input in .cool format(hicMatrix/replicateMerged.Corrected_20kb.cool). It works when I take input in .h5 (hicMatrix/replicateMerged.Corrected_20kb.h5)format.

Thanks for your support in advance.

(base) [amol@heracles HIC_explorer]$ hicPlotMatrix --log1p --dpi 300 --matrix hicMatrix/replicateMerged.Corrected_20kb.cool --region chrX --title "Corrected Hi-C matrix for mESC : chrX" --outFileName plots/replicateMerged_Corrected-20kb_plot-chrX.png
Traceback (most recent call last):
File "/home/amol/anaconda2/bin/hicPlotMatrix", line 7, in
main()
File "/home/amol/anaconda2/lib/python2.7/site-packages/hicexplorer/hicPlotMatrix.py", line 614, in main
start_pos=start_pos1, start_pos2=start_pos2, pNorm=norm, pAxis=ax1, pBigwig=bigwig_info)
File "/home/amol/anaconda2/lib/python2.7/site-packages/hicexplorer/hicPlotMatrix.py", line 195, in plotHeatmap
img3 = axHeat2.pcolormesh(xmesh.T, ymesh.T, ma, vmin=args.vMin, vmax=args.vMax, cmap=cmap, norm=pNorm)
File "/home/amol/anaconda2/lib/python2.7/site-packages/matplotlib/init.py", line 1867, in inner
return func(ax, *args, **kwargs)
File "/home/amol/anaconda2/lib/python2.7/site-packages/matplotlib/axes/_axes.py", line 6021, in pcolormesh
X, Y, C = self._pcolorargs('pcolormesh', *args, allmatch=allmatch)
File "/home/amol/anaconda2/lib/python2.7/site-packages/matplotlib/axes/_axes.py", line 5590, in _pcolorargs
C.shape, Nx, Ny, funcname))
TypeError: Dimensions of C (7979, 7979) are incompatible with X (7978) and/or Y (7978); see help(pcolormesh)

[joachimwolff]

Hi, The error shows you use a Python 2.7 installation, but we dropped support for this beginning of this year. Please update to Python 3.6 and to HiCExplorer 3.3. Best, Joachim

[ajaykumarsaw]

Hi,

As per your suggestion, I install python 3.6 (conda create --name 3point6 python=3.6) in anaconda3 and install hicexplorer (conda install -c bioconda hicexplorer) from anaconda cloud.

But still, I face the same problem as I discussed my previous mail.

I simply run Hi-C analysis of mouse ESCs using HiCExplorer (https://hicexplorer.readthedocs.io/en/latest/content/mES-HiC_analysis.html) except I changed extension in .cool format in the place of .h5 format from the tutorial.

[joachimwolff]

Hi, How have you changed it? Best, Joachim

[ajaykumarsaw]

I am working in anaconda3 enviroment.
here I install python3.6 using the command
"conda create --name 3point6 python=3.6".

Thenafter I enter the virtual enviroment in 3point6 using the command
"conda activate 3point6".

Thenafter, I install all package as given in installation instruction (https://hicexplorer.readthedocs.io/en/latest/content/installation.html) from anaconda cloud.

after installation we follow the instruction, I follow the instruction (https://hicexplorer.readthedocs.io/en/latest/content/mES-HiC_analysis.html) for hic analysis using mouse dataset.

[ajaykumarsaw]

The error message is given below:

(3point6) [amol@heracles 1]$ hicPlotMatrix --log1p --dpi 300 --matrix hicMatrix/replicateMerged.Corrected_20kb.cool --region chrX --title "Corrected Hi-C matrix for mESC : chrX" --outFileName plots/replicateMerged_Corrected-20kb_plot-chrX.png
INFO:numexpr.utils:Note: NumExpr detected 64 cores but "NUMEXPR_MAX_THREADS" not set, so enforcing safe limit of 8.
INFO:numexpr.utils:NumExpr defaulting to 8 threads.
Traceback (most recent call last):
File "/home/amol/anaconda3/envs/3point6/bin/hicPlotMatrix", line 7, in
main()
File "/home/amol/anaconda3/envs/3point6/lib/python3.6/site-packages/hicexplorer/hicPlotMatrix.py", line 614, in main
start_pos=start_pos1, start_pos2=start_pos2, pNorm=norm, pAxis=ax1, pBigwig=bigwig_info)
File "/home/amol/anaconda3/envs/3point6/lib/python3.6/site-packages/hicexplorer/hicPlotMatrix.py", line 195, in plotHeatmap
img3 = axHeat2.pcolormesh(xmesh.T, ymesh.T, ma, vmin=args.vMin, vmax=args.vMax, cmap=cmap, norm=pNorm)
File "/home/amol/anaconda3/envs/3point6/lib/python3.6/site-packages/matplotlib/init.py", line 1601, in inner
return func(ax, *map(sanitize_sequence, args), **kwargs)
File "/home/amol/anaconda3/envs/3point6/lib/python3.6/site-packages/matplotlib/axes/_axes.py", line 6157, in pcolormesh
X, Y, C = self._pcolorargs('pcolormesh', *args, allmatch=allmatch)
File "/home/amol/anaconda3/envs/3point6/lib/python3.6/site-packages/matplotlib/axes/_axes.py", line 5752, in _pcolorargs
C.shape, Nx, Ny, funcname))
TypeError: Dimensions of C (7979, 7979) are incompatible with X (7978) and/or Y (7978); see help(pcolormesh)

[joachimwolff]

Hi, Thanks, can you show the command of h5 to cool? Best, Joachim

[ajaykumarsaw]

Hi,

I not able to install tools which convert .h5 format to .cool format. Please suggest me the right command so that I can install tools which can conver .h5 format file into .cool format. Therefore, I directly used .cool extension for generating output. The command which I used is given below:

After generating .bam file

Build Hi-c matrix:

hicBuildMatrix --samFiles SRR1956527_1.bam SRR1956527_2.bam --binSize 10000 --restrictionSequence GATC --outBam SRR1956527_ref.bam --outFileName hicMatrix/SRR1956527_10kb.cool --QCfolder hicMatrix/SRR1956527_10kb_QC --threads 8 --inputBufferSize 400000

hicBuildMatrix --samFiles SRR1956528_1.bam SRR1956528_2.bam --binSize 10000 --restrictionSequence GATC --outBam SRR1956528_ref.bam --outFileName hicMatrix/SRR1956528_10kb.cool --QCfolder hicMatrix/SRR1956528_10kb_QC --threads 8 --inputBufferSize 400000

hicBuildMatrix --samFiles SRR1956529_1.bam SRR1956529_2.bam --binSize 10000 --restrictionSequence GATC --outBam SRR1956529_ref.bam --outFileName hicMatrix/SRR1956529_10kb.cool --QCfolder hicMatrix/SRR1956529_10kb_QC --threads 8 --inputBufferSize 400000

Merge (sum) matrices from replicates

hicSumMatrices --matrices hicMatrix/SRR1956527_10kb.cool hicMatrix/SRR1956528_10kb.cool hicMatrix/SRR1956529_10kb.cool --outFileName hicMatrix/replicateMerged_10kb.cool

Merge matrix bins for plotting

hicMergeMatrixBins --matrix hicMatrix/replicateMerged_10kb.cool --numBins 100 --outFileName hicMatrix/replicateMerged.100bins.cool

Plot the corrected Hi-C matrix

hicPlotMatrix --matrix hicMatrix/replicateMerged.100bins.cool --log1p --dpi 300 --clearMaskedBins --chromosomeOrder chr1 chr2 chr3 chr4 chr5 chr6 chr7 chr8 chr9 chr10 chr11 chr12 chr13 chr14 chr15 chr16 chr17 chr18 chr19 chrX chrY --colorMap jet --title "Hi-C matrix for mESC" --outFileName plots/plot_1Mb_matrix.png

Correct Hi-C Matrix

hicMergeMatrixBins --matrix hicMatrix/replicateMerged_10kb.cool --numBins 2 --outFileName hicMatrix/replicateMerged.matrix_20kb.cool

hicCorrectMatrix diagnostic_plot --chromosomes chr1 chr2 chr3 chr4 chr5 chr6 chr7 chr8 chr9 chr10 chr11 chr12 chr13 chr14 chr15 chr16 chr17 chr18 chr19 chrX chrY --matrix hicMatrix/replicateMerged.matrix_20kb.cool --plotName hicMatrix/diagnostic_plot.png

hicCorrectMatrix correct --chromosomes chr1 chr2 chr3 chr4 chr5 chr6 chr7 chr8 chr9 chr10 chr11 chr12 chr13 chr14 chr15 chr16 chr17 chr18 chr19 chrX --matrix hicMatrix/replicateMerged.matrix_20kb.cool --filterThreshold -1.5 2 --perchr --outFileName hicMatrix/replicateMerged.Corrected_20kb.cool

Thereafter, I use given below command and got the error.

hicPlotMatrix --log1p --dpi 300 --matrix hicMatrix/replicateMerged.Corrected_20kb.cool --region chrX --title "Corrected Hi-C matrix for mESC : chrX" --outFileName plots/replicateMerged_Corrected-20kb_plot-chrX.png

--One more issue, I generated TADs by following further instruction using .cool format file. I generated TADs(marks_et-al_TADs.png), but no TADs generated inside the .png file.

[joachimwolff]

Hi,

This seems all correct to me, which means we have to debug this a bit.

We provide the tool hicConvertFormat to convert h5 to cool or the other way round. It is part of HiCExplorer. See: https://hicexplorer.readthedocs.io/en/latest/content/tools/hicConvertFormat.html#hicconvertformat

Could you please test the following:

1. Convert your cool file to h5 and test the plotting again. Does it also crash?
2. Can you build a h5 file from scratch and see if the plotting works with it?
3. Can you plot the cool file but a non-merge version maybe just on a chromosome region? E.g. hicPlotMatrix --log1p --dpi 300 --matrix hicMatrix/SRR1956527_10kb.cool --region chr1:15000000-22000000 --title "Corrected Hi-C matrix for mESC : chr1 18-22mb" --outFileName plots/replicateMerged_Corrected-20kb_plot-chr1_18mb_22mb.png

Thanks a lot!

Best,

Joachim

[ajaykumarsaw]

Hi,
I successfully run hicexplorer using .h5 format file and successfully generated file (replicateMerged_Corrected-20kb_plot-chrX.png) using the command:

"hicPlotMatrix --log1p --dpi 300 --matrix hicMatrix/replicateMerged.Corrected_20kb.cool --region chrX --title "Corrected Hi-C matrix for mESC : chrX" --outFileName plots/replicateMerged_Corrected-20kb_plot-chrX.png"

Additionally, I successfully converted .h5 into .cool using hicConvertFormat command which is

"hicConvertFormat --matrices hicMatrix/replicateMerged.Corrected_20kb.h5 --outFileName replicateMerged.Corrected_20kb.h5.cool --inputFormat h5 --outputFormat cool"

BUT I faced another problem. I face trouble in png file of TAD, the command is written below and I will not get TADs but got .png file which I attached

(3point6) [amol@heracles 1]$ hicPlotTADs --tracks track.ini --region chrX:98000000-105000000 --dpi 300 --outFileName plots/marks_et-al_TADs.png --title "Marks et. al. TADs on X"
Exception ignored in: <_io.FileIO name='track.ini' mode='rb' closefd=True>
ResourceWarning: unclosed file <_io.TextIOWrapper name='track.ini' mode='r' encoding='UTF-8'>

title not set for 'section 2. [tads]'
Exception ignored in: <_io.FileIO name='mm10_genes_sorted.bed' mode='rb' closefd=True>
ResourceWarning: unclosed file <_io.BufferedReader name='mm10_genes_sorted.bed'>
INFO:pygenometracks.tracksClass:time initializing track(s):
INFO:pygenometracks.tracksClass:4.83548378944397
DEBUG:pygenometracks.tracksClass:Figure size in cm is 40 x 32. Dpi is set to 300

DEBUG:pygenometracks.tracksClass:ylim 2859.9999999999995,0
Exception ignored in: <_io.FileIO name='track.ini' mode='rb' closefd=True>
ResourceWarning: unclosed file <_io.TextIOWrapper name='track.ini' mode='r' encoding='UTF-8'>

[joachimwolff]

Hi,

thanks for the heads up. It could be a bug in pyGenomeTracks, but I need to reproduce it. Will come back to you.

Best,

Joachim

[lldelisle]

Hi,
I imagine it is too late but just in case, if you could give us the ini file and a part of your TAD file this would be great.
Thanks and sorry for the late answer...

[joachimwolff]

I could not reproduce it, maybe it is already fixed with the current versions of HiCExplorer and pyGenomeTracks. Please reopen if the issue is still present.