RSA-tools - Tutorials - matrix-clustering

Contents

1. Prerequisite
2. Introduction
3. Study case
4. Test sets
5. Tuning matrix-clustering parameters
6. Interpreting the result
7. Additional exercises
8. References

Prerequisite

This tutorial assumes that you are familiar with the concepts developed in the following parts of the theoretical course.

1. PSSM theory
2. Motif comparison

It is better to follow the corresponding tutorials before this one.

1. Position-specific scoring matrices.
2. peak-motifs: discover cis-regulatory motifs and predict putative TFBS from a set of peak sequences identified by high-throughput methods such as ChIP-seq.

Introduction

The program matrix-clustering enables to discover and align groups of similarities among motif collections displaying the results with different motif-representation formats.

Transcription factor binding motifs (TFBM) are classically represented either as consensus strings (stric consensus, IUPAC or regular expressions), or as position-specific scoring matrices (PSSM).

Thousands of curated TFBM are available in specialized databases (JASPAR, RegulonDB, TRANSFAC, etc). These PSSMs were traditionally built from collections of transcription factor binding sites (TFBS) obtained by various experimental methods (e.g. EMSA, DNAse footprinting, SELEX).

TFBMs can also be discovered ab initio from genome-scale datasets: promoters of co-expressed genes, ChIP-seq peaks, phylogenetic footprints, etc.

Motif collections (databases as well as ab initio motif discovery results) sometimes contain groups of similar motifs, for different reasons: curation of alternative motifs for a same TF; homologous proteins sharing a particular DNA binding domain, motifs discovered with analytic workflows combining several algorithms (e.g. RSAT peak-motifs, or MEME-chip).

The RSAT tool matrix-clustering addresses these needs, and includes several interesting features which will be illustrated in this tutorial.

1. For the computation of inter-matrices distances, support for a large series of alternative metrics (Pearson correlation, Euclidian distance, SSD, Sandelin-Wasserman, logo dot product, and length-normalized version of these scores).
2. Possibility to select a custom combination between several of these similarity metrics, in order to compute an integrative threshold.
3. The set of input motifs is split into separate clusters, each of which canbe displayed in user-interactive ways.
4. User-friendly display of motif trees with aligned logos and consensuses.
5. At each level of the hierarchical tree, all the descendent matrices are aligned (multiple alignement), and a merged motif is computed (branch motif), which can be represented schematically as a branch logo or a branch consensus.

In this tutorial, we explain how to tune the parameters and interpret of results of matrix-clustering.

1. Motif redundancy: examples in motif-discovery results and in motif databases.
2. Thresholds: setting a combination of similarity measures values as a threshold to define the groups of similarities.
3. Impact of parameters: some example showing how changing the values of the parameters can affect cluster composition or tree topology.

Study cases

Study case 1

Goal: clustering a set of partly redundant motifs discovered by various algorithms.

Data set: To illustrate the use of motif clustering to filter out redundancy, we will analyze a set of motifs discovered with the workflow peak-motifs, which combines several motif discovery approaches:

1. oligo-analysis
2. position-analysis
3. local-word-analysis
4. dyad-analysis

We ran these algorithms on a set of peak sequences obtained by pulling down genomic regions bound by the transcription factor Oct4 in mouse ES cells. This experiment had been performed in the context of a wider study, where Chen and colleagues characterized the binding location of 12 transcription factors involved in mouse embryonic stem cell differentiation (Chen et al., 2008).

In this example,

1. We will run matrix-clustering with its default parameters, which were tuned to generally provide a suitable result;
2. We will then run the same analysis with alternative threshold values, in order to analyze the impact of this crucial parameter on the result.
3. Finally, we will run a negative control by randomizing the motifs (random permutations of the columns for each matrix) and running the clustering. In principle, the clustering algorithm should assign each matrix to a singleton (cluster containing only one element).

Study case 2

Beware: this tutorial requires to let the program running for a while, and mobilize important computer resources. It is thus only available for the command line version of the tool.

Goal: cluster all motifs from specific sections (Vertebtrates, Insects) of the JASPAR database (Sandelin et al., 2004), in order to identify groups of similar motifs, which may be bound by TFs of the same family (e.g. several factors of the Hox family).

Note that in some cases factors with unrelated DNA-binding domain may recognize partly similar motifs (e.g. yeast Pho4p and Met4p recognize a motif centered on CACGTG, but differing by the flanking residues).

Test sets

• Test set 1

  The set of 21 motifs discovered with peak-motifs in Oct4 ChIP-seq.

  Discovered motifs in Oct4 ChIP-seq peaks

• Test set 2

  JASPAR vertebrate and insect motifs are available on the section data of RSAT.

  JASPAR insect core motifs (136 matrices, 2013 release)

  JASPAR vertebrate core motifs (263 matrices, 2013 release)

Tuning matrix-clustering parameters

• Study case 1

  Running matrix-clustering to group partly redundant motifs.

  NOTE: This section will be done using the command line

  1. Open a terminal.

  2. Move to a directory where the result will be saved.

  3. Execute this command (matrix-clustering demo):

     		 matrix-clustering -v 2 -i \
     		 $RSAT/public_html/demo_files/peak-motifs_Oct4_matrices.tf \
     		 -matrix_format tf -cons -hclust_method average \
     		 -lth Ncor 0.4 -lth cor 0.6 -lth w 5 -label name,consensus \
     		 -o results/peak-motifs_Oct4/Oct4_analysis_Ncor_0.4_cor_0.6/Oct4_analysis_Ncor_0.4_cor_0.6
     	       

  4. Execute this command (changing the threshold values affect the resulting clusters):

     		matrix-clustering -v 2 -i \
     		$RSAT/public_html/demo_files/peak-motifs_Oct4_matrices.tf \
     		-matrix_format tf -cons -hclust_method average \
     		-lth Ncor 0.55 -lth cor 0.625 -lth w 5 -label name,consensus \
     		-o results/peak-motifs_Oct4/Oct4_analysis_Ncor_0.55_cor_0.625/Oct4_analysis_Ncor_0.55_cor_0.625
     	      

     You can read the help of the parameters used in matrix-clustering typing on therminal: matrix-clustering -h

     • Note: You can change the lower threshold values with the parameter -lth [cor|Ncor|w]. Example: -lth Ncor 0.6 -lth cor 0.75 -lth w 5

     • Note: You can change the input motif with the parameter -i. Example: -i folder/motif_file.tf -format tf

     • Note: You can change the output folder name with the parameter -o. Example: -o folder/prefix

     • Note: You can change the clustering method with the parameter -hclust_method [average|complete|single]. Example: -hclust_method single

  
  

• Negative control

  NOTE: This section will be done using the command line

  1. Open a terminal.

  2. Move to a directory where the negative control results will be saved.

  3. Execute this command:

     		convert-matrix -i $RSAT/public_html/demo_files/peak-motifs_Oct4_matrices.tf -from tf -to tf -perm 1 -o peak-motifs_Oct4_matrices_permuted.tf
     	      

     You can read the help of the parameters used in convert-matrix typing on therminal: convert-matrix -h

  4. Execute this command:

     		matrix-clustering -v 2 -i peak-motifs_Oct4_matrices_permuted.tf \
     		-matrix_format tf -cons -hclust_method average \
     		-lth Ncor 0.4 -lth cor 0.6 -lth w 5 -label name,consensus \
     		-o results/peak-motifs_Oct4/Oct4_analysis_Ncor_0.4_cor_0.6_permuted/Oct4_analysis_Ncor_0.4_cor_0.6_permuted
     	      




• Study case 2

  Running matrix-clustering to identify groups of TFs belonging to the same family

  NOTE: This section will be done using the command line

  1. Open a terminal.

  2. Move to a directory where the result will be saved.

  3. Execute the next commands:

     • JASPAR insect core (136 motifs):

     		  matrix-clustering -v 2 -i \
     		  $RSAT/data/motif_databases/JASPAR/jaspar_core_insects_2013-11.tf \
     		  -matrix_format tf -cons -hclust_method average \
     		  -lth Ncor 0.5 -lth cor 0.7 -lth w 5 -label name,consensus \ 
     		  -o results/JASPAR_insects/Jaspar_insects
     		

     • JASPAR vertebrate core (263 motifs):

     		  matrix-clustering -v 2 -i
     		  $RSAT/data/motif_databases/JASPAR/jaspar_core_vertebrates_2013-11.tf
     		  -matrix_format tf -cons -hclust_method average -lth
     		  Ncor 0.5 -lth cor 0.7 -lth w 5 -label name,consensus
     		  -o results/JASPAR_vertebrates/Jaspar_vertebrates
     		

     Remember: you can read the help of the parameters used in matrix-clustering typing: matrix-clustering -h

     Note:These are bigger dataset than the study case 1 and matrix-clustering will take much more time to analyse them.

Interpreting the results

• Study case 1

  Running matrix-clustering with thresholds

  In this example we will study a set of motifs dicovered in a ChIP-seq experiment done for the TF Oct4 (Pou5f1) which is an essential TF in cell fate decision, ES cells and early embryonic development, it binds the canonical sequence 5'-ATGCAAAT-3'.

  In ES cells, Oct4 often interacts with another TF, Sox2, which binds to an adjacent Sox motif 5'-CATTGTA-3'. Together, both TFs co­regulate specific genes.

  During the analysis of Oct4 or Sox2 binding peaks, the so-called SOCT motif is usually found, which is a composite motif encompassing both Oct and Sox motifs. (Figure 1)

  1. Go to the results folder: results/peak-motifs_Oct4/Oct4_analysis_Ncor_0.4_cor_0.6
  2. Open in a web browser (Firefox is recommended) the file: Oct4_analysis_Ncor_0.4_cor_0.6_SUMMARY.html

  This file has the link to all the output files and show separately the clusters found in the analysis.

  You will find the next files:

  1. Summary table: shows the parameters used for the analysis, the number of input motifs and the number of clusters.
  2. General Alignment table: shows the global alignment of all input motifs, separated on their corresponding clusters.
  3. Buttons with the results: There is one button for each cluster analysis.
     • Cluster aligment table
     • Cluster tree
     • Cluster branch-motif table
  4. List with additional files: output files with general information of all motifs (not cluster specific)
     • Distance table
     • Internal node attributes table
     • Logo Forest
     • Pairwise comparison table
     • Motif description table
     • Consensus tree
     
     
  3. Observe the Alignment table. (Figure 2)
     • You can notice how the motifs are separated in different clusters, observe that each aligment has a different width.
     • Some motifs are not asociated to a cluster and remain as singleton.
     
     
  4. Click the cluster buttons and display the aligment tables and logo trees.
  5. In the logo tree you can observe (for clusters with at least two motifs) that each branch has a consensus, we called it branch-consensus which summarizes all the descendant motifs. (Figure 3)
  6. Display the branch-motifs table. You can donwload the motifs corresponding to these branch-motifs by clicking on the links in this table. (Figure 4)
  7. Go to the list of additional files and display the logo forest. In this file you can observe a set of trees (Forest) with all the logo trees of this analysis. (Figure 5)
     • Some single motifs are considered as a cluster with a single member (singleton).
     • Let's focus on cluster 1. On its logo tree we can observe two groups, one group with 6 members sharing the motif ATGCAAAT (which actually corresponds to Oct4 motif) and the second group with five members sharing the bigger motif TTGATATGCAAAT (corresponding to the SOCT motif).
     • By clicking on the branch-consensus for these groups you can observe the logo. We use the program compare-matrices to compare these branch-motifs against the JASPAR vertebrates motifs, the short motif in fact correspond to Oct4 and the other to Sox2::Pou5f1 (SOCT). (Figure 6)
     • This means some peaks only contain Oct4 bound, however, others have bound Oct4 + Sox2, and for these reason the SOCT motif was founded.
  
  
  8. Now we are going to show the differences of the trees when we change the hierarchical clustering agglomeration rule.
  • By default the rule used in matrix-clustering is the average method, but also can be used the single and complete methods.
  • Remember you can change this option with the parameter -hclust_method .
  
  
  9. In Figure 7 you can observe the differences in the tree topologies and cluster size when we use exactly the same 21 motifs and thresholds but with a different agglomeration rule selected.
  10. Go to the results folder: results/peak-motifs_Oct4/Oct4_analysis_Ncor_0.55_cor_0.625
  11. Open in a web browser (Firefox is recommended) the file: Oct4_analysis_Ncor_0.55_cor_0.625_SUMMARY.html
  12. Notice in the alignment table that there are more clusters (8 and 6 in this and the previous example, respectively)
  13. Click on the buttons to display the results of cluster_1 and cluster_3
      • Compare both clusters with the cluster_1 in the previous example (Ncor>=0.6; cor>=0.4). Notice how the cluster_1 of the previous example was split and actually these motifs are those belonging to the cluster_1 and cluster_3 in the current analaysis. (Figure 8)
      • These clusters correspond to SOCT and Oct4 motifs, respectively. See Figure 6.
      • This example shows how the number of clusters and their composition can vary according the user-selected threshold values.
  
  
  14. Go to the negative results folder: results/peak-motifs_Oct4/Oct4_analysis_Ncor_0.4_cor_0.6_permuted
  15. In the summary and alignment table you can see the clusters for the randomized motifs. (Figure 9)

      NOTE: the number of clusters in the analysis of the randomized set of motifs may variate.

      NOTE: in this dataset there are two motifs which are almost similar to each other, the A-rich motifs (position_6nt_m2 and position_7nt_m3). Notice that after randomization these motifs remain almost similar and is probably that they will group together.

  
  
  
  Fig case 1:
  Figure 1
  Figure 2
  Figure 3
  Figure 4
  Figure 5
  Figure 6
  Figure 7
  Figure 8
  Figure 9
  
  
  
  

  Figure 1. 3D model showing the cooperative binding between Sox2 and Oct4 TFs whose closely interact to bind DNA. Together, they recognize a composite motif called the SOCT motif (SOx+OCT).

  Oct4 ChIP-seq discovered motifs table WITH thresholds

  

  Figure 2. Table with the 21 motifs discovered by peak-motifs in the Oct4 ChIP-seq peaks andalized with matrix-clustering. Ncor<=0.4; cor>=0.6:

  Cluster 1 logo tree

  

  Figure 3. Logo tree of the cluster 1 found in the Oct4 ChIP-seq motifs. The hierarchical tree displays the logo aligment in both orientations. For each branch is calculated a branchwise-motif.

  Cluster 1 branch-motifs table

  

  Figure 4. Branch-motif table for cluster 1. You can download the motif in TRANSFAC format or the logo in both orientations by clicking on them.

  Logo Forest

  

  Figure 5. Logo Forest with the 21 motifs discovered by peak-motifs in the Oct4 ChIP-seq peaks. Using a combination of values as threshold (cor = 0.6; Ncor = 0.4) these motif were separated in 6 different clusters and each one is displayed in a tree.

  Branch-motif analysis

  

  Figure 6. The logo tree of cluster one showing the branch-motifs

  
  
  

  Consensus tree - Average method

  
  
  
  
  

  Consensus tree - Complete method

  
  
  

  Consensus tree - Single method

  

  Figure 7. Three examples of consensus tree when we are using the same data (21 motifs discovered by peak-motifs in the Oct4 ChIP-seq) and the same threslhold values (cor >= 0.4; Ncor >= 0.6).

  In this picture we only change the hierarchical clustering agglomeration rule. From top to down: average, complete, single linkage.Each cluster is represented with a different color. Observe how the number of clusters and the tree topology change depending on the selected method.

  Logo tree for cluster_1

  

  
  

  Logo tree for cluster_3

  

  Figure 8. Logo trees for cluster 1 and 3 which actually correspond to the SOCT and Oct4 motifs respectively. The threshold parameters used were: Ncor>=0.55 and cor>=0.625 .

  Summary table

  

  Figure 9. Summary table of an example of matrix-clustering results with randomized motifs

• 
  
  Study case 2

  Identifying groups of motifs belonging to the same TF Family with matrix-clustering

  matrix-clustering can be used to group motifs bound by TF belonging to the same TF family because usually they share a common DNA binding domain.

  We will analyze separately two sections of the 'core' JASPAR database, corresponding to insect and vertebrate motifs.

  It must be noticed that even the curated databases may contain different motifs for the same TF because more than one different technique can be used to discover the binding motif.

  Before the analysis with matrix-clustering, for both cases, the motifs of each collection where compared to each other, using RSAT compare-matrices, the resulting motif-to-motif similarities were visualized as a motif-to-motif network using Cytoscape where each motif is represented as a vertex and the edges denote pairs of similar motif, with color and thickness reflecting the similarity scores.(Figures 10 and 11).

  This network representation allows a visualization of the groups, however this visualization does not provide a partition of the data into proper clusters. So we will run matrix-clustering to study the similar motifs.

  1. Go to the results folder

     results/JASPAR_insects
     results/JASPAR_vertebrates

  2. For each analysis open the Summary file (We recommend to use Firefox)
  3. Insects motifs:
     • In the motif-to-motif network (Figure 10) you can see a big group of motifs encompassing most motif in this database, as they are highly connected in the network this means they are very similar to each other.
     • To visualize the high similarity you can observe either:
       • Alignment table
       • Logo Forest
       • Consensus tree
       • Cluster 4 results (Figure 12)
       
       
     • Observe that the motifs in this big cluster have the sequence 5'-TTAATT-3' which correspond to Hox family TFs. We could think most insect TFs correspond to Hox family, however the topology observed in the motif-to-motif network can be explained because there is a high enrichment of motifs belonging to this TF family in JASPAR, many research groups study these TFs and upload to this database the motifs they found.
       
  4. Vertebrate motifs:
     • In the motif-to-motif network (Figure 10) you can see a many groups of motifs (i.e. GATA, SOX, FOX).
     • Open the Logo Forest file note that in many clusters the motifs grouped belong to the same Family. (Figure 13)
  
  
  Fig case 2:
  Figure 10
  Figure 11
  Figure 12
  Figure 13
  
  
  
  

  Figure 10. Motif-to-motif network done with Cytoscape for all the JASPAR insect motifs. Notice the big group encompassing most of the motifs of the network.

  

  Figure 11. Motif-to-motif network done with Cytoscape for all the JASPAR vertebrate motifs.

  

  Figure 12. Logo tree showing the highly similar motif visualized in the motif-to-motif insects network.

  

  Figure 13. Some clusters resulting from matrix-clustering are signaled in the motif-to-motif network of vertebrate motifs.

References

1. Chen, X., Xu, H., Yuan, P., Fang, F., Huss, M., Vega, V. B., Wong, E., Orlov, Y. L., Zhang, W., Jiang, J., Loh, Y. H., Yeo, H. C., Yeo, Z. X., Narang, V., Govindarajan, K. R., Leong, B., Shahab, A., Ruan, Y., Bourque, G., Sung, W. K., Clarke, N. D., Wei, C. L. and Ng, H. H. (2008). Integration of external signaling pathways with the core transcriptional network in embryonic stem cells. Cell 133, 1106-17. [Pubmed 18555785].
2. Thomas-Chollier, M., Herrmann, C., Defrance, M., Sand, O., Thieffry, D. and van Helden, J. (2011). RSAT peak-motifs: motif analysis in full-size ChIP-seq datasets Nucleic Acids Research doi:10.1093/nar/gkr1104, 9. [Open access]
3. Mathelier, A., Zhao, X., Zhang, A. W., Parcy, F., Worsley-Hunt, R., Arenillas, D. J., Buchman, S., Chen, C.-y., Chou, A., Ienasescu, H., Lim, J., Shyr, C., Tan, G., Zhou, M., Lenhard, B., Sandelin, A. and Wasserman, W. W. JASPAR 2014: an extensively expanded and updated open-access database of transcription factor binding profiles Nucleic Acids Research, 2013 [Open access]