Commit c311749f authored by Christian Arnold's avatar Christian Arnold
Browse files

Documentation updates regarding the cluster usage of diffTF and updated...

Documentation updates regarding the cluster usage of diffTF and updated cluster configuration templates
parent 012f9f32
Copyright (c) 2017 Christian Arnold (carnold@embl.de) & Ivan Berest (berest@embl.de)
Copyright (c) 2018 Christian Arnold (carnold@embl.de) & Ivan Berest (berest@embl.de)
Permission is hereby granted, free of charge, to any person obtaining a copy of this software
and associated documentation files (the "Software"), to deal in the Software without
......
......@@ -3,11 +3,11 @@
Try it out now!
============================================================
``diffTF`` runs on Linux and macOS and is even independent on the operating system if combined with ``Singularity``. The following quick start briefly summarizes the necessary steps to install and use it.
*diffTF* runs on Linux and macOS and is even independent on the operating system if combined with ``Singularity``. The following quick start briefly summarizes the necessary steps to install and use it.
Principally, there are two ways of installing ``diffTF`` and the proper tools:
Principally, there are two ways of installing *diffTF* and the proper tools:
1a. The "easy" way: Using ``Singularity`` and our preconfigured ``diffTF`` containers that contain all necessary tools, R, and R libraries
1a. The "easy" way: Using ``Singularity`` and our preconfigured *diffTF* containers that contain all necessary tools, R, and R libraries
You only need to install Snakemake (see below for details) and ``Singularity``. Snakemake supports Singularity in Versions >=2.4. You can check whether you already have ``Singularity`` installed by simply typing
......@@ -94,7 +94,7 @@ Principally, there are two ways of installing ``diffTF`` and the proper tools:
Prerequisites for the "easy" way
==================================
The only prerequisite here is that Snakemake and ``Singularity`` must be installed on the system you want to run ``diffTF``. See above for details with respect to the supported versions etc. For details how to install Snakemake, see below.
The only prerequisite here is that Snakemake and ``Singularity`` must be installed on the system you want to run *diffTF*. See above for details with respect to the supported versions etc. For details how to install Snakemake, see below.
Prerequisites for the "manual" way
......@@ -110,7 +110,7 @@ Please ensure that you have at least version 5.3 installed. Principally, there a
*samtools*, *bedtools*, *Subread*
----------------------------------
In addition, `samtools <http://www.htslib.org/download>`_, `bedtools <http://bedtools.readthedocs.io>`_ and `Subread <http://subread.sourceforge.net>`_ are needed to run ``diffTF``. We recommend installing them, along with all the other required software, via conda.
In addition, `samtools <http://www.htslib.org/download>`_, `bedtools <http://bedtools.readthedocs.io>`_ and `Subread <http://subread.sourceforge.net>`_ are needed to run *diffTF*. We recommend installing them, along with all the other required software, via conda.
R and R packages
......@@ -134,9 +134,9 @@ Running your own analysis is almost as easy as running the example analysis. Car
1. Copy the files ``config.json`` and ``startAnalysis.sh`` to a directory of your choice.
2. Modify the file ``config.json`` accordingly. For example, we strongly recommend running the analysis for all TF instead of just 50 as for the example analysis. For this, simply change the parameter “TFs” to “all”. See Section :ref:`configurationFile` for details about the meaning of the parameters. Do not delete or rename any parameters or sections.
3. Create a tab-separated file that defines the input data, in analogy to the file ``sampleData.tsv`` from the example analysis, and refer to that in the file ``config.json`` (parameter ``summaryFile``)
3. Create a **tab-separated** file that defines the input data, in analogy to the file ``sampleData.tsv`` from the example analysis, and refer to that in the file ``config.json`` (parameter ``summaryFile``)
4. Adapt the file ``startAnalysis.sh`` if necessary (the exact command line call to Snakemake and the various Snakemake-related parameters). If you run with Singularity, see the section below for modifications.
5. Since running the pipeline is often computationally demanding, read Section :ref:`timeMemoryRequirements` and decide on which machine to run the pipeline. In most cases, we recommend running ``diffTF`` in a cluster environment (see Section :ref:`clusterEnvironment` for details). The pipeline is written in Snakemake, and we strongly suggest to also read Section :ref:`workingWithPipeline` to get a basic understanding of how the pipeline works.
5. Since running the pipeline is often computationally demanding, read Section :ref:`timeMemoryRequirements` and decide on which machine to run the pipeline. In most cases, we recommend running *diffTF* in a cluster environment (see Section :ref:`clusterEnvironment` for details). The pipeline is written in Snakemake, and we strongly suggest to also read Section :ref:`workingWithPipeline` to get a basic understanding of how the pipeline works.
.. _docs-singularityNotes:
......@@ -145,6 +145,6 @@ Adaptations and notes when running with Singularity
============================================================
You only have to add the ``--use-singularity`` argument to Snakemake. In that case, each rule will be executed in pre-configured isolated containers that contain all necessary tools. Please note the following important issues related to ``Singularity``:
- You may want to add the ``--singularity-prefix`` argument to store all ``Singularity`` containers in a central place as opposed to being stored in the local ``.snakemake`` directory. If you intend to run multiple ``diffTF`` analyses in different locations, you can save space and time because the containers won't have to be downloaded each time and stored in multiple locations
- You may want to add the ``--singularity-prefix`` argument to store all ``Singularity`` containers in a central place as opposed to being stored in the local ``.snakemake`` directory. If you intend to run multiple *diffTF* analyses in different locations, you can save space and time because the containers won't have to be downloaded each time and stored in multiple locations
- .. warning:: If you use ``Singularity`` version 3, make sure you have at least version 3.0.2 installed or the latest pull from version 3.0.1, as there was an issue with Snakemake and particular ``Singularity`` versions. For more details, see `here <https://bitbucket.org/snakemake/snakemake/issues/1017/snakemake-process-suspended-upon-execution>`_.
- .. warning:: If you reference files in the ``config.json`` that are located outside of the directory from which you call Snakemake (that is, parent directories), you have to use the ``--singularity-args`` command line argument to bind additional parent directories to the container so they are available inside the container as well. Otherwise, only (!) the directory from which you start the analysis and subfolders are visible inside the container, but no parent folders. In addition, make sure the mounted paths are identical inside and outside the container. For example, if you reference the files ``/g/group1/user1/mm10.fa`` and ``/g/group2/user1/files/bla.txt`` in the config file, use ``--singularity-args "--bind /g:/g"``. Thus, you have to mount the parent directory of all files you reference outside of your current directory. If both files were located in ``/g/group1``, you could therefore also use ``--bind /g/group1:/g/group1``
......@@ -346,6 +346,8 @@ Details
5. Score
6. Strand
.. warning:: *diffTF* will take a long time to run if the number of peaks is too high. We recommend having less than 100,000 peaks. If the number of peaks is higher for your analysis, we strongly recommend filtering the peaks beforehand to include only the most relevant peaks.
.. _parameter_peakType:
......@@ -880,13 +882,55 @@ If *diffTF* should be run in a cluster environment, the changes are minimal due
--notemp --rerun-incomplete --reason --keep-going \
--cores 16 --local-cores 1 --jobs 400 \
--cluster-config path/to/clusterconfigfile \
--cluster " sbatch -p {cluster.queueSLURM} -J {cluster.name} \
-A {cluster.group} -C {cluster.nodes} --cpus-per-task {cluster.nCPUs} \
--cluster " sbatch -p {cluster.queue} -J {cluster.name} \
--cpus-per-task {cluster.nCPUs} \
--mem {cluster.memory} --time {cluster.maxTime} -o \"{cluster.output}\" \
-e \"{cluster.error}\" --mail-type=None --parsable "
Note that the ``--cluster`` argument is the only part that has to be adjusted for your cluster system. In it, you refer to the cluster configuration file via the ``cluster.`` string, followed by the name of the parameter in the cluster configuration. Essentially, you link the content of the configuration file to the cluster system you want to submit the jobs to.
- the corresponding cluster configuration file might look like this:
.. code-block:: json
{
"__default__": {
"queue": "htc",
"nCPUs": "{threads}",
"memory": 10000,
"name": "{rule}.{wildcards}",
"output": "{rule}.{wildcards}.out",
"error": "{rule}.{wildcards}.err"
},
"intersectPeaksAndPWM": {
"queue": "1day",
"memory": 5000
},
"intersectPeaksAndBAM": {
"queue": "1day",
"memory": 5000
},
"sortPWM": {
"memory": 1000
},
"filterSexChromosomesAndSortPeaks": {
"memory": 1000
},
"intersectTFBSAndBAM": {
"queue": "1day",
"memory": 5000
}
}
A few motes might help you to get started:
- **each** name in the ``--cluster`` argument string from the command line (here: ``queue``, ``name`` ``nCPUs``, ``memory``, ``maxTime``, ``output``, and ``error``) must appear also in the ``__default__`` section of the referenced cluster configuration file (via ``--cluster-config``)
- the ``--cluster`` argument is the only part that has to be adjusted for your cluster system. It is quite simple really, you essentially just link the content of the configuration file to the cluster system you want to submit the jobs to. More specifically, you refer to the cluster configuration file via the ``cluster.`` string, followed by the name of the parameter in the cluster configuration. For parameters that refer to filenames, an extra escaped quotation mark ``\"`` has been added so that the command also works in case of spaces in filenames (which should *always* be avoided at all costs)
- the cluster configuration file has multiple sections defined that correspond to the names of the rules as defined in the Snakefile, plus the special section ``__default__`` at the very top, the latter of which specifies the default cluster options that apply to all rules unless overwritten via its own rule-specific section
- **each** name (e.g., here: ``queue``, ``name`` ``nCPUs``, ```memory``, ``maxTime``, ``output``, and ``error``) **must be defined** in the ``__default__`` section of the cluster configuration file
- note that in this example, we provided some extra parameters for convenience such as ``name`` (so the cluster job will have a reasonable name and can be recognized) that are not strictly necessary
- the ``{threads}`` syntax of the ``nCPUs`` name can be generally used and is a placeholder for the specified number of threads for the particular rule, as specified in the corresponding ``Snakefile``
- in our example, memory is given in Megabytes, so 5000 refers to roughly 5 GB. Queue names are either ``htc`` or ``1day``. Adjust this accordingly to your cluster system.
- .. note:: From a practical point of view, just try to mimic the parameters that you usually use for your cluster system, and modify the cluster configuration file accordingly. For example, if you need an additional argument such as ``-A`` (which stands for the *group* you are in for a SLURM-based system), simply add ``-A {cluster.group}`` to the command line call and add a ``group`` parameter to the ``__default__`` section (see also the note below).
Frequently asked questions
......
.fa:before{-webkit-font-smoothing:antialiased}.clearfix{*zoom:1}.clearfix:before,.clearfix:after{display:table;content:""}.clearfix:after{clear:both}@font-face{font-family:FontAwesome;font-weight:normal;font-style:normal;src:url("../font/fontawesome_webfont.eot");src:url("../font/fontawesome_webfont.eot?#iefix") format("embedded-opentype"),url("../font/fontawesome_webfont.woff") format("woff"),url("../font/fontawesome_webfont.ttf") format("truetype"),url("../font/fontawesome_webfont.svg#FontAwesome") format("svg")}.fa:before{display:inline-block;font-family:FontAwesome;font-style:normal;font-weight:normal;line-height:1;text-decoration:inherit}a .fa{display:inline-block;text-decoration:inherit}li .fa{display:inline-block}li .fa-large:before,li .fa-large:before{width:1.875em}ul.fas{list-style-type:none;margin-left:2em;text-indent:-0.8em}ul.fas li .fa{width:0.8em}ul.fas li .fa-large:before,ul.fas li .fa-large:before{vertical-align:baseline}.fa-book:before{content:""}.icon-book:before{content:""}.fa-caret-down:before{content:""}.icon-caret-down:before{content:""}.fa-caret-up:before{content:""}.icon-caret-up:before{content:""}.fa-caret-left:before{content:""}.icon-caret-left:before{content:""}.fa-caret-right:before{content:""}.icon-caret-right:before{content:""}.rst-versions{position:fixed;bottom:0;left:0;width:300px;color:#fcfcfc;background:#1f1d1d;border-top:solid 10px #343131;font-family:"Lato","proxima-nova","Helvetica Neue",Arial,sans-serif;z-index:400}.rst-versions a{color:#2980B9;text-decoration:none}.rst-versions .rst-badge-small{display:none}.rst-versions .rst-current-version{padding:12px;background-color:#272525;display:block;text-align:right;font-size:90%;cursor:pointer;color:#27AE60;*zoom:1}.rst-versions .rst-current-version:before,.rst-versions .rst-current-version:after{display:table;content:""}.rst-versions .rst-current-version:after{clear:both}.rst-versions .rst-current-version .fa{color:#fcfcfc}.rst-versions .rst-current-version .fa-book{float:left}.rst-versions .rst-current-version .icon-book{float:left}.rst-versions .rst-current-version.rst-out-of-date{background-color:#E74C3C;color:#fff}.rst-versions .rst-current-version.rst-active-old-version{background-color:#F1C40F;color:#000}.rst-versions.shift-up .rst-other-versions{display:block}.rst-versions .rst-other-versions{font-size:90%;padding:12px;color:gray;display:none}.rst-versions .rst-other-versions hr{display:block;height:1px;border:0;margin:20px 0;padding:0;border-top:solid 1px #413d3d}.rst-versions .rst-other-versions dd{display:inline-block;margin:0}.rst-versions .rst-other-versions dd a{display:inline-block;padding:6px;color:#fcfcfc}.rst-versions.rst-badge{width:auto;bottom:20px;right:20px;left:auto;border:none;max-width:300px}.rst-versions.rst-badge .icon-book{float:none}.rst-versions.rst-badge .fa-book{float:none}.rst-versions.rst-badge.shift-up .rst-current-version{text-align:right}.rst-versions.rst-badge.shift-up .rst-current-version .fa-book{float:left}.rst-versions.rst-badge.shift-up .rst-current-version .icon-book{float:left}.rst-versions.rst-badge .rst-current-version{width:auto;height:30px;line-height:30px;padding:0 6px;display:block;text-align:center}@media screen and (max-width: 768px){.rst-versions{width:85%;display:none}.rst-versions.shift{display:block}}
/*# sourceMappingURL=badge_only.css.map */
.fa:before{-webkit-font-smoothing:antialiased}.clearfix{*zoom:1}.clearfix:before,.clearfix:after{display:table;content:""}.clearfix:after{clear:both}@font-face{font-family:FontAwesome;font-weight:normal;font-style:normal;src:url("../fonts/fontawesome-webfont.eot");src:url("../fonts/fontawesome-webfont.eot?#iefix") format("embedded-opentype"),url("../fonts/fontawesome-webfont.woff") format("woff"),url("../fonts/fontawesome-webfont.ttf") format("truetype"),url("../fonts/fontawesome-webfont.svg#FontAwesome") format("svg")}.fa:before{display:inline-block;font-family:FontAwesome;font-style:normal;font-weight:normal;line-height:1;text-decoration:inherit}a .fa{display:inline-block;text-decoration:inherit}li .fa{display:inline-block}li .fa-large:before,li .fa-large:before{width:1.875em}ul.fas{list-style-type:none;margin-left:2em;text-indent:-0.8em}ul.fas li .fa{width:.8em}ul.fas li .fa-large:before,ul.fas li .fa-large:before{vertical-align:baseline}.fa-book:before{content:""}.icon-book:before{content:""}.fa-caret-down:before{content:""}.icon-caret-down:before{content:""}.fa-caret-up:before{content:""}.icon-caret-up:before{content:""}.fa-caret-left:before{content:""}.icon-caret-left:before{content:""}.fa-caret-right:before{content:""}.icon-caret-right:before{content:""}.rst-versions{position:fixed;bottom:0;left:0;width:300px;color:#fcfcfc;background:#1f1d1d;font-family:"Lato","proxima-nova","Helvetica Neue",Arial,sans-serif;z-index:400}.rst-versions a{color:#2980B9;text-decoration:none}.rst-versions .rst-badge-small{display:none}.rst-versions .rst-current-version{padding:12px;background-color:#272525;display:block;text-align:right;font-size:90%;cursor:pointer;color:#27AE60;*zoom:1}.rst-versions .rst-current-version:before,.rst-versions .rst-current-version:after{display:table;content:""}.rst-versions .rst-current-version:after{clear:both}.rst-versions .rst-current-version .fa{color:#fcfcfc}.rst-versions .rst-current-version .fa-book{float:left}.rst-versions .rst-current-version .icon-book{float:left}.rst-versions .rst-current-version.rst-out-of-date{background-color:#E74C3C;color:#fff}.rst-versions .rst-current-version.rst-active-old-version{background-color:#F1C40F;color:#000}.rst-versions.shift-up{height:auto;max-height:100%}.rst-versions.shift-up .rst-other-versions{display:block}.rst-versions .rst-other-versions{font-size:90%;padding:12px;color:gray;display:none}.rst-versions .rst-other-versions hr{display:block;height:1px;border:0;margin:20px 0;padding:0;border-top:solid 1px #413d3d}.rst-versions .rst-other-versions dd{display:inline-block;margin:0}.rst-versions .rst-other-versions dd a{display:inline-block;padding:6px;color:#fcfcfc}.rst-versions.rst-badge{width:auto;bottom:20px;right:20px;left:auto;border:none;max-width:300px}.rst-versions.rst-badge .icon-book{float:none}.rst-versions.rst-badge .fa-book{float:none}.rst-versions.rst-badge.shift-up .rst-current-version{text-align:right}.rst-versions.rst-badge.shift-up .rst-current-version .fa-book{float:left}.rst-versions.rst-badge.shift-up .rst-current-version .icon-book{float:left}.rst-versions.rst-badge .rst-current-version{width:auto;height:30px;line-height:30px;padding:0 6px;display:block;text-align:center}@media screen and (max-width: 768px){.rst-versions{width:85%;display:none}.rst-versions.shift{display:block}}
This source diff could not be displayed because it is too large. You can view the blob instead.
require=(function e(t,n,r){function s(o,u){if(!n[o]){if(!t[o]){var a=typeof require=="function"&&require;if(!u&&a)return a(o,!0);if(i)return i(o,!0);var f=new Error("Cannot find module '"+o+"'");throw f.code="MODULE_NOT_FOUND",f}var l=n[o]={exports:{}};t[o][0].call(l.exports,function(e){var n=t[o][1][e];return s(n?n:e)},l,l.exports,e,t,n,r)}return n[o].exports}var i=typeof require=="function"&&require;for(var o=0;o<r.length;o++)s(r[o]);return s})({"sphinx-rtd-theme":[function(require,module,exports){
var jQuery = (typeof(window) != 'undefined') ? window.jQuery : require('jquery');
// Sphinx theme nav state
function ThemeNav () {
var nav = {
navBar: null,
win: null,
winScroll: false,
winResize: false,
linkScroll: false,
winPosition: 0,
winHeight: null,
docHeight: null,
isRunning: false
};
nav.enable = function () {
var self = this;
if (!self.isRunning) {
self.isRunning = true;
jQuery(function ($) {
self.init($);
self.reset();
self.win.on('hashchange', self.reset);
// Set scroll monitor
self.win.on('scroll', function () {
if (!self.linkScroll) {
self.winScroll = true;
}
});
setInterval(function () { if (self.winScroll) self.onScroll(); }, 25);
// Set resize monitor
self.win.on('resize', function () {
self.winResize = true;
});
setInterval(function () { if (self.winResize) self.onResize(); }, 25);
self.onResize();
});
};
};
nav.init = function ($) {
var doc = $(document),
self = this;
this.navBar = $('div.wy-side-scroll:first');
this.win = $(window);
// Set up javascript UX bits
$(document)
// Shift nav in mobile when clicking the menu.
.on('click', "[data-toggle='wy-nav-top']", function() {
$("[data-toggle='wy-nav-shift']").toggleClass("shift");
$("[data-toggle='rst-versions']").toggleClass("shift");
})
// Nav menu link click operations
.on('click', ".wy-menu-vertical .current ul li a", function() {
var target = $(this);
// Close menu when you click a link.
$("[data-toggle='wy-nav-shift']").removeClass("shift");
$("[data-toggle='rst-versions']").toggleClass("shift");
// Handle dynamic display of l3 and l4 nav lists
self.toggleCurrent(target);
self.hashChange();
})
.on('click', "[data-toggle='rst-current-version']", function() {
$("[data-toggle='rst-versions']").toggleClass("shift-up");
})
// Make tables responsive
$("table.docutils:not(.field-list)")
.wrap("<div class='wy-table-responsive'></div>");
// Add expand links to all parents of nested ul
$('.wy-menu-vertical ul').not('.simple').siblings('a').each(function () {
var link = $(this);
expand = $('<span class="toctree-expand"></span>');
expand.on('click', function (ev) {
self.toggleCurrent(link);
ev.stopPropagation();
return false;
});
link.prepend(expand);
});
};
nav.reset = function () {
// Get anchor from URL and open up nested nav
var anchor = encodeURI(window.location.hash);
if (anchor) {
try {
var link = $('.wy-menu-vertical')
.find('[href="' + anchor + '"]');
// If we didn't find a link, it may be because we clicked on
// something that is not in the sidebar (eg: when using
// sphinxcontrib.httpdomain it generates headerlinks but those
// aren't picked up and placed in the toctree). So let's find
// the closest header in the document and try with that one.
if (link.length === 0) {
var doc_link = $('.document a[href="' + anchor + '"]');
var closest_section = doc_link.closest('div.section');
// Try again with the closest section entry.
link = $('.wy-menu-vertical')
.find('[href="#' + closest_section.attr("id") + '"]');
}
$('.wy-menu-vertical li.toctree-l1 li.current')
.removeClass('current');
link.closest('li.toctree-l2').addClass('current');
link.closest('li.toctree-l3').addClass('current');
link.closest('li.toctree-l4').addClass('current');
}
catch (err) {
console.log("Error expanding nav for anchor", err);
}
}
};
nav.onScroll = function () {
this.winScroll = false;
var newWinPosition = this.win.scrollTop(),
winBottom = newWinPosition + this.winHeight,
navPosition = this.navBar.scrollTop(),
newNavPosition = navPosition + (newWinPosition - this.winPosition);
if (newWinPosition < 0 || winBottom > this.docHeight) {
return;
}
this.navBar.scrollTop(newNavPosition);
this.winPosition = newWinPosition;
};
nav.onResize = function () {
this.winResize = false;
this.winHeight = this.win.height();
this.docHeight = $(document).height();
};
nav.hashChange = function () {
this.linkScroll = true;
this.win.one('hashchange', function () {
this.linkScroll = false;
});
};
nav.toggleCurrent = function (elem) {
var parent_li = elem.closest('li');
parent_li.siblings('li.current').removeClass('current');
parent_li.siblings().find('li.current').removeClass('current');
parent_li.find('> ul li.current').removeClass('current');
parent_li.toggleClass('current');
}
return nav;
};
module.exports.ThemeNav = ThemeNav();
if (typeof(window) != 'undefined') {
window.SphinxRtdTheme = { StickyNav: module.exports.ThemeNav };
}
},{"jquery":"jquery"}]},{},["sphinx-rtd-theme"]);
/* sphinx_rtd_theme version 0.4.2 | MIT license */
/* Built 20181005 13:10 */
require=function r(s,a,l){function c(e,n){if(!a[e]){if(!s[e]){var i="function"==typeof require&&require;if(!n&&i)return i(e,!0);if(u)return u(e,!0);var t=new Error("Cannot find module '"+e+"'");throw t.code="MODULE_NOT_FOUND",t}var o=a[e]={exports:{}};s[e][0].call(o.exports,function(n){return c(s[e][1][n]||n)},o,o.exports,r,s,a,l)}return a[e].exports}for(var u="function"==typeof require&&require,n=0;n<l.length;n++)c(l[n]);return c}({"sphinx-rtd-theme":[function(n,e,i){var jQuery="undefined"!=typeof window?window.jQuery:n("jquery");e.exports.ThemeNav={navBar:null,win:null,winScroll:!1,winResize:!1,linkScroll:!1,winPosition:0,winHeight:null,docHeight:null,isRunning:!1,enable:function(e){var i=this;void 0===e&&(e=!0),i.isRunning||(i.isRunning=!0,jQuery(function(n){i.init(n),i.reset(),i.win.on("hashchange",i.reset),e&&i.win.on("scroll",function(){i.linkScroll||i.winScroll||(i.winScroll=!0,requestAnimationFrame(function(){i.onScroll()}))}),i.win.on("resize",function(){i.winResize||(i.winResize=!0,requestAnimationFrame(function(){i.onResize()}))}),i.onResize()}))},enableSticky:function(){this.enable(!0)},init:function(i){i(document);var t=this;this.navBar=i("div.wy-side-scroll:first"),this.win=i(window),i(document).on("click","[data-toggle='wy-nav-top']",function(){i("[data-toggle='wy-nav-shift']").toggleClass("shift"),i("[data-toggle='rst-versions']").toggleClass("shift")}).on("click",".wy-menu-vertical .current ul li a",function(){var n=i(this);i("[data-toggle='wy-nav-shift']").removeClass("shift"),i("[data-toggle='rst-versions']").toggleClass("shift"),t.toggleCurrent(n),t.hashChange()}).on("click","[data-toggle='rst-current-version']",function(){i("[data-toggle='rst-versions']").toggleClass("shift-up")}),i("table.docutils:not(.field-list,.footnote,.citation)").wrap("<div class='wy-table-responsive'></div>"),i("table.docutils.footnote").wrap("<div class='wy-table-responsive footnote'></div>"),i("table.docutils.citation").wrap("<div class='wy-table-responsive citation'></div>"),i(".wy-menu-vertical ul").not(".simple").siblings("a").each(function(){var e=i(this);expand=i('<span class="toctree-expand"></span>'),expand.on("click",function(n){return t.toggleCurrent(e),n.stopPropagation(),!1}),e.prepend(expand)})},reset:function(){var n=encodeURI(window.location.hash)||"#";try{var e=$(".wy-menu-vertical"),i=e.find('[href="'+n+'"]');if(0===i.length){var t=$('.document [id="'+n.substring(1)+'"]').closest("div.section");0===(i=e.find('[href="#'+t.attr("id")+'"]')).length&&(i=e.find('[href="#"]'))}0<i.length&&($(".wy-menu-vertical .current").removeClass("current"),i.addClass("current"),i.closest("li.toctree-l1").addClass("current"),i.closest("li.toctree-l1").parent().addClass("current"),i.closest("li.toctree-l1").addClass("current"),i.closest("li.toctree-l2").addClass("current"),i.closest("li.toctree-l3").addClass("current"),i.closest("li.toctree-l4").addClass("current"))}catch(o){console.log("Error expanding nav for anchor",o)}},onScroll:function(){this.winScroll=!1;var n=this.win.scrollTop(),e=n+this.winHeight,i=this.navBar.scrollTop()+(n-this.winPosition);n<0||e>this.docHeight||(this.navBar.scrollTop(i),this.winPosition=n)},onResize:function(){this.winResize=!1,this.winHeight=this.win.height(),this.docHeight=$(document).height()},hashChange:function(){this.linkScroll=!0,this.win.one("hashchange",function(){this.linkScroll=!1})},toggleCurrent:function(n){var e=n.closest("li");e.siblings("li.current").removeClass("current"),e.siblings().find("li.current").removeClass("current"),e.find("> ul li.current").removeClass("current"),e.toggleClass("current")}},"undefined"!=typeof window&&(window.SphinxRtdTheme={Navigation:e.exports.ThemeNav,StickyNav:e.exports.ThemeNav}),function(){for(var r=0,n=["ms","moz","webkit","o"],e=0;e<n.length&&!window.requestAnimationFrame;++e)window.requestAnimationFrame=window[n[e]+"RequestAnimationFrame"],window.cancelAnimationFrame=window[n[e]+"CancelAnimationFrame"]||window[n[e]+"CancelRequestAnimationFrame"];window.requestAnimationFrame||(window.requestAnimationFrame=function(n,e){var i=(new Date).getTime(),t=Math.max(0,16-(i-r)),o=window.setTimeout(function(){n(i+t)},t);return r=i+t,o}),window.cancelAnimationFrame||(window.cancelAnimationFrame=function(n){clearTimeout(n)})}()},{jquery:"jquery"}]},{},["sphinx-rtd-theme"]);
\ No newline at end of file
......@@ -24,27 +24,19 @@
<link rel="stylesheet" href="_static/css/theme.css" type="text/css" />
<link rel="index" title="Index"
href="genindex.html"/>
<link rel="search" title="Search" href="search.html"/>
<link rel="top" title="diffTF 1.0.0 documentation" href="index.html"/>
<link rel="next" title="Workflow" href="chapter2.html"/>
<link rel="prev" title="Welcome to the documentation of diffTF!" href="index.html"/>
<link rel="stylesheet" href="_static/css/theme.css" type="text/css" />
<link rel="stylesheet" href="_static/pygments.css" type="text/css" />
<link rel="index" title="Index" href="genindex.html" />
<link rel="search" title="Search" href="search.html" />
<link rel="next" title="Workflow" href="chapter2.html" />
<link rel="prev" title="Welcome to the documentation of diffTF!" href="index.html" />
<script src="_static/js/modernizr.min.js"></script>
</head>
<body class="wy-body-for-nav" role="document">
<body class="wy-body-for-nav">
<div class="wy-grid-for-nav">
......@@ -129,7 +121,7 @@
<section data-toggle="wy-nav-shift" class="wy-nav-content-wrap">
<nav class="wy-nav-top" role="navigation" aria-label="top navigation">
<nav class="wy-nav-top" aria-label="top navigation">
<i data-toggle="wy-nav-top" class="fa fa-bars"></i>
<a href="index.html">diffTF</a>
......@@ -137,9 +129,10 @@
</nav>
<div class="wy-nav-content">
<div class="rst-content">
......@@ -183,9 +176,9 @@
<div class="section" id="try-it-out-now">
<span id="docs-quickstart"></span><h1>Try it out now!<a class="headerlink" href="#try-it-out-now" title="Permalink to this headline"></a></h1>
<p><code class="docutils literal notranslate"><span class="pre">diffTF</span></code> runs on Linux and macOS and is even independent on the operating system if combined with <code class="docutils literal notranslate"><span class="pre">Singularity</span></code>. The following quick start briefly summarizes the necessary steps to install and use it.</p>
<p>Principally, there are two ways of installing <code class="docutils literal notranslate"><span class="pre">diffTF</span></code> and the proper tools:</p>
<p>1a. The “easy” way: Using <code class="docutils literal notranslate"><span class="pre">Singularity</span></code> and our preconfigured <code class="docutils literal notranslate"><span class="pre">diffTF</span></code> containers that contain all necessary tools, R, and R libraries</p>
<p><em>diffTF</em> runs on Linux and macOS and is even independent on the operating system if combined with <code class="docutils literal notranslate"><span class="pre">Singularity</span></code>. The following quick start briefly summarizes the necessary steps to install and use it.</p>
<p>Principally, there are two ways of installing <em>diffTF</em> and the proper tools:</p>
<p>1a. The “easy” way: Using <code class="docutils literal notranslate"><span class="pre">Singularity</span></code> and our preconfigured <em>diffTF</em> containers that contain all necessary tools, R, and R libraries</p>
<blockquote>
<div><p>You only need to install Snakemake (see below for details) and <code class="docutils literal notranslate"><span class="pre">Singularity</span></code>. Snakemake supports Singularity in Versions &gt;=2.4. You can check whether you already have <code class="docutils literal notranslate"><span class="pre">Singularity</span></code> installed by simply typing</p>
<div class="highlight-Bash notranslate"><div class="highlight"><pre><span></span>singularity --version
......@@ -277,7 +270,7 @@ conda install snakemake bedtools samtools subread
</div>
<div class="section" id="prerequisites-for-the-easy-way">
<span id="docs-prerequisites"></span><h1>Prerequisites for the “easy” way<a class="headerlink" href="#prerequisites-for-the-easy-way" title="Permalink to this headline"></a></h1>
<p>The only prerequisite here is that Snakemake and <code class="docutils literal notranslate"><span class="pre">Singularity</span></code> must be installed on the system you want to run <code class="docutils literal notranslate"><span class="pre">diffTF</span></code>. See above for details with respect to the supported versions etc. For details how to install Snakemake, see below.</p>
<p>The only prerequisite here is that Snakemake and <code class="docutils literal notranslate"><span class="pre">Singularity</span></code> must be installed on the system you want to run <em>diffTF</em>. See above for details with respect to the supported versions etc. For details how to install Snakemake, see below.</p>
</div>
<div class="section" id="prerequisites-for-the-manual-way">
<h1>Prerequisites for the “manual” way<a class="headerlink" href="#prerequisites-for-the-manual-way" title="Permalink to this headline"></a></h1>
......@@ -288,7 +281,7 @@ conda install snakemake bedtools samtools subread
</div>
<div class="section" id="samtools-bedtools-subread">
<h2><em>samtools</em>, <em>bedtools</em>, <em>Subread</em><a class="headerlink" href="#samtools-bedtools-subread" title="Permalink to this headline"></a></h2>
<p>In addition, <a class="reference external" href="http://www.htslib.org/download">samtools</a>, <a class="reference external" href="http://bedtools.readthedocs.io">bedtools</a> and <a class="reference external" href="http://subread.sourceforge.net">Subread</a> are needed to run <code class="docutils literal notranslate"><span class="pre">diffTF</span></code>. We recommend installing them, along with all the other required software, via conda.</p>
<p>In addition, <a class="reference external" href="http://www.htslib.org/download">samtools</a>, <a class="reference external" href="http://bedtools.readthedocs.io">bedtools</a> and <a class="reference external" href="http://subread.sourceforge.net">Subread</a> are needed to run <em>diffTF</em>. We recommend installing them, along with all the other required software, via conda.</p>
</div>
<div class="section" id="r-and-r-packages">
<h2>R and R packages<a class="headerlink" href="#r-and-r-packages" title="Permalink to this headline"></a></h2>
......@@ -306,16 +299,16 @@ biocLite<span class="p">(</span><span class="kt">c</span><span class="p">(</span
<ol class="arabic simple">
<li>Copy the files <code class="docutils literal notranslate"><span class="pre">config.json</span></code> and <code class="docutils literal notranslate"><span class="pre">startAnalysis.sh</span></code> to a directory of your choice.</li>
<li>Modify the file <code class="docutils literal notranslate"><span class="pre">config.json</span></code> accordingly. For example, we strongly recommend running the analysis for all TF instead of just 50 as for the example analysis. For this, simply change the parameter “TFs” to “all”. See Section <a class="reference internal" href="chapter2.html#configurationfile"><span class="std std-ref">General configuration file</span></a> for details about the meaning of the parameters. Do not delete or rename any parameters or sections.</li>
<li>Create a tab-separated file that defines the input data, in analogy to the file <code class="docutils literal notranslate"><span class="pre">sampleData.tsv</span></code> from the example analysis, and refer to that in the file <code class="docutils literal notranslate"><span class="pre">config.json</span></code> (parameter <code class="docutils literal notranslate"><span class="pre">summaryFile</span></code>)</li>
<li>Create a <strong>tab-separated</strong> file that defines the input data, in analogy to the file <code class="docutils literal notranslate"><span class="pre">sampleData.tsv</span></code> from the example analysis, and refer to that in the file <code class="docutils literal notranslate"><span class="pre">config.json</span></code> (parameter <code class="docutils literal notranslate"><span class="pre">summaryFile</span></code>)</li>
<li>Adapt the file <code class="docutils literal notranslate"><span class="pre">startAnalysis.sh</span></code> if necessary (the exact command line call to Snakemake and the various Snakemake-related parameters). If you run with Singularity, see the section below for modifications.</li>
<li>Since running the pipeline is often computationally demanding, read Section <a class="reference internal" href="chapter2.html#timememoryrequirements"><span class="std std-ref">Executing diffTF - Running times and memory requirements</span></a> and decide on which machine to run the pipeline. In most cases, we recommend running <code class="docutils literal notranslate"><span class="pre">diffTF</span></code> in a cluster environment (see Section <a class="reference internal" href="chapter2.html#clusterenvironment"><span class="std std-ref">Running diffTF in a cluster environment</span></a> for details). The pipeline is written in Snakemake, and we strongly suggest to also read Section <a class="reference internal" href="chapter2.html#workingwithpipeline"><span class="std std-ref">Working with diffTF and FAQs</span></a> to get a basic understanding of how the pipeline works.</li>
<li>Since running the pipeline is often computationally demanding, read Section <a class="reference internal" href="chapter2.html#timememoryrequirements"><span class="std std-ref">Executing diffTF - Running times and memory requirements</span></a> and decide on which machine to run the pipeline. In most cases, we recommend running <em>diffTF</em> in a cluster environment (see Section <a class="reference internal" href="chapter2.html#clusterenvironment"><span class="std std-ref">Running diffTF in a cluster environment</span></a> for details). The pipeline is written in Snakemake, and we strongly suggest to also read Section <a class="reference internal" href="chapter2.html#workingwithpipeline"><span class="std std-ref">Working with diffTF and FAQs</span></a> to get a basic understanding of how the pipeline works.</li>
</ol>
</div>
<div class="section" id="adaptations-and-notes-when-running-with-singularity">
<span id="docs-singularitynotes"></span><h1>Adaptations and notes when running with Singularity<a class="headerlink" href="#adaptations-and-notes-when-running-with-singularity" title="Permalink to this headline"></a></h1>
<p>You only have to add the <code class="docutils literal notranslate"><span class="pre">--use-singularity</span></code> argument to Snakemake. In that case, each rule will be executed in pre-configured isolated containers that contain all necessary tools. Please note the following important issues related to <code class="docutils literal notranslate"><span class="pre">Singularity</span></code>:</p>
<ul>
<li><p class="first">You may want to add the <code class="docutils literal notranslate"><span class="pre">--singularity-prefix</span></code> argument to store all <code class="docutils literal notranslate"><span class="pre">Singularity</span></code> containers in a central place as opposed to being stored in the local <code class="docutils literal notranslate"><span class="pre">.snakemake</span></code> directory. If you intend to run multiple <code class="docutils literal notranslate"><span class="pre">diffTF</span></code> analyses in different locations, you can save space and time because the containers won’t have to be downloaded each time and stored in multiple locations</p>
<li><p class="first">You may want to add the <code class="docutils literal notranslate"><span class="pre">--singularity-prefix</span></code> argument to store all <code class="docutils literal notranslate"><span class="pre">Singularity</span></code> containers in a central place as opposed to being stored in the local <code class="docutils literal notranslate"><span class="pre">.snakemake</span></code> directory. If you intend to run multiple <em>diffTF</em> analyses in different locations, you can save space and time because the containers won’t have to be downloaded each time and stored in multiple locations</p>
</li>
<li><div class="first admonition warning">
<p class="first admonition-title">Warning</p>
......@@ -332,9 +325,7 @@ biocLite<span class="p">(</span><span class="kt">c</span><span class="p">(</span
</div>
<div class="articleComments">
</div>
</div>
<footer>
......@@ -352,11 +343,11 @@ biocLite<span class="p">(</span><span class="kt">c</span><span class="p">(</span
<div role="contentinfo">
<p>
&copy; Copyright 2017, Christian Arnold, Ivan Berest, Judith B. Zaugg.
&copy; Copyright 2017, Christian Arnold, Ivan Berest, Judith B. Zaugg
</p>
</div>
Built with <a href="http://sphinx-doc.org/">Sphinx</a> using a <a href="https://github.com/snide/sphinx_rtd_theme">theme</a> provided by <a href="https://readthedocs.org">Read the Docs</a>.
Built with <a href="http://sphinx-doc.org/">Sphinx</a> using a <a href="https://github.com/rtfd/sphinx_rtd_theme">theme</a> provided by <a href="https://readthedocs.org">Read the Docs</a>.
</footer>
......@@ -371,36 +362,23 @@ biocLite<span class="p">(</span><span class="kt">c</span><span class="p">(</span
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT:'./',
VERSION:'1.0.0',
COLLAPSE_INDEX:false,
FILE_SUFFIX:'.html',
HAS_SOURCE: true,
SOURCELINK_SUFFIX: '.txt'
};
</script>
<script type="text/javascript" src="_static/jquery.js"></script>
<script type="text/javascript" src="_static/underscore.js"></script>
<script type="text/javascript" src="_static/doctools.js"></script>
<script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS-MML_HTMLorMML"></script>
<script type="text/javascript" id="documentation_options" data-url_root="./" src="_static/documentation_options.js"></script>
<script type="text/javascript" src="_static/jquery.js"></script>
<script type="text/javascript" src="_static/underscore.js"></script>
<script type="text/javascript" src="_static/doctools.js"></script>
<script type="text/javascript" src="_static/js/theme.js"></script>
<script type="text/javascript" src="_static/js/theme.js"></script>
<script type="text/javascript">
jQuery(function () {
SphinxRtdTheme.StickyNav.enable();
SphinxRtdTheme.Navigation.enable(true);
});
</script>
</script>
</body>
</html>
\ No newline at end of file
......@@ -24,27 +24,19 @@
<link rel="stylesheet" href="_static/css/theme.css" type="text/css" />
<link rel="index" title="Index"
href="genindex.html"/>
<link rel="search" title="Search" href="search.html"/>
<link rel="top" title="diffTF 1.0.0 documentation" href="index.html"/>
<link rel="next" title="Biological motivation" href="projectInfo.html"/>
<link rel="prev" title="Try it out now!" href="chapter1.html"/>
<link rel="stylesheet" href="_static/css/theme.css" type="text/css" />
<link rel="stylesheet" href="_static/pygments.css" type="text/css" />
<link rel="index" title="Index" href="genindex.html" />
<link rel="search" title="Search" href="search.html" />
<link rel="next" title="Biological motivation" href="projectInfo.html" />
<link rel="prev" title="Try it out now!" href="chapter1.html" />
<script src="_static/js/modernizr.min.js"></script>
</head>
<body class="wy-body-for-nav" role="document">
<body class="wy-body-for-nav">
<div class="wy-grid-for-nav">
......@@ -232,7 +224,7 @@
<section data-toggle="wy-nav-shift" class="wy-nav-content-wrap">
<nav class="wy-nav-top" role="navigation" aria-label="top navigation">
<nav class="wy-nav-top" aria-label="top navigation">
<i data-toggle="wy-nav-top" class="fa fa-bars"></i>
<a href="index.html">diffTF</a>
......@@ -240,9 +232,10 @@
</nav>