- The ability to log into the CLC Genomics Server can now be restricted to members of specified groups.
- Data created in the CLC Genomics Server 11.0 and stored in a file server location will be internally compressed by default. Internal compression is also the default in the corresponding client Workbench software. Internal compression is enabled per default, but can be disabled using the admin interface for the server (under Main configuration | Data compression). Corresponding functionality is also available in the client Workbenches. Data that is internally compressed can be exported to CLC or ZIP format without this compression using options provided in the client software.
- The tools delivered with a server plugin can now be listed by clicking on the name of an installed plugin under the Plugins tab in the web administrative interface.
- When cancelling or re-queuing jobs listed in the Queue tab of the web administrative interface, confirmation is now sought before any action is taken.
- The size a of file in a server Import/export location can be now be seen within the import wizards of CLC Workbench client software by hovering the mouse cursor over the filename.
- Various minor improvements
- Fixed an issue when using launchd on macOS, where the Restart option under the Status and Management tab of the web administrative interface would stop the server but not restart it, which could affect restarting the service when the system was rebooted and when the service was started up automatically at the end of installing an upgraded version.
- Various minor bugfixes
A flag in the CLCGenomicsServer.vmoptions file must be removed when upgrading in place to the CLC Genomics Server 11.0 on macOS.
- Licenses for CLC Genomics Server modules on a grid node or job node setup are now only needed on the master server. Previously, licenses were also needed for the nodes.
- The license formerly required to run tools and workflows delivered as part of the Biomedical Genomics Server solution is no longer needed. To get access to tools and workflows relevant to biomedical genomics and QIAseq panel data analysis, a valid CLC Genomics Server license is needed and the Biomedical Genomics Analysis Server Plugin must be installed.
- For systems enabled to connect to an SQL database,
- JDBC drivers need to be installed before configuring a database location. Previously this was only the case for MySQL and Oracle databases.
- The H2 database is no longer supported and is not included with the product.
Please delete "-d64" from the CLCGenomicsServer.vmoptions file, which can be found in the CLC Genomics Server installation area and then restart the CLC Genomics Server service. The -d64 option is not supported by recent versions of java. Its inclusion in the vmoptions file on macOS systems will stop the CLC Genomics Server from starting up.
Shared with workbenches
Bisulfite Sequencing Analysis
- Three tools for analyzing cytosine methylation data are now available: Map Bisulfite Reads to Reference, Call Methylation Levels, and Create RRBS-fragment Track. These tools reveal methylated cytosines genome wide and at single base level resolution, support statistical comparison between samples accommodating different experimental designs, and support reduced representation sequencing. These tools were formerly available via the Bisulfite Sequencing Server Plugin, but are now integrated into the server software.
- The Map Bisulfite Reads to Reference offers the option to enable global alignments to produce read mappings with no unaligned ends, which was not formerly possible.
- The default "cost of insertions and deletions" in the Map Bisulfite Reads to Reference tool is now "affine". This improves results on internal benchmarks because it breaks a symmetry in the default "linear" scoring for reads ending in homopolymers (which are abundant in bisulfite mapping due to in-silico conversion of the reads and references to a 3 letter alphabet). This symmetry meant that either a mismatch or an insertion could be introduced at the ends of some of these reads without changing the mapping score. In practice the mismatch is more plausible, and this is favored by the affine penalties.
- Import Primer Pairs for importing primer pair locations from a generic text format file or from a QIAGEN gene panel primer file. This tool was formerly only available in the Biomedical-enabled CLC Genomics Servers.
- Import QIAGEN GeneReader for importing QIAGEN GeneReader data.
- Copy Number Variation Detection (CNVs), for detecting copy number variations (CNVs) from targeted resequencing experiments. Using read mappings and target regions as input, it produces amplification and deletion annotations. This tool was formerly only available in the Biomedical-enabled CLC Genomics Servers.
- Remove Information from Variants for removing annotations on variants. This tool was formerly only available in the Biomedical-enabled CLC Genomics Servers.
- Remove Orphan Reference Variants for removing reference allele variants that lack a corresponding non-reference variant allele.
- Differential Expression in Two Groups to be used instead of the more general Differential Expression for RNA-Seq tool for testing differential expression between a single treatment group and a control group. Both these tools take the same input, but Differential Expression in Two Groups does not require a metadata table to describe the experimental design.
- Variant tracks now include Forward coverage and Reverse coverage annotations.
- The following tools have changed name:
RNA-Seq Analysis tool improvements
- The RNA-Seq Analysis tool supports the alignment and quantification of reads that wrap around the ends of circular chromosomes.
- The tool caches the data structure used by the read mapper to map reads to known mRNA annotations. This reduces run time by up to 3 minutes per sample, with the greatest benefits being observed when using large numbers of mRNA annotations on systems with few cpu cores.
- A new row has been added to the "Strand specificity" section of the report produced by the tool. The row contains the number of "Reads with known strand", which is used in determining the percentage of reads ignored due to being on the wrong strand.
- The "Detected transcripts" column has been renamed to "Uniquely identified transcripts" for both the gene-level and transcript-level expression tracks.
- The "Reference Sequence" section of the report now lists the number and length of all chromosomes used during read mapping. Previously it reported only the length and number of chromosomes with at least one genes or transcript.
- The RNA-Seq Analysis and Map Reads to Reference tools can now share cached copies of the read mapper indexes. This means that the average run time over many samples will be reduced if both tools are frequently used.
- The RNA-Seq Analysis tool is now sometimes able to avoid writing the reference to disk. The changes are most noticeable when batch processing many samples against the same large reference.
- Read mappings produced by the RNA-Seq Analysis tool previously ignored deletions and insertions at exon-intron boundaries. This meant that such deletions/insertions would not be detectable in downstream variant calling. The tool has been updated to keep the deletions and insertions in the mapping, implicitly favoring the hypothesis of a deletion/insertion over a novel splice junction. This change does not affect expression levels.
Amino Acid Changes tool improvements
- The Amino Acid Changes tool previously used square brackets to describe coding region and amino acid changes when a single variant affected multiple transcripts or proteins, e.g., NM_207170.3:c.[140C>T]; NM_015484.4:c.[266C>T]. These brackets have now been removed (e.g., NM_207170.3:c.140C>T; NM_015484.4:c.266C>T) to comply with the HGVS standards, which reserve the brackets for the reporting of alleles. These changes are also reported by the variant callers when run on a standalone read-mapping with CDS annotations.
- The tool describes replacements in the compact format preferred by HGVS (112_117delinsTG). Previously the description included the reference sequence (112_117delAGGTCAinsTG). These changes are also reported by the variant callers when run on a standalone read-mapping with CDS annotations.
- We implemented the 3' HGVS compliance rule for c. annotation of variants: When doing p. annotations (protein-level HGVS) we similarly annotate insertions that really are duplications as such.
- The tool uses all positions covered by a variant when describing coding region changes, in accordance with HGVS recommendations. Previously the tool restricted its change descriptions to positions within a transcript (if supplied) or CDS. This fix will therefore mainly affect the descriptions of deletions that partially overlap a transcript. These changes are also reported by the variant callers when run on a standalone read-mapping with CDS annotations.
- An option can add c. annotations (HGVS DNA-level) for variants that are within a certain distance from the transcript boundaries. The distance can be configured but defaults are set to 5 kb upstream and 3 kb downstream.
- An option in the Amino Acid Changes tool allows users to output a variant track HGVS compliant.
- An option allows the prioritization of a single transcript when several annotations are available for one variant.
VCF importer and exporter improvements
- The VCF exporter and importer have been improved and now support VCF v4.2.
- VCF Export "Enforce diploid" option has been replaced with an improved and more general "Enforce ploidy" option set by default to 2. This option gives more control over the exported genotype and better compatibility with external applications such as Ingenuity Variant Analysis.
- Four complex variant representations can now be handled by the VCF importer and exporter, including the common reference overlap representation.
- The VCF exporter has an option to write variant annotations as INFO fields.
- In the VCF importer, we fixed an issue with the import of INFO IDs that contained non-alphabetical characters.
BED importer and exporter improvements
- The BED exporter now replaces spaces in feature names with underscores, since white space is not allowed in the BED feature names.
- The BED file exporter now always exports to BED12 format.
- The BED importer limit for name lengths has been raised from 80 to 256 characters.
- The Local Realignment tool has been optimized to run more quickly.
- The De Novo Assembly tool has been updated to use the same version of the read mapper as the one used by the Map Reads to Contigs tool. This typically leads to more accurate mappings. For larger assemblies the run time is expected to decrease on average, but for small assemblies run time is likely to increase.
- Filter Against Known Variants no longer adds duplicate annotations from known variants tracks. In addition, Overlap, Exact match and Partial MNV match annotations are now always added to the output variant track.
- The Import Ion Torrent and Import PacBio tools support import of reads from SAM or BAM format files. Mapping information is discarded during this import. To import a read mapping from SAM or BAM format files, use the existing Import | SAM/BAM Mapping Files... tool.
- Handling of RNA-Seq reads by the InDels and Structural Variants tool has been improved. This change affects breakpoint p-values and as a result, affects the number of breakpoints and variants reported. In addition, we have improved the calculations of the values reported for the "perfectly mapped" and "not perfectly mapped" breakpoint annotations.
- Improvements in the Differential Expression for RNA-Seq tool
- It now accepts RNA-Seq panel samples (including QIAseq panel samples) as input and offers additional normalization options.
- It now outputs statistical comparison tables in addition to the statistical comparison tracks. Tables offer the same functionality as the tracks, except for the track view.
- Improvements to the Annotate with Overlap Information tool:
- It now adds "Fold Change" annotations if you annotate with a Statistical Comparison track.
- It now has an option to "Keep only one copy of duplicate annotations".
- The Download BLAST Databases tool now requires less disk space when downloading and installing BLAST databases.
- The history information associated with results from the BLAST and BLAST at NCBI tools now includes the version of the BLAST software used for the search.
- The Reverse Sequencetool now names the output sequence name with the input sequence name followed by "-R" .
- For the Gene Set Test tool, the name of the columns "Occurrences in all genes", "Genes (universe)", "Occurrences in subset", "Genes (subset)" have been renamed to "Detected Genes", "Detected Genes (Names)', "DE Genes", "DE Genes (Names)".
- For the GO Enrichment Analysis, the name of the columns "Occurrences in all genes", "Genes (universe)", "Occurrences in sample", "Genes (overlap)" have been renamed to "Matched Genes", "Matched Genes (Names)", "Genes with Variations", "Genes with Variations (Names)".
- Fixed an issue affecting the Map Reads to Reference tool when it was included in a workflow, where if the References parameter was connected to an input, and a masking track was configured, an error was reported stating that the masking track was incompatible with the reference genome, whether or not it was compatible.
- Fixed an issue with the Low Frequency Variant Detection and Fixed Ploidy Variant Detection tools that caused a small minority of variants to go unreported under certain conditions expected to arise rarely.
- Fixed a bug in the Identify Candidate Variants tool, (now called Filter Variants on Custom Criteria), where no results were returned when one or more criteria used a comparison operator with more than one term (e.g. ">=", "abs value <").
- Fixed a bug in the Import Tracks tool where one nucleotide exons would be skipped during import of GTF files. A consequence of this fix means that we do not support the import of UCSC SNPs typed as exons any longer.
- Fixed a bug where the "Unaligned end" field provided in the Breakpoint track output of the Indel and Structural Variants tool was left blank when the value should have been "Mixed consensus" on all but one chromosome. The field is now filled for all chromosomes.
- Fixed bug that caused import of empty text files to stall.
- Fixed an issue found in the History of a result generated by the Extract Annotations tool, that would incorrectly show that a reference sequence track was used when it was not.
- Specifying a reference cache size greater than 2GB was not possible when using a readmapper.properties file.
- The mapping tool used in tools involving a mapping stage, such as Map Reads to References, Map Reads to Contigs and RNA-Seq Analysis has been updated:
- Fixed an issue that led to some deletions being reported as multiple, separate deletions instead of a single, larger deletion when affine gap costs were used.
- Fixed a very rare bug in the read mapper, where an alignment with a leading unaligned end could get a wrong score.
- On Windows 10 and Windows Server 2016, it now runs with 'below normal' as the priority. Previously, it ran with 'normal' priority.
- On Windows 10 and Windows Server 2016, the underlying program launched when running the Sample Reads tool now runs with 'below normal' as the priority. Previously, it ran with 'normal' priority.
- The underlying read mapper and de novo binaries included in the CLC Genomics Server 11.0 are from CLC Assembly Cell 5.1.1.
- The SOLiD Importer has been retired. It was previously in Legacy Tools. As a consequence:
- The tools Map Reads to Reference, Map Reads to Contigs, Trim Reads, De Novo Assembly, Extract and Count, and Annotate and Merge no longer have special handling of SOLiD colorspace data. They will continue to work as expected for SOLiD data, but will not make use of color information to correct for phase shifts.
- Import | SAM/BAM Mapping Files and Standard Import | Reads from SAM/BAM files no longer allow import of data where colorspace information is provided in the form of CS flags and sequence data is omitted (SEQ = "*") .
- Export | SAM, Export | BAM, and Export | Fastq no longer have special handling of SOLiD colorspace data. They will continue to work as expected for SOLiD data, but will not make use of color information to correct for phase shifts.
- The *.cas importer found in Import -> Standard Import no longer allows the import of read mappings where SOLiD color information has been used as part of the mapping algorithm.
- The Import Tracks tool no longer supports the import of files in Complete Genomics master VAR file format. To import such files, it is necessary to first convert them to VCF using the tools provided by Complete Genomics.
- The column "Ignored reads (wrong strand)" has been removed from the "Strand specificity" section of the report produced by the Create Combined RNA-Seq Report tool. The column has been removed to better fit the report's purpose of only providing high-level relevant information.
- Biomedical Genomics Analysis Server Plugin 1.0 Installing this plugin on a CLC Genomics Server provides the functionality formerly available by installing a Biomedical Genomics Server Extension license on a CLC Genomics Server and installing the now-retired QIAseq Targeted Panel Analysis Server Plugin.
- Bisulfite Sequencing Sever Plugin The tools delivered by this plugin have been integrated into the CLC Genomics Server and can be launched from the CLC Genomics Workbench or CLC Command Line Tools client software.
- QIAseq Targeted Panel Analysis Server Plugin and QIAGEN GeneRead Panel Analysis Plugin These plugins were formerly available for use on Biomedical-enabled CLC Genomics Servers. Their functionality is now available via the Biomedical Genomics Analysis Server Plugin when installed on a CLC Genomics Server.
- Support for paired-end reads in the Ion Torrent importer
- Import of SCARF and QSEQ format files in the Illumina importer
The following tools will be removed in a future release of the software:
- Roche 454 NGS import
- Compare Sample Variant Tracks
- Create Track from Experiment
- Identify Differentially Expressed Gene Groups and Pathways
- Add Fold Changes
- Add Information from Overlapping Genes
- Create Fold Change Track
- Download Reference Genome Data (The functionality via the Reference Data Manager is unaffected by this.)
If you are concerned about these proposed changes, please contact our Support team by emailing [email protected]
The following are the corresponding clients for the CLC Genomics Server 11.0
Please see the CLC Genomics Server 12.0 listing for the details about the new tools and features listed below.
- CLC Genomics Workbench 12.0
- CLC Main Workbench 8.1
- CLC Command Line Tools 6.0
- Bisulfite Sequencing Analysis Three tools for analyzing cytosine methylation data are now available and can be called using the following commands:
These tools were formerly available via the Bisulfite Sequencing Server Plugin, but are now integrated into the server software.
- cnv_detection launches the Copy Number Variation Detection (CNVs) tool for detecting copy number variations (CNVs) from targeted resequencing experiments. This tool was formerly only available when running commands on a Biomedical-enabled CLC Genomics Server.
- differential_expression_two_groups launches the Differential Expression in Two Groups tool, which can be used instead of the more general Differential Expression for RNA tool for testing differential expression between a single treatment group and a control group.
- remove_information_from_variants is a tool for removing annotations on variants.
- remove_orphan_reference_variants removes reference allele variants that lack a corresponding non-reference variant allele.
- primer_pair_import launches the Import Primer Pairs tool, which imports primer pair locations from a generic text format file or from a QIAGEN gene panel primer file. This tool was formerly only available when running commands on a Biomedical-enabled CLC Genomics Server.
- ngs_import_genereader is an import tool for QIAGEN GeneReader data.
- differential_expression_rna_seq - new options added
- amino_acid_changes - new options added
- download_sra - new option added
- annotate_overlapping - new option added
- extract_overlapping_reads - new option added
- export -e clc - new option added
- --maxcompat When set to true, data is exported without internal data compression, allowing the data to be imported into older CLC software. Set to false by default.
- export -e zip - new option added
- --maxcompat When set to true, data is exported without internal data compression, allowing the data to be imported into older CLC software. Set to false by default.
- ls - new option added
- -e When running -A ls -e with a data element specified using the -t option, the export formats supported for that data element are listed.
- The ngs_import_iontorrent and ngs_import_pacbio tools now support the import of reads from SAM or BAM format files. Mapping information from such imports are discarded. To import a read mapping from SAM or BAM format files, use the ngs_import_sam tool.
- The clcserver command can now be run with just the -V option, which will return the version of the CLC Command Line Tools being used. By running the clcserver command with both the -V and -S options, the version of the CLC Genomics Server indicated will also be returned.
- The human readable URLs (ClcUrl Simple) returned when using the "-A ls" command are now of the form: <host:port form here - see comments> , that is, with the host and port of the CLC Genomics Server specified explicitly. Earlier, the generic URL form <generic form here - see comments> was used.
- Various minor improvements
- Fixed an issue where the commands "list_workflows" and "uninstall_plugin_and_restart" would not work for installed workflows that had an associated icon.
- SOLiD colorspace data is no longer supported. Please see the CLC Genomics Server 11.0 listings for details. For the CLC Server Command Line Tools, this has resulted in the following changes:
Command removed: ngs_import_solid
contig_read_mapping Removed options: --color-error-cost, --color-space
denovo_assembly Removed options: --long-reads-color-error-cost, --long-reads-color-space
read_mapping Removed options: --color-error-cost, --color-space
rna_seq Removed options: --color-error-cost, --color-space
small_rna_annotate Removed options: --color-space
small_rna_sampling Removed options: --color-space
trim Removed options: --color-space
Some tools previously only available for use with Biomedical-enabled CLC Genomics Servers are now available for any CLC Genomics Server 12.0, but are legacy tools and will be retired in a future release of the software. These tools are placed within the Legacy folder of CLC Genomics Workbench 12.0. The relevant CLC Server Command Line Tools commands are: