better resolution for dense file with xyplot

[alessandro Guida]

Hi,

I am using Gbrowse 1.69 to display the results of RNA-seq and Chip-seq analysis of a new genome. I would like to use the xyplot glyph to display a dense wig file (~500 Mb) with the count of reads mapped in each bp of the genome. 

I have used the script wiggle2gff3.pl however, because of the compression to a 8bit binary file, I have lost a significant amount of information. To be more specific, the numbers of read per bp ranges from 0 to 25.000. The script mentioned before, represent my range with only 256 number (8bits). The final results is a really discontinuous (i.e. Blocky)  xyplot . 

Is there any way to reduce these compression or any alternative way to display dense file in GBrowse ?

I am using mySQL  Bio::DB::SeqFeature::Store.

Thanks in advance,

Alessandro

 

-------------------------

Coverage.wig example)

track type=bedGraph name="TopHat - read coverage"

chr1   0       631     0

chr1   631     673     2

chr1    673     1033    0

chr1    1033    1075    1

chr1    1075    1372    0

chr1    1372    1390    1

chr1    1390    1393    2

chr1    1393    1395    3

chr1    1395    1402    4

chr1    1402    1412    8

chr1    1412    1414    11

chr1    1414    1416    10

chr1    1416    1417    12

chr1    1417    1427    14

chr1    1427    1432    16

chr1    1432    1433    15

chr1    1433    1434    19

chr1    1434    1435    20

chr1    1435    1437    21

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now.  http://p.sf.net/sfu/bobj-july
_______________________________________________
Gmod-gbrowse mailing list
Gmod-gbrowse@...
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse

[Vivek Krishnakumar]

Hi Alessandro,

If I’m not mistaken, since this read coverage corresponds to the subset of reads mapped by TopHat, you are bound to get a so-called “blocky” xyplot because of the spliced mapping done by TopHat. If the coverage data were to come from Bowtie, then the picture would be completely different and would see more of a continuous smear with varying intensities.

As for the wiggle2gff3.pl script, it definitely does compress the numbers to a scale of 1-255, but my feeling is that this scale is quite good for visualizing this kind of data.
Have I made sense here or am I missing something? Maybe the experts will have very good answer to this!!

Thanks.
~Vivek


On 11/5/09 9:23 AM, "alessandro Guida" <alessandro.guida@...> wrote:

Hi,

I am using Gbrowse 1.69 to display the results of RNA-seq and Chip-seq analysis of a new genome. I would like to use the xyplot glyph to display a dense wig file (~500 Mb) with the count of reads mapped in each bp of the genome.

I have used the script wiggle2gff3.pl <http://wiggle2gff3.pl>  however, because of the compression to a 8bit binary file, I have lost a significant amount of information. To be more specific, the numbers of read per bp ranges from 0 to 25.000. The script mentioned before, represent my range with only 256 number (8bits). The final results is a really discontinuous (i.e. Blocky)  xyplot .

Is there any way to reduce these compression or any alternative way to display dense file in GBrowse ?
I am using mySQL  Bio::DB::SeqFeature::Store.

Thanks in advance,

Alessandro
 
-------------------------
Coverage.wig example)

track type=bedGraph name="TopHat - read coverage"
chr1   0       631     0
chr1   631     673     2
chr1    673     1033    0
chr1    1033    1075    1
chr1    1075    1372    0
chr1    1372    1390    1
chr1    1390    1393    2
chr1    1393    1395    3
chr1    1395    1402    4
chr1    1402    1412    8
chr1    1412    1414    11
chr1    1414    1416    10
chr1    1416    1417    12
chr1    1417    1427    14
chr1    1427    1432    16
chr1    1432    1433    15
chr1    1433    1434    19
chr1    1434    1435    20
chr1    1435    1437    21

---

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now.  http://p.sf.net/sfu/bobj-july

---

_______________________________________________
Gmod-gbrowse mailing list
Gmod-gbrowse@...
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now.  http://p.sf.net/sfu/bobj-july
_______________________________________________
Gmod-gbrowse mailing list
Gmod-gbrowse@...
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse

[alessandro Guida]

Hi Vivek,

Thanks for your quick replay. I could very well be mistaken, but I think the blocky look of the xyplot could not due to Tophat.

Before converting GBrowse to mySQL I played around a little bit with it using the "memory" adaptor. Although the browser was really slow compared to the mySQL version, I managed to visualize xyplots loading the dense file (of one single chromosome at the time) as a GFF3. These xyplots looked nice and smoother than the one generated with wiggle2gff3.pl.

Here a screenshot that compares the plot generated with wiggle2gff3.pl (bottom) and the histogram generated in the way I described before.

http://screencast.com/t/AG8z3ZrILLN

However, this process turned to be computationally inefficient and the browser required long waiting time in order to generate a new picture. I therefore decided to try mySQL. Unfortunately when I load the dense file as a gff3, I can no longer display any xyplots. After a little bit of searching and reading I found out about wiggle2gff3.pl and I encountered the problem we are discussing.

I am definitely going to try bowtie and see if there are any improvements but if you have any comments, I really appreciate your input.  

Alessandro

On 5 Nov 2009, at 16:39, Krishnakumar, Vivek wrote:

Hi Alessandro,

If I’m not mistaken, since this read coverage corresponds to the subset of reads mapped by TopHat, you are bound to get a so-called “blocky” xyplot because of the spliced mapping done by TopHat. If the coverage data were to come from Bowtie, then the picture would be completely different and would see more of a continuous smear with varying intensities.

As for the wiggle2gff3.pl script, it definitely does compress the numbers to a scale of 1-255, but my feeling is that this scale is quite good for visualizing this kind of data.
Have I made sense here or am I missing something? Maybe the experts will have very good answer to this!!

Thanks.
~Vivek


On 11/5/09 9:23 AM, "alessandro Guida" <alessandro.guida@...> wrote:

Hi,

I am using Gbrowse 1.69 to display the results of RNA-seq and Chip-seq analysis of a new genome. I would like to use the xyplot glyph to display a dense wig file (~500 Mb) with the count of reads mapped in each bp of the genome.

I have used the script wiggle2gff3.pl <http://wiggle2gff3.pl>  however, because of the compression to a 8bit binary file, I have lost a significant amount of information. To be more specific, the numbers of read per bp ranges from 0 to 25.000. The script mentioned before, represent my range with only 256 number (8bits). The final results is a really discontinuous (i.e. Blocky)  xyplot .

Is there any way to reduce these compression or any alternative way to display dense file in GBrowse ?
I am using mySQL  Bio::DB::SeqFeature::Store.

Thanks in advance,

Alessandro
 
-------------------------
Coverage.wig example)

track type=bedGraph name="TopHat - read coverage"
chr1   0       631     0
chr1   631     673     2
chr1    673     1033    0
chr1    1033    1075    1
chr1    1075    1372    0
chr1    1372    1390    1
chr1    1390    1393    2
chr1    1393    1395    3
chr1    1395    1402    4
chr1    1402    1412    8
chr1    1412    1414    11
chr1    1414    1416    10
chr1    1416    1417    12
chr1    1417    1427    14
chr1    1427    1432    16
chr1    1432    1433    15
chr1    1433    1434    19
chr1    1434    1435    20
chr1    1435    1437    21

---

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now.  http://p.sf.net/sfu/bobj-july

---

_______________________________________________
Gmod-gbrowse mailing list
Gmod-gbrowse@...
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now.  http://p.sf.net/sfu/bobj-july
_______________________________________________
Gmod-gbrowse mailing list
Gmod-gbrowse@...
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse

[Vivek Krishnakumar]

Hi Alessandro,

When you mentioned that you want to display TopHat read coverage where the data is in wiggle format, I assumed that you wanted to display a “wiggle density” plot. Might I suggest that you use the following configuration for your track and see if that suits your requirement?

[TopHat_Read_Coverage]
feature      = <insert_your_feature_name>
category     = <choose_a_category>
glyph        = wiggle_density
height       = 20
bgcolor      = red
description  = 1

The “feature” here will be the feature that you chose when you used wiggle2gff3.pl to produce the .wig files and the .gff3 file which links GBrowse to the .wig files.



This should produce a track as shown above in the image. (please see the 2nd track)

Since you have used TopHat to produce the coverage data (in wiggle format), I’m assuming that you would have run Bowtie first to perform the unspliced mapping of the reads to a reference genome followed by the TopHat spliced mapping component.

HTH.
Vivek

On 11/5/09 2:07 PM, "alessandro Guida" <alessandro.guida@...> wrote:

Hi Vivek,

Thanks for your quick replay. I could very well be mistaken, but I think the blocky look of the xyplot could not due to Tophat.

Before converting GBrowse to mySQL I played around a little bit with it using the "memory" adaptor. Although the browser was really slow compared to the mySQL version, I managed to visualize xyplots loading the dense file (of one single chromosome at the time) as a GFF3. These xyplots looked nice and smoother than the one generated with wiggle2gff3.pl <http://wiggle2gff3.pl/> .

Here a screenshot that compares the plot generated with wiggle2gff3.pl <http://wiggle2gff3.pl/>  (bottom) and the histogram generated in the way I described before.
http://screencast.com/t/AG8z3ZrILLN

However, this process turned to be computationally inefficient and the browser required long waiting time in order to generate a new picture. I therefore decided to try mySQL. Unfortunately when I load the dense file as a gff3, I can no longer display any xyplots. After a little bit of searching and reading I found out about wiggle2gff3.pl <http://wiggle2gff3.pl/>  and I encountered the problem we are discussing.

I am definitely going to try bowtie and see if there are any improvements but if you have any comments, I really appreciate your input.  

Alessandro


On 5 Nov 2009, at 16:39, Krishnakumar, Vivek wrote:

Hi Alessandro,

If I’m not mistaken, since this read coverage corresponds to the subset of reads mapped by TopHat, you are bound to get a so-called “blocky” xyplot because of the spliced mapping done by TopHat. If the coverage data were to come from Bowtie, then the picture would be completely different and would see more of a continuous smear with varying intensities.

As for the wiggle2gff3.pl <http://wiggle2gff3.pl>  script, it definitely does compress the numbers to a scale of 1-255, but my feeling is that this scale is quite good for visualizing this kind of data.
Have I made sense here or am I missing something? Maybe the experts will have very good answer to this!!

Thanks.
~Vivek


On 11/5/09 9:23 AM, "alessandro Guida" <alessandro.guida@... <x-msg://190/alessandro.guida@...> > wrote:

Hi,

I am using Gbrowse 1.69 to display the results of RNA-seq and Chip-seq analysis of a new genome. I would like to use the xyplot glyph to display a dense wig file (~500 Mb) with the count of reads mapped in each bp of the genome.

I have used the script wiggle2gff3.pl <http://wiggle2gff3.pl>  <http://wiggle2gff3.pl <http://wiggle2gff3.pl/> >  however, because of the compression to a 8bit binary file, I have lost a significant amount of information. To be more specific, the numbers of read per bp ranges from 0 to 25.000. The script mentioned before, represent my range with only 256 number (8bits). The final results is a really discontinuous (i.e. Blocky)  xyplot .

Is there any way to reduce these compression or any alternative way to display dense file in GBrowse ?
I am using mySQL  Bio::DB::SeqFeature::Store.

Thanks in advance,

Alessandro
 
-------------------------
Coverage.wig example)

track type=bedGraph name="TopHat - read coverage"
chr1   0       631     0
chr1   631     673     2
chr1    673     1033    0
chr1    1033    1075    1
chr1    1075    1372    0
chr1    1372    1390    1
chr1    1390    1393    2
chr1    1393    1395    3
chr1    1395    1402    4
chr1    1402    1412    8
chr1    1412    1414    11
chr1    1414    1416    10
chr1    1416    1417    12
chr1    1417    1427    14
chr1    1427    1432    16
chr1    1432    1433    15
chr1    1433    1434    19
chr1    1434    1435    20
chr1    1435    1437    21

---

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now.  http://p.sf.net/sfu/bobj-july

---

_______________________________________________
Gmod-gbrowse mailing list
Gmod-gbrowse@... <x-msg://190/Gmod-gbrowse@...>
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse

---

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now.  http://p.sf.net/sfu/bobj-july

---

_______________________________________________
Gmod-gbrowse mailing list
Gmod-gbrowse@...
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now.  http://p.sf.net/sfu/bobj-july
_______________________________________________
Gmod-gbrowse mailing list
Gmod-gbrowse@...
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse

[Jan-Ole Christian]

I just load these kind of data as gff3 files via bp_seqfeature_load.pl into the MySQL Database. It could be waste of disk-space, but It is really fast enough to display in the browser. My gff3-files consists of millions of lines.
You can use the "score" field of the gff-line to encode the number of reads.
Regards,
Ole.



Am 05.11.2009 15:23, schrieb alessandro Guida:

Hi,

I am using Gbrowse 1.69 to display the results of RNA-seq and Chip-seq analysis of a new genome. I would like to use the xyplot glyph to display a dense wig file (~500 Mb) with the count of reads mapped in each bp of the genome. 

I have used the script wiggle2gff3.pl however, because of the compression to a 8bit binary file, I have lost a significant amount of information. To be more specific, the numbers of read per bp ranges from 0 to 25.000. The script mentioned before, represent my range with only 256 number (8bits). The final results is a really discontinuous (i.e. Blocky)  xyplot . 

Is there any way to reduce these compression or any alternative way to display dense file in GBrowse ?

I am using mySQL  Bio::DB::SeqFeature::Store.

Thanks in advance,

Alessandro

 

-------------------------

Coverage.wig example)

track type=bedGraph name="TopHat - read coverage"

chr1   0       631     0

chr1   631     673     2

chr1    673     1033    0

chr1    1033    1075    1

chr1    1075    1372    0

chr1    1372    1390    1

chr1    1390    1393    2

chr1    1393    1395    3

chr1    1395    1402    4

chr1    1402    1412    8

chr1    1412    1414    11

chr1    1414    1416    10

chr1    1416    1417    12

chr1    1417    1427    14

chr1    1427    1432    16

chr1    1432    1433    15

chr1    1433    1434    19

chr1    1434    1435    20

chr1    1435    1437    21

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day 
trial. Simplify your report design, integration and deployment - and focus on 
what you do best, core application coding. Discover what's new with
Crystal Reports now.  http://p.sf.net/sfu/bobj-july

_______________________________________________
Gmod-gbrowse mailing list
Gmod-gbrowse@...
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse
  

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now.  http://p.sf.net/sfu/bobj-july
_______________________________________________
Gmod-gbrowse mailing list
Gmod-gbrowse@...
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse

[Wes Barris]

Jan-Ole Christian wrote:
> I just load these kind of data as gff3 files via bp_seqfeature_load.pl
> into the MySQL Database. It could be waste of disk-space, but It is
> really fast enough to display in the browser. My gff3-files consists of
> millions of lines.
> You can use the "score" field of the gff-line to encode the number of reads.

This is exactly what I do too.  I have found that the same data can
be displayed faster and using less memory than if done using the wiggle
method.

> Regards,
> Ole.
>
>
>
> Am 05.11.2009 15:23, schrieb alessandro Guida:
>> Hi,
>>
>> I am using Gbrowse 1.69 to display the results of RNA-seq and Chip-seq
>> analysis of a new genome. I would like to use the *xyplot glyph* to
>> display a dense *wig* file (~500 Mb) with the count of reads mapped in
>> each bp of the genome.
>>
>> I have used the script *wiggle2gff3.pl
>> <http://wiggle2gff3.pl> *however, because of the compression to a 8bit
>> binary file, I have lost a significant amount of information. To be
>> more specific, the numbers of read per bp ranges from 0 to 25.000. The
>> script mentioned before, represent my range with only 256 number
>> (8bits). The final results is a really discontinuous (/i.e. /Blocky)  
>> xyplot .
>>
>> Is there any way to reduce these compression or any alternative way to
>> display dense file in GBrowse ?
>> I am using mySQL  Bio::DB::SeqFeature::Store.
>>
>> Thanks in advance,
>>
>> Alessandro
>>  
>> -------------------------
>> Coverage.wig example)
>>
>> track type=bedGraph name="TopHat - read coverage"
>> chr1   0       631     0
>> chr1   631     673     2
>> chr1    673     1033    0
>> chr1    1033    1075    1
>> chr1    1075    1372    0
>> chr1    1372    1390    1
>> chr1    1390    1393    2
>> chr1    1393    1395    3
>> chr1    1395    1402    4
>> chr1    1402    1412    8
>> chr1    1412    1414    11
>> chr1    1414    1416    10
>> chr1    1416    1417    12
>> chr1    1417    1427    14
>> chr1    1427    1432    16
>> chr1    1432    1433    15
>> chr1    1433    1434    19
>> chr1    1434    1435    20
>> chr1    1435    1437    21
>>
>>
>> ------------------------------------------------------------------------------
>> Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
>> trial. Simplify your report design, integration and deployment - and focus on
>> what you do best, core application coding. Discover what's new with
>> Crystal Reports now.  http://p.sf.net/sfu/bobj-july
>>
>>
>> _______________________________________________
>> Gmod-gbrowse mailing list
>> Gmod-gbrowse@...
>> https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse
>>  
>

--
Wes Barris <wes.barris@...>

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now.  http://p.sf.net/sfu/bobj-july
_______________________________________________
Gmod-gbrowse mailing list
Gmod-gbrowse@...
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse