APEX

Calculating a histogram of microarray measurement data (Part 2)

A lot of time has passed since I posted the first part of this blog series, which is mostly due to customer projects. Lately, I have also been busy implementing our NGS cancer analysis pipeline and software for representing the results. The product is really exciting, and we will post about it in the near future.

In the first post of this series I implemented an efficient query that calculates a histogram of microarray measurement data. In this post I will show my favourite method of implementing visualization in web environment. In the early days of web development we used to produce lots of code writing HTML tags to a stream to the browser. Today, several frameworks exist to make the life of the developer a lot easier. My favourite tool to build a data-centric web application is Oracle APEX. It is a declarative rapid web application development tool and it executes completely inside the database. Instead of writing the explicit steps how to create a user interface, APEX provides a point-and-click application builder (which is also implemented using APEX) where the developer creates pages using built-in components and plugins. APEX provides configurable components (reports, charts, buttons, input fields, etc) and wizards to help creating pages or regions based on existing database objects, such as tables.

A declarative model is great for productivity in cases where the requirements can be implemented using the provided functionality. The downside is the decreased productivity when something a bit more exotic is needed. If the provided component does not support extending the functionality, then the functionality has to be implemented by printing HTML/javascript/css fragments.

Calculating a histogram of microarray measurement data (Part 1)

I have been implementing a microarray data management application using Oracle APEX, which is a database centric rapid application builder. One task was to calculate a histogram of measurement data and show it in a chart. The histogram shows the distribution of values and can reveal problems concerning the data. So, the user selects one or more samples of interest and the application shows the histogram chart. The problem with this is performance. One microarray sample contains approximately 20000 measurements of gene expression levels and the user may select several samples. The total amount of rows hiling the measurements can be millions.

The first step is to create the query that calculates the data for the graph. Oracle has an analytic function called dense_rank, which computes the rank of a row in an ordered group of rows. In this case, the order comes from rounded measurement value. The rank is used as a bin, the rows are grouped by the bin and row count calculated for each bin. The result is the histogram.To limit the number of rows returned, the query combines  buckets 150 and above together. APEX will truncate the result if there are more rows than the chart is configured for.

The query is executed against all measurements (about 12 million) to get the upper limit for the execution time:

Syndicate content
  • Euformatics Oy, Keilaranta 4, 02150 Espoo, Finland
© Euformatics Oy 2012