Author Archives: paalh


About 2.8 millions of new luminal GI cancers (esophagus, stomach, colorectal) are detected yearly in the world, and the mortality is about 65%. In addition to these cancers, numerous other chronic diseases affect the human GI tract. The most common ones include gastroesophageal reflux disease, peptic ulcer disease, inflammatory bowel disease, celiac disease and chronic infections. All have a significant impact on the patients’ health-related quality of life. Consequently, gastroenterology is one of the most significant medical branches.

If we for example look at colorectal cancer (CRC) with one of the highest incidences and mortality of the diseases in the GI tract, early detection is essential for prognosis. Mini-invasive endoscopic and surgical treatment is most often curative in early stages (I- II) with a 5-year survival probability of about 90%, but in advanced stages (III-IV), radiation and/or chemotherapy is often required, and it has a 5-year survival of only 10-30%. In this case, colonoscopy is considered to be the gold standard for the examination of the colon for early detection of cancer and precancerous pathology. However, it is not the ideal screening test. On average 20% of polyps are missed or incompletely removed (i.e., the risk of getting CRC largely depend on the endoscopists ability to detect polyps). It is also a demanding procedure requiring a significant time investment from the medical professional, and the procedure is unpleasant and can cause great discomfort for the patient. This may lead to reduced participation rates and less efficient screening. As a result, ongoing colonoscopy screening programs have a low attendance rate.

Our overall idea is to develop a system for automatic analysis of (video) data from the entire GI tract. As a first approach and to show how complex the target is we developed a multimedia system that supports doctors in disease detection in the GI tract. The main requirements of such a system are (i) easy to use, (ii) easy to extend to different diseases, (iii) real time handling of multimedia content, (iv) being able to be used as a live system and (v) high classification performance with minimal false negative classification results. Therefore, the system consists of three main parts: The annotation sub-system, the detection and automatic analysis sub-system, and the visualization and computer aided diagnosis sub-system.

Using the ASU Mayo polyp dataset, our prototype EIR, based on content-based visual information retrieval, achieved a detection accuracy of above 90% at a speed of about 300 frames per second, i.e., real-time feedback is enabled.


Selected publications:

“Multimedia and Medicine: Teammates for Better Disease Detection and Survival”, Michael Riegler, Mathias Lux, Carsten Griwodz, Concetto Spampinato, Thomas de Lange, Sigrun L. Eskeland, Konstantin Pogorelov, Wallapak Tavanapong, Peter T. Schmidt, Cathal Gurrin, Dag Johansen, Håvard Johansen, Pål Halvorsen, Proceedings of ACM Multimedia (ACM MM), Amsterdam, The Netherlands, October 2016, pp. 968-977 [pdf] [DOI: 10.1145/2964284.2976760] [slides]

“GPU-accelerated Real-time Gastrointestinal Diseases Detection”, Konstantin Pogorelov, Michael Riegler, Pål Halvorsen, Peter Thelin Schmidt, Carsten Griwodz, Dag Johansen, Sigrun Losada Eskeland, Thomas de Lange, Proceedings of the International Symposium on Computer-Based Medical Systems (CBMS), Dublin, Ireland/Belfast, Northern Ireland, June 2016 [pdf]

“EIR – Efficient Computer Aided Diagnosis Framework for Gastrointestinal Endoscopies”, Michael Riegler, Konstantin Pogorelov, Pål Halvorsen, Thomas de Lange, Carsten Griwodz, Peter Thelin Schmidt, Sigrun Losada Eskeland, Dag Johansen, Proceedings of the International Workshop on Content-based Multimedia Indexing (CBMI), Bucharest, Romania, June 2016 [pdf]

“Explorative Hyperbolic-Tree-Based Clustering Tool for Unsupervised Knowledge Discovery”, Michael Riegler, Konstantin Pogorelov, Mathias Lux, Pål Halvorsen, Carsten Griwodz, Sigrun Losada Eskeland, Thomas de Lange, Proceedings of the International Workshop on Content-based Multimedia Indexing (CBMI), Bucharest, Romania, June 2016 [pdf]

“Computer Aided Disease Detection System for Gastrointestinal Examinations”, Michael Riegler, Konstantin Pogorelov, Jonas Markussen, Mathias Lux, Håkon Kvale Stensland, Thomas de Lange, Carsten Griwodz, Pål Halvorsen, Dag Johansen, Peter Thelin Schmidt, Sigrun L. Eskeland, Proceedings of the ACM Multimedia Systems Conference (MMSys), Klagenfurt am Wörthersee, Austria, May 2016 [DOI:10.1145/2910017.2910629]

“Efficient Processing of Videos in a Multi Auditory Environment Using Device Lending of GPUs”, Konstantin Pogorelov, Michael Riegler, Jonas Markussen, Hålkon Kvale Stensland, Pål Halvorsen, Carsten Griwodz, Sigrun Losada Eskeland, Thomas de Lange, Proceedings of the ACM Multimedia Systems Conference (MMSys),
Klagenfurt am Wörthersee, Austria, May 2016 [DOI: 10.1145/2910017.2910636]

“Expert Driven Semi-Supervised Elucidation Tool for Medical Endoscopic Videos” (demo), Zeno Albisser, Michael Riegler, Pål Halvorsen, Jiang Zhou, Carsten Griwodz, Ilangko Balasingham, Cathal Gurrin, Proceedings of the ACM Multimedia Systems Conference (MMSys), Portland, OR, USA, March 2015, pp. 73-76, [pdf] [DOI: 10.1145/2713168.2713184]

“Event Understanding in Endoscopic Surgery Videos”, Mario Guggenberger, Michael Riegler, Mathias Lux, Pål Halvorsen, Proceedings of the ACM International Workshop on Human Centered Event Understanding from Multimedia (HuEvent), Orlando, FL, USA, November 2014 pp. 17-22 [DOI: 10.1145/2660505.2660509]

Vamsidhar Reddy Gaddam defended his PhD



On Thursday 2 June, Vamsidhar Reddy Gaddam defended his PhD thesis “Next Generation Broadcasting System for Arena Sports: A Football Stadium Scenario”. Vamsi has worked on the Bagadus system which is used at the Alfheim and Ullevaal stadiums.

Read more info from the university and from Simula. The PhD thesis can be found here.

Vamsi is currently an engineer in the Media Processing Group at ARM focusing on mobile graphics.



Vamsi-NRKBagadus has again been shown on national tv. Vamsi demonstrated the system for the Schrödingers katt from NRK during the national game against Eastland last November. See the news clip here.


NB! A few minutes earlier in the program, another clip was shown about the use of ZXY. ZXY sports tracking is also related to Bagadus, and used for example to track players (and control the virtual cameras) .

Håkon Kvale Stensland successfully defended his PhD

Stensland-Håkon-KvaleOn 27th February 2015, Håkon Kvale Stensland defended his PhD thesis“Processing Multimedia Workloads on Heterogeneous Multicore Architectures“. The defense took place Simula Research Laboratory, Fornebu.



Processor architectures have been evolving quickly since the introduction of the central processing unit. For a very long time, one of the important means of increasing performance was to increase the clock frequency. However, in the last decade, processor manufacturers have hit the so-called power wall, with high heat dissipation. To overcome this problem, processors were designed with reduced clock frequencies but with multiple cores and, later, heterogeneous processing elements. This shift introduced a new challenge for programmers: Legacy applications, written without parallelization in mind, gain no benefits from moving to multicore and heterogeneous architectures. Another challenge for the programmers is that heterogeneous architecture designs are very different with respect to caches, memory types, execution unit organization, and so forth and, in the worst case, a programmer must completely rewrite the application to obtain the best performance on the new architecture.

Multimedia workloads, such as video encoding, are often time sensitive and interactive. These workloads differ from traditional batch processing workloads with no real-time requirements. This work investigates how to use modern heterogeneous architectures efficiently to process multimedia workloads. To do so, we investigate both simple and complex workloads on multiple architectures to learn about the properties of these architectures. When programing multimedia workloads, it is very important to know how the algorithms perform on the target architecture. In addition, achieving high performance on heterogeneous architectures is not a trivial task, often requiring detailed knowledge about the architecture. We therefore evaluate several optimizations so we can learn how best to write programs for these architectures and avoid potential pitfalls. We later use the knowledge gained to propose a framework design and language called Parallel Processing Graph (P2G). The P2G framework is designed for multimedia workloads and supports heterogeneous architectures. To demonstrate the feasibility of the framework, we construct a proof-of-concept implementation. Two simple workloads show that we can express multimedia workloads in the system. We also demonstrate the scalability of the designed solution.

2 open PhD positions

In the EONS (Efficient Execution of Large Workloads on Elastic Heterogeneous Resources) project at Simula, we have 2 available PhD positions. The aim of the research project is to perform basic research in the area of development of parallel programming and parallel processing in the context of future distributed large-scale heterogeneous systems. We aim to research and develop concepts and mechanisms that will enable the development of software for these next-generation big-data applications, especially in the context of real-time (multimedia) systems. This is achieved by solving fundamental challenges for the dispatching, division, scheduling and identification of tasks that can run correctly in parallel in a shared distributed system of heterogeneous computing resources in complex topologies.

The EONS project will research “system support” for future large scale interactive distributed applications with large requirements for both I/O and processing power. This means that we aim for a more efficient development and execution system in order to ease distributed parallel application development and improve resource utilization in distributed heterogeneous environments, respectively. Thus, the following are important areas to pursue:

  • Formalization of a high level parallel programming model and programming language. There exists several approaches to specify potential parallelism, but for workloads with processing and/or time dependencies, we need to add notions of deadlines and execution orders.
  • Compiler and multi-core run-time system. Many run-time systems have been built and are in use, but there are large potentials for more efficient execution and run-time support for the dependencies must be added. Scheduling and mapping of tasks to processing engines will here be important.
  • Distributed implementation and high-level scheduler optimization. Adding support for multiple machines makes the previous item more complex. The heterogeneity and complexity increase and the communication costs vary more. A high-level scheduler therefore must take this into account, in addition to the competition for resources from different concurrent workloads.

All project participants have experience in the area of experimental research, and the EONS project will therefore use evolutionary prototyping to accomplish its goal of investigating and implementing a mechanisms that enable the development and execution of complex, time-dependent and computationally intensive applications. Using simulations to prove the validity of our high and low-level schedulers, scalability, etc., would be possible, but past experience has shown that we are frequently unable to model the complexity of the system correctly. Furthermore, the best venues for publishing research results in this area today require test results from real systems. As such, developing a proof-of-concept prototype running real (multimedia) applications is the only feasible approach for solving issues related to this project.

See the full project proposal text here.

Contact Pål Halvorsen ( for more information.



Best Presentation Award @ NOSSDAV 2014

OLYMPUS DIGITAL CAMERAIn an earlier blogpost, we informed that the paper “Interactive Zoom and Panning from Live Panoramic Video” by Vamsidhar Reddy Gaddam, Ragnar Langseth, Sigurd Ljødal, Pierre Gurdjos, Vincent Charvillat, Carsten Griwodz and Pål Halvorsen was accepted at NOSSDAV 2013.

Even though his PC broke an hour before the session and the powerpoint slides had to be re-made, Vamsi gave a perfect presentation, and at the conference banquet, he received the BEST PRESENTATION AWARD.

Håvard Espeland successfully defended his PhD

SONY DSCOn 25th February 2014, Håvard Espeland defended his PhD thesis Processing Cyclic Multimedia Workloads on Modern Architectures“. The defense took place Simula Research Laboratory, Fornebu.




Working with modern architectures for high performance applications is increasingly more diffi- cult for programmers as the complexity of both the system architectures and software continue to increase. The level of hand tuning and native adaptations required to achieve high performance comes at the cost of limiting the portability of the software. For instance, we show that a compute intensive DCT algorithm performs better on graphic processors than the best algorithm for x86. In particular, limited portability is true for cyclic multimedia workloads, a set of programs that run continuously with strict requirements for high performance and low latency. An example of a typical multimedia workload is a pipeline of many small image processing algorithms working in tandem to complete a particular task. The input can be videos from one or more live cam- eras, and the output is a set of video frames with elements from several of the source videos, for example as stitched panorama frames or 3D warped video. Such a setup runs continuously and potentially needs to adapt to various degrees of changes in the setup without interruptions or downtime.

To reach the performance goal required by multimedia pipelines, modern, heterogeneous architectures are considered instead of the traditional symmetric multi-processing architectures. We also investigate variations between recent microarchitectures of symmetric processors to iden- tify differences that a low-level scheduler must take into account. Further, since multimedia workloads often need to adapt to various external conditions, e.g., adding another participant to a video conference, we also investigate elastic and portable processing of multimedia work- loads. To do this, we propose a framework design and language, which we call P2G. In the age of Big Data, this idea differs from the typical frameworks used for distributed processing, such as MapReduce and Dryad, in that it is designed for continuous operation instead of batch process- ing of large workloads. We emphasize heterogeneous support and expose parallel opportunities in workloads in a way that is easy to target since it is similar to sequential execution with mul- tidimensional arrays. The framework ideas are implemented as a prototype and released as an open source platform for further experimentation and evaluation

« Older Entries