Category Archives: UiO

Reconstruction of Indoor Environments Using LiDAR and IMU

Today there is a trend towards reconstruction of 3D scenes with movement over time, in both image-based and point cloud based reconstruction systems. The main challenge in point cloud-based systems is the lack of data. Most of the existing data sets are made from 3D-reconstructed meshes, but the density of these constructions is unrealistic.

Point cloud from a fixed LIDAR scan (left) to a LIDAR sweep (right)

In order to do proper research into this field, it must be possible to generate real data sets of high-density point clouds. To deal with this challenge, we have been supplied with a VLP-16 laser scanner and a Tinkerforge IMU Brick 2.0. In our final setup, we position the IMU at the top center of the VLP-16 by utilizing a 3D printed mounting plate. This assembly is fastened to a tripod, in order to move the assembly about well-defined axes. Because most laser scanners acquire points sequentially, these devices do not have the same concept of frame as for images where all data are captured in the same instant. To deal with this issue we divide one scan, i.e., a 360◦ LiDAR sweep, into data packets and transform these data packets using the associated pose to global space. We compensate for mismatch in sampling frequency between the VLP-16 and the IMU by linear interpolation between the acquired orientations. We generate subsets of the environment by changing the laser scanner orientation in static positions and estimate the translation between static positions using point- to-plane ICP. The registration of these subsets is also done using point- to-plane ICP. We conclude that at subset level, our reconstruction system can reconstruct high-density point clouds of indoor environments with a precision that is mostly limited to the inherent uncertainties of the VLP- 16. We also conclude that the registration of several subsets obtained from different positions is able to preserve both visual appearance and reflective intensity of objects in the scene. Our reconstruction system can thus be utilized to generate real data sets of high-density point clouds.

Reducing Packet Loss in Real-Time Wireless Multicast Video Streams with Forward Error Correction

Download PDF.

Average packet loss without and with FEC

Wireless multicast suffers from severe packet loss due to interference and
lack of link layer retransmission. In this work, we investigate wether the
most recent Forward Error Correction (FEC) draft is suitable for realtime wireless multicast live streaming, with emphasis on three main points:
packet reduction effectivity, and latency and overhead impact. We design
and perform an experiment in which we simulate wireless packet loss in
multicast streams with a Gilbert model pattern of ≈ 16% random packet
loss. We check all FEC configurations (L and D values) within several
constraints: maximum 500 milliseconds repair window (latency impact),
66.67% overhead, and a maximum L value of 20. For all these L and D
values we stream the tractor sample three times, to avoid possible outliers
in the data. We show that packet loss reduction in the most recent FEC
draft is effective, at most reducing from ≈ 16% down to ≈ 1.02%. We
also show that low latency streaming can be conducted, but it requires a
minimum of 160 milliseconds additional latency for our stream file. The
overhead for such low latency can be as high as 66.67%.

Implementation of a virtual reality design review application using vision-based gesture recognition technology

Download PDF.

Choosing annotation visibility in the virtual design review application.

Classification societies date back to the second half of the 18th century, where marine insurers developed a system for independent technical assessment of the ships presented to them for insurance cover. Today, a major part of a classification society’s responsibilities is to review the designs of enormous maritime vessels. This usually involves working with big and complex 3D models and 3D tools, but without support to do many of the tasks required in a design review. As a consequence, the workflow is often just partially digital, and many important tasks, such as annotating or commentating on aspects of the models, are done on paper. DNV GL, the world’s largest maritime classification society, is interested in digitalizing this process more, and make it more interactive and efficient by utilizing an application that allows for virtual design review meetings in the 3D models. In these virtual design review meetings, the designer and reviewer could remotely interact, survey the model together, and annotate it instead of model-printouts. As the sense of scale is important in a 3D model review, virtual reality technology is deemed promising as it gives a unique sense of scale and a depth, which is hard to match by regular 2D screens. DNV GL is also interested in alternate interaction methods, as mouse and keyboard can have some limitation when working in 3D environments. Gesture Recognition Technology has been of special interest as this can potentially offer unique approaches to working with 3D models. This thesis implements such a design review application using state-ofthe- art virtual reality- and vision-based gesture recognition technologies, coupled with the Unity game engine, a popular cross-platform game development platform and software framework. After discussing these technologies’ theoretical foundations, the thesis reviews the requirements and design of the design review application, in addition to documenting its implementation and evaluating its performance by conducting user tests. In the implemented design review application the user is able to navigate 3D models, annotate them and perform various other actions, all performed by gestures.

Implementation and initial assessment of VR for scientific visualisation: Extending Unreal Engine 4 to visualise scientific data on the HTC Vive

Download PDF.

In the visualizer: outline (left), depth buffer (middle), stencil buffer (right)

Virtual Reality (VR) for scientific visualization has been researched from the 90s, but there has been little research into the fundamental aspects of VR for scientific visualisation. Questions like “Is VR ready for adoption?”, “How does VR design differ from design for monocular systems?” are two examples of fundamental questions yet to addressed. In this paper a scientific visualiser based on the game engine Unreal Engine 4 (UE4) was developed and tested by educators and researchers. A full ray marcher was successfully implemented and a near zero-cost cutting tool was developed. VR is found to have a lot of potential for improving visualisation of data sets with structural “interleaved complexity”. VR has also been deemed ready for limited mass adoption. Through field testing visualisations of volumetric and geometric models, three major issues are identified:

Current VR hardware lacks adequate input options. Menu and interaction design must be reinvented. Furthermore, 90 FPS is required for comfortable and extended VR use, which makes most current algorithms and data sets incompatible with VR. The conclusion reached through analysis of and feedback regarding the computational cost and design challenges of VR is that VR is best utilised as a tool in already existing monocular visualisation tool kits. By using a monocular system to perform most of the encoding and filtering and then use VR for inspecting the pre-processed model, it is possible to obtain the best of both worlds.

A 3D stateroom editor

Download PDF.

Today’s software systems reach easily hundreds of thousands of lines of code, and such systems do frequently benefit from the use of state machines, which help in managing system complexity by guaranteeing completeness and consistency. To visualize such state machines, statecharts have been introduced, which also offer a formalism for orthogonal and hierarchical states. Many developers use state machines, but even with statecharts as a tool, it is a major challenge to keep an overview of the machine and its effects. Gaining an overview as a newcomer to a project is an even larger challenge. In this paper, we argue that a 3D statechart can alleviate these challenges somewhat, and present an editor for state machines that are given in SCXML, an XML notation for statecharts. This editor allows the user to explore the state machine by navigating freely in a 3D environment and make changes to the machine. The editor is a prototype and incomplete. It is an attempt to reflect the idea of having statecharts presented in 3D space.

A testbed to compare Dynamic Adaptive Streaming over HTTP network configurations

Download PDF.

Streaming video over the internet has become vastly popular over the
past decade. In recent years there have been a shift towards using the
Hypertext Transfer Protocol (HTTP) for delivery of layered, segmented,
video content. These solutions go under the name HTTP Adaptive bit-rate
Streaming (HAS). One of the streaming solutions using this is the recent
international streaming standard Dynamic Adaptive Streaming over HTTP
(DASH). The increased popularity of HAS has significantly increased the
chance that multiple HAS clients will share the same network bottleneck.
Studies show that this can introduce unwanted network characteristics,
but as of today there are no good way of running realistic evaluations of
how different network configurations will impact HAS players sharing the
same bottleneck. To solve this, we have set up a testbed capable of running
automated streaming sessions, and have performed three experiments
using the DASH industry forum’s reference player, DASH.js, to present the
capabilities of the testbed.