The Application and Prospect of Mistika VR Video Stitching and Processing Software

Virtual reality technology (VR) is widely used in various industries, opening up a new form of sensory experience for human beings. VR video stitching is a kind of "work hard" like Rotoscope. But if someone can make this thing easier, it will definitely be welcomed, so Mistika VR comes into being.


Overview
First of all, it is necessary to introduce Mistika's developer, Spain's SGO (www.sgo.es), which is well-known in the field of top film production. It mainly develops and produces ultra-high-end post-synthesis, coloring and stereo effect processing systems. The Mistika series solution produced by SGO is adopted by the highest-level Hollywood and domestic blockbusters as the core interactive system for coloring, special effect preview, stereo effect production and the completion of the whole film, as well as the tool for processing and output of the final film master edition such as "Singular Dr." "Galaxy Guard 2", "Star Wars 7", "Star Trek Series", "Mission Impossible Series", "Hobbit Series", "Transformers Series", "Seeking Dragons", " "Hunting the Demon", "Intelligence Takes the Tiger Mountain", "Di Renjie Series".
Mistika is a total solution that combines Dolby Vision, HDR High Dynamic Video, HFR High Frame Rate, UHD & 8K High Resolution, and VR Processing Capabilities. It is widely adopted by Hollywood and domestic major film and television companies. Here you can see some of the latest videos that Mistika has served: https://www.sgo.es/on-screen/.
Perhaps seeing the prospects of the VR industry, and to open up more mid and low-end markets, at this year's NAB exhibition, SGO officially has announced that its trump card system, Mistika technology, will be decentralized, packaged into different sub-software to meet the needs of different industries. Mistika VR is the first software they released, using the same advanced technology as Mistika's large system, so that ordinary users can quickly complete VR video production. Just because of this background, I have great expectations for the effect of Mistika VR.

The prospect of VR video
After introducing SGO company, I want to talk about VR video itself. In 2016, known as the first year of VR, Facebook and HTC released VR helmets respectively. Then SONY released the VR helmet based on PS games, and the major domestic video sites set up VR sub-channels, and there are even "the first monster stocks in history" such as Storm Video with continuous multiple trading limits. In a lively and dazzling, the Chinese have also began to explore the VR video production.
After a year of development, Google announced the Daydream program for VR. At the Apple WWDC Global Developers Conference in June this year, Apple released a series of new hardware products, announced that it has prepared for VR, and began to promote the H265 encoded format video. This shows that both the computer platform and the mobile platform, the two major camps (PC-MAC, IOS-Android) have begun to fully support VR, ready for software and hardware, it can be said that the big guys are "going on board." Therefore, according to the current form, VR should be far more than a short-lived new technology wave, but a new technology that will change the way people listen, and even become the next universal technology platform. Therefore, as one of the components in visual aspects, VR video is very promising.

The importance of VR stitching technology
The full name of VR is Virtual Reality. What it wants to create is an effect called immersive experience, which, in short, makes the viewer confuse reality from virtuality. To do this, technically, the video must reach a resolution of more than 4K for each eye, and refresh 60 frames per second. In this way, the human eye will no longer feel pixel granularity, but also to maximize the avoidance of "dizziness" situation. Coupled with spatial audio technology, and other sensing technology, it can achieve the effect to make people immerse.
Therefore, for professional VR video shooting, as far as possible to create high resolution, high frame rate video is the key to improve immersion. And to do that, you need a device that can shoot at very high resolution and 360 degrees video. Unfortunately, with the current hardware technology of photographic equipment, it is not yet possible to create such a high standard equipment with only one camera. Therefore, the current general approach is to use multiple cameras combined to form a circular array according to a certain angle to cover the 360-degree environment. Eventually, each camera will take part of the images separately, and achieve VR video of ultra-high resolution through post-synthesis.

(various VR cameras)
The picture below is Gopro's official Omni camera, which is a combination of 6 Gopro. 6 different pictures are taken, and finally VR video is stitched through the later stage.
(Gopro Omni VR camera) (Materials taken by each lens of the VR camera) Due to the relationship between resolution and calculation accuracy, the camera is generally unable to achieve perfect stitching when shooting, So making VR videos, stitching up the VR camera material and turning them into a complete video is probably the most important and time-consuming step in VR production today.
(VR video after stitching) Due to the huge amount of material captured by film and television, the accuracy and speed of VR video stitching determine the efficiency of VR production at the current stage. Famous VR video producer and director Lucas Wilson says: We are now in the early stages of VR. People are now working on a variety of projects, short or long films, reality shows or advertisements, but the real bottleneck is early video processing and the right process. We all know that, sooner or later, VR stitching should be done automatically, with one click. But before that technology emerges, whoever can provide a quick and accurate VR video stitching process that can quickly and accurately stitch and process a large number of VR videos, then he will achieve commercial success at this stage.
In other words, VR video stitching is "hard work" just like Rotoscope. But if someone can make it easier, it will be welcomed, so Mistika VR comes into being.

Project management of Mistika VR
After Mistika VR is installed and activated, double-click the software icon. When the software opens, the first thing you see is the project management window of Mistika VR. It can be said that Mistika VR manages VR video stitching work in a project unit. Users can create new projects. After clicking "New Project", the new project window will be popped up.

New project window
In the window, the user can output the project name, resolution and frame rate, where the resolution rate is the output resolution rate of the final stitched VR video. And the frame rate is also the final output frame rate, it is best to keep the same frame rate as the original shooting material, otherwise it may lead to frame rate mismatch and shadow phenomenon.
It is worth mentioning that I later find that the resolution and frame rate of the project can be modified at any time, without affecting the result of stitching, so it seems very flexible. For example, I build a project based on 1920X960 and prepare to output it to Youtube. After the stitching is completed, the director suddenly decides to want a 4K version. At this time, I don't need to redo the stitched part, just need to go to the project settings, change the output resolution rate to 4K, and re-output the VR video. This takes into account the actual situation of daily work and is very intimate. After setting up the project, double-click on the project and you will be taken to the Mistika VR software. The user will see a blank interface.

Mistika VR's initial interface
At this time, you also need to establish a timeline. If a project is a workshop, then the timeline is one station in the workshop. A project can have multiple timelines to target different cameras or shoot content. This architecture makes me feel very flexible.

File operation menu
Users can create or read the timeline at any time and keep the current timeline. Then, the user drags the captured material into the Mistika interface in groups, and can start the stitching work directly. Mistika VR has been very comfortable to me so far. Whether it's project-timeline-material management framework or material import, it feels very intuitive and doesn't need to be considered or calculated. So let's take a look at how the real stitching of Mistika VR behaves.

Mistika VR module
The software interface of Mistika VR is mainly divided into 8 parts, among which 6 modules are commonly deployed by default, and 2 are not commonly used. All modules can adjust the size of the area by dragging the edges, and the lower parameter modules can be folded at any time to make more space for other modules. The whole software is in such a whole framework. There is no secondary interface or multiple menus, and it can be said at a glance.

Unique template-based workflow
After introducing the working interface of Mistika VR, let's take a look at its workflow. Unlike the complex edge-checking and labeling operations I first imagined, Mistika VR simplifies all this through the template approach. First of all, SGO believes that as long as the mainstream VR cameras in the market can create templates for them, as long as they are captured by these cameras, directly import the template after importing Mistika VR, 90% of the work of VR video stitching can be completed.
Just right-click on the blank area of the list of materials and select "Load Preset" we can read the official pre-created stitching template.

Ability to create custom templates
Sometimes, we use our own DIY or 3D printer camera combination tools, or new VR cameras. If your camera doesn't have a template, Mistika VR offers two other solutions. One is to submit the relevant parameters of the material and photographic equipment directly to the official, who will customize a special stitching template for you. Another is that Mistika VR provides the option to use third-party software to help create templates, and users can choose to use PTGui or Hugin (both famous panorama-making software, the former charges, and the latter is free of charge) inside the software to help create stitching templates and import them into Mistika VR.
Through this template based workflow, it can be said to maximize the reduction of duplication of labor. Material taken by the same equipment can be quickly and perfectly stitched by applying template and fine-tuning. SGO officials are very considerate about using different templates if there are multiple different devices shooting the material.

You can output video in Prores format
If you have multiple stitching materials, you can choose "All Shots" for all output. What's more, the output speed of Mistika is very amazing, which can be said to be 10-20 times the current output speed of similar software, which is particularly gratifying.
The above is the routine process of Mistika VR handling VR video stitching, but the efficiency and convenience of the software is believed to have been fully reflected. In the whole process, I need to manually adjust the brightness of only one lens, and the rest is all about clicking a button or dragging a mouse, and Mistika VR algorithm automatically helps me to complete the rest steps. And the whole operation is very smooth, any step can be completed in a few seconds, without waiting.