Arm today is announcing their Mali-D71 display processor and two IP blocks, CoreLink MMU-600 and Assertive Display 5, angling for 4K VR and HDR implementations on mobile displays.

All three blocks were developed together, and in turn they possess mutual optimizations in order to achieve VR-capable performance and HDR functionality. The D71 and MMU-600 are tightly coupled to target high resolution and frame rates, while Assertive Display 5 enables HDR capabilities and adaptation to panel variety.

Mali-D71

ARM's next generation display processor, codenamed 'Cetus', has officially been launched as the Mali-D71 display processor.

According to ARM, the Mali-D71 limits the workload needing to be handled by the GPU by performing composition, in-line rotation, high quality scaling gamma/de-gamma and other advanced imaging tasks in fixed function hardware. It does this in the final stage of the multimedia pipeline, before sending the final output to the screen, meaning the GPU never has to be involved at all. In addition, these operations are all performed with a single pass through memory as opposed to multiple passes back and forth, resulting in system power savings. For example, ARM claims that downscaling a 4K video layer for a 1440p device and composing it with a complex UI graphics layer results in a total SoC power saving of 30% compared to running the same operations with GPU software.

Mali-D71 achieves twice the performance in the same area as its predecessor when operating in side-by-side mode. Unlike its predecessor, when Mali-D71 drives a single display it can reuse the resources of the secondary display. This results in doubling the number of full frame layers it can composite, rotate and scale without adding to the overall area. This means that, also within the same silicon area as the previous generation product, Mali-D71 can also offer new enhancements such as scaling split operation, AFBC encoding of uncompressed layers, faster AFBC decoding and MMU optimizations. When the Mali-D71 Display processor is implemented with the CoreLink MMU-600, launched in parallel, the integrated Translation Buffer Unit (TBU) and tight-coupling of the two via the DTI interface dramatically reduces MMU latency.

In addition, Mali-D71 can sustain up to 4x the delay on the system bus for the same display performance compared to its predecessor, Mali-DP650. Mali-D71 adds optimizations to the Memory Subsystem. It doubles Outstanding Transaction capability, removes uncompressed rotation from the real-time path and converts uncompressed linear layers into AFBC1.2 tiled ones for more efficient rotation. This is important in high performance display processing where 4K frames must be fed to the output at 60-120 fps. For that to happen, the display processor needs to make the best use of the time it has on the system bus by prefetching pixels in the microseconds when the display is blank so that its buffers are always full of content. If the display has not received pixels in time, it's starved of content and drops frames, resulting in glitches or visible artefacts on the screen and compromising the visual quality.

Finally, Mali-D71 doubles the pixel throughput in order to achieve premium VR 4K120 performance. It does so when driving a single display in the new side-by-side operating mode. Side-by-side mode splits the image in half, efficiently using both sets of resources to process half the image each, whilst only powering one display output. For 4K60 workloads and below, side-by-side mode can be used to half the clock frequency, enabling lower voltages and saving power. For 4K120 workloads, side-by-side is mandated - in essence doubling the pixel throughput for the same target frequency. Without side-by-side mode it's only possible to reach 4K60, so the ability to halve the frame processing by performing it in parallel means you can halve the power, or double the performance.

HDR content has been coded across a broader dynamic range to incorporate greater subtlety of colour and contrast. Essentially this makes the dark areas on an image darker and richer, and the lighter areas crisper, cleaner and with better saturation, avoiding that washed out appearance you sometimes see in images with bright sunlight, for example. Mali-D71 works with Assertive Display 5 to take HDR content from providers, like Netflix and Amazon Video, and display it in full HDR quality on any type of panel, even if it's SDR. Mali-D71 itself takes the HDR video and the graphical UI overlay and blends them together into one single frame coded in standard gamma with full colour gamut, then sends it to Assertive Display 5 to convert in to the correct colour range for an SDR display.

Mobile VR presents a technical challenge. Meeting the real-time latency and throughput requirements, not to mention the pixel quality required when they're right in front of your face, pushes the display processor, and the rest of the system, to their very limits. This is where the latency tolerance comes in, as well as the power saving and performance boosting capabilities of side-by-side mode. However, when you add in the new CoreLink MMU-600 the Mali-D71 really comes into its own. The way in which the MMU-600 optimises memory subsystem allows the Mali-D71 to tolerate such long delays in the system bus by making the most of the memory subsystem available to drive the highest performing VR displays up to 4K120fps.

Where once we would sit at our desktop with half the screen showing the webinar we were ignoring while the other half showed emails or your Facebook timeline, now expect to be able to enjoy the same levels of multitasking (or procrastination) on mobile. This means the display subsystem has to work even harder in order to deliver these different activities simultaneously. Previous generations of display processor could handle up to 4 layers, whereas Mali-D71 has doubled that capability to deliver up to 8 Android composition layers in single display mode. Coupled with the ability to split your screen, this means the Mali-D71 can handle your UI, navigation bars, status info, as well as a couple of totally separate apps, without breaking a sweat.

CoreLink MMU-600

CoreLink MMU-600, Arm's next generation System MMU protects real time low latency high bandwidth 4K content. Media content protection relies on CoreLink MMU-600 to deploy TrustZone Media Protection v2 (TZMP2).

TZMP2 systems use master side filtering to avoid extensive system memory carve-out for media protection. By removing the need to allocate dedicated memory on power up, as is done in current generation systems, CoreLink MMU-600 saves approximately $3 per device.

CoreLink MMU-600 is a ground up design based on the Arm SMMUv3.1 specification and is compatible with Armv8.2 page tables. CoreLink MMU-600 supports stage-1 and/or stage-2 translations defined in Arm Distributed Virtual Memory (DVM). The mechanism extends all the Core virtualization and protection to Non-Core accelerators and devices allowing each accelerator driver to work in its own domain. Shared page tables with cores simplifies memory management between CPU and Non-CPU devices.

Arm has focused on integrating a specialized rendered version of CoreLink MMU-600 with the Mali-D71. This coupling leads to a 55% area saving for the combined solution and a 50% latency improvement by hiding MMU latency from the real-time path.

Assertive Display 5

Arm Assertive Display 5 is here, and it brings with it a complete set of HDR management features for display pipelines.

Assertive Display has long been heralded as the market-leading outdoor viewability solution. Powered by a local tone-mapping engine, it continuously adapts each individual pixel to compensate for sub-optimal conditions - such as limitations of the display, ambient light conditions and reduced backlight - to maintain a high-quality viewing experience, whether indoors or outside in bright sunshine.

Historically, display technologies have struggled to replicate what the human eye sees: traditional Standard Dynamic Range (SDR) content, such as sRGB, has a contrast ratio around 2,000:1 which means that, onscreen, shadow can become a dark mass, and the texture of light areas - such as cloud - tends to be bleached out.

HDR content, however, can achieve contrast ratios around 100,000:1.
Of course, in an ideal world, we'd all be watching HDR content on a top-end HDR panel in a darkened room to preserve the nuances of light and shade and conserve that superlative HDR image quality. Unfortunately, most of us wantonly watch our movies in a range of non-ideal viewing conditions.

Assertive Display 5 builds on its predecessor's sunlight visibility improvements, offering HDR management for display pipelines, improved color and gamut management, and enhanced power savings.

Assertive Display 5 can present the HDR content (HDR10 or HLG) to any characteristic of display (any dynamic range and gamut), delivering an HDR experience even on an SDR display. Ambient light compensation is even more important for HDR content since it will have more valuable information in dark parts. ARM says that Assertive Display's features enable users to see the real details in an image, just as they were intended.

Powered by a local tone-mapping engine, it continuously adapts each individual pixel to compensate for sub-optimal conditions, such as limitations of the display, ambient light conditions and reduced backlight. Additionally offering advanced power-saving features, it maintains a high-quality viewing experience, whether indoors or outside in bright sunshine.

Assertive Display 5's HDR management feature gives control of every stage of the display pipeline journey, enabling optimal performance - along with power-saving:

• HDR to SDR mapping: accurate mapping of HDR content to the specification of the display and compensation for ambient light conditions
• HDR to HDR mapping: compensates for HDR's vulnerability to ambient light conditions, which would otherwise negatively impact the value of HDR content and adversely affect the viewing experience
• Preserves HDR experience even at reduced backlight - especially for handset devices and notebooks - offering power saving in HDR viewing
• Supports HDR10 and HLG standards: PQ, PQ10 and HLG10