MemVerge is a software defined memory solution which aggregates DRAM and Intel Optane Persistent Memory – presenting it back to applications as a continuous pool of memory.  MemVerge has created this solution to solve memory based problems with modern applications:

• Data set required by the application is too large for DRAM, requiring it to be cached in other tiers of memory and resulting in continuous reads and writes – which slows the applications down and also can degrade flash based memory media. This is a common problem especially with machine learning, and data analytics / data science work.
• Snapshot back-ups covering DRAM and other tiers of data become overly large and disruptive to processing tasks. This leads to less frequent snapshotting, and slower restores with the risk of lost transactions. This scenario is increasingly common with large analytic databases, for example in financial services and trading environments.
• Finally, server DRAM is reaching a capacity limit, and the potential to aggregate memory across types allows for fewer servers to be deployed while simultaneously increasing performance. Less cost, more performance is the result.

The magic behind the curtain is intelligent tiering between the DRAM and Intel Optane Persistent Memory so that applications always have what they need in hot tier of memory. This lowers application latency, removes I/O processing of data and dramatically speeds up applications.

Genomics

One key use case for MemVerge is genomics where the primary bottleneck is processing matrix heavy computation with intermediate stage writes and reads to storage. Processing just one pipeline of data can take days, to weeks, with normal processing hardware. Analysis of genomics workloads shows that these intermediate I/O writes and reads can consume up to 61% of the time required in processing. Using MemVerge Memory Machine the team was able to reduce this I/O load to 3%, resulting in 800x faster load times, 25x quicker execution of applications and 60% faster job throughput at all stages.

Read the full solution brief here. More details about MemVerge solutions in life sciences and research is available here on the MemVerge website.

Visual Effects (VFX) and Animation

Visual special effects and animation rely heavily on compute intensive calculations to render the high resolution scenes we all enjoy. Visual effects and animation are now a natural and expected part of all aspects of the media and entertainment industry – from broadcast TV, to movies. The data sets required to create these effects are extremely large to match the Ultra-High Definition and 4K resolutions demanded by viewers. Along with large data sets, the applications used to process these applications can crash as visual effects artists push the boundaries of their infrastructure.

Recovery from a crash can take hours – both interrupting the design process and artistic flow … and risk putting teams behind strict deadlines.

Utilising MemVerge ZeroIO snapshots in Memory Machine allows visual effects and animation teams to take regular snapshots of their work and data sets. These snapshots do not impact the operation as Memory Machine replicates from memory to memory – not out to slower tiers of disk. Likewise, recovery takes place at the speed of memory, providing a non-disruptive recovery that can recover hundreds of gigabytes in a few seconds.

Learn more from this session from the Infinity Festival, or download the Intel/Memverge case study here.

 

MemVerge and the Cloud

MemVerge Memory Machine solves problems for server based applications using large data sets, and in November 2021 they announced Big Memory Cloud technology to extend these benefits to the cloud. For applications that rely on a large amount of data in memory, or which are highly non-fault-tolerant – MemVerge Big Memory Cloud provides an containerisation approach that allows these applications to leverage cloud services.

MemVerge Big Memory Cloud brings two key technologies to market to enable more applications to migrate to the cloud. AppCapsules provide an encapsulating shell which includes all data, settings and information to run the application. Once encapsulated, the application is now fault tolerant and can be recovered, replications, migrated all at the speed of memory. The application can now take advantage of cloud services such as spot instances, move from cloud to cloud, and burst more easily from on-prem to cloud.

A Memory Machine can be configured for each app so the 3 major capabilities can be tailored to the specific needs of each app. In addition, 4 enterprise-class data services are available which are based on ZeroIO Snapshots: Time Travel, AutoSave, Thin Clone and App Migration.

MemVerge and XENON

XENON is pleased to offer the MemVerge solutions to our Australian and New Zealand customers. XENON sees MemVerge as an ideal fit for many of our HPC, Research and data science customers, unlocking new compute and price horizons and potentially opening up hybrid cloud infrastructure to new applications.

If you would like to discuss if your applications and datasets can benefit from Memory Machine – contact the XENON team today. 

Talk to a Solutions Architect