Polyglot Data Manager

Invariant Polyglot Data Manager works with the data platform to provide SQL query support for interactive and batch workloads. It can work with both small and large amount of data and scale to meet enterprise needs.

At the core of PDM is a distributed query engine that allows users and applications to processes data in parallel across multiple servers.  It allows users to query across a variety of data sources without the need of complex ETL to copy the data to a central location.  Analysts  can run both ad-hoc and batch workloads with the engine, to analyze large amounts of distributed data using SQL. 

At the core of the polyglot data manager is the Presto engine, which provides the ability to distributed query and processes data in parallel across multiple servers.  There are two types of servers - coordinators and workers.  

• Coordinator - This server handles all the user queries. It parses and analyzes the query, creates a plan and schedules the query for execution on the worker nodes.
• Workers - These servers execute the tasks as directed by the coordinator. They retrieve the data from the various data source, process it and return the results.

PDM is  designed to query  terabytes or even petabytes of data using distributed queries including Hadoop as well as object stores., where this data can be analyzed with all of the data in the lake for further analytics use cases.  

PDM runs on Linux servers – RedHat 8.x. The administration of the server is supported by a Restful API and CLI. A cluster is deployed for increased availability. 

PDM provides the ability for fast interactive queries on large datasets stored in Invariant data platforms. It can be used for analytical workloads as well as interactive business reporting use cases. 

• High Performance:  PDM is built on Presto, which provides the ANSI SQL engine and allows for fast query  ability.
• Distributed: The engines workload can easily be scaled and distributed across servers improving resiliency and performance
• Ease of Use: PDM comes with a rich set of functions that can be used by the analyst as well as drivers for connecting from a variety of IDEs.