How Technology like Snowflake and Simcorp Dimension Can Optimize Data Warehouse Storage

Editor’s note: This article is co-authored by Gaurav Jangid, Global Head of Data and Cloud Technology at Axxsys Consulting, Jakob Banning, Senior Manager at Axxsys Consulting, and Maxwell Crooks, Senior Consultant for Limited Partner Services at Lionpoint Group.

Platforms for investment management are evolving to meet the demands of a rapidly changing landscape. Collecting and using massive amounts of data for complex market and risk analysis, combined with increasingly detailed reporting requirements for environmental and social factors, necessitates new considerations when establishing or expanding investment platforms to ensure they meet future needs.

Local servers and data warehouses provide greater control of storage and computing power, but pose challenges in managing exponentially growing data. SimCorp has been expanding its range of cloud-compatible applications and services, known collectively as SCDaaS, to address this issue. It recently announced a partnership with Snowflake to leverage its cloud data warehouse platform.

Snowflake is a cloud-native data platform that provides powerful tools for storing, analyzing, and integrating data from the marketplace. Snowflake, built in the cloud, offers elastic scaling of processing power and storage, very high levels of encryption and data protection, and high availability exceeding 99.999%. Because of its ability to dynamically scale resources, users only pay for the computing power and storage that they use, making it a cost-effective solution for investment platforms.

Scaling resources on-the-fly is a key benefit of a cloud data warehouse. By elastically adjusting to the needs of your organization, users only pay for the computing power and storage used. Further, management and projection of infrastructure requires proficient architects and requires valuable time being spent before implementation gets started.

The Difference Between SimCorp Dimension and Snowflake

Because of the tasks they are designed for, SimCorp Dimension (SCD) and Snowflake run on fundamentally different infrastructure. SCD is based on an Oracle transactional database (OLTP), whereas Snowflake is based on an analytical database (OLAP). This means that SCD is extremely fast at retrieving and updating data, whereas Snowflake is extremely efficient at storage and security. Users can access market data from vendors such as FactSet, MSCI, S&P, Rimes, Qontigo, and many more through Snowflake’s data marketplace.

It is possible to connect SCD and Snowflake using the Data Warehouse Manager and selecting Snowflake as the destination for the data model, reducing the need for bespoke connectors. Snowflake as a cloud data warehouse also offers industry-leading features like up to 60-day time travel, zero-copy clones, and simple data sharing. Snowflake also seamlessly integrates with data visualization tools like Power BI to help investors make more informed decisions.

Time-travel gives a grace period of up to 60 days (configurable) of reversing any destructive action on any object in the account, e.g. tables may easily be un-dropped. Likewise, cloning of objects is near instantaneous since data is not replicated but is essentially referenced under a new object name. Providing real-time data to clients is also efficient: either through creating a reader account with access to authorized data or publishing data as a data provider in the Data Marketplace.

From a business perspective, Snowflake integration provides numerous advantages:

• Faster and easier data integrations – No bespoke connections
• A rich Data Marketplace with validated data
• Easy roll-back due to improved data retention

Snowflake also seamlessly integrates with data visualization tools such as Power BI, to support investors in making more informed investment decisions. Power BI is a data visualization and business intelligence tool that can be highly effective in private markets, it allows users to connect and analyze large amounts of data and present it in a clear and interactive format. The dashboards and reports generated by Power BI can be used to monitor key performance indications, identify trends, and generate alpha.

Snowflake’s cloud-based architecture enables easy collaboration, data and insight sharing within an organization, and modular architecture via decoupled integrations. The integration of data warehousing tools, industry-leading platforms, and data visualization tools enables real-time data analysis, dynamic data refresh, and seamless team and system collaboration. Organizations can advance their business by effectively managing their data and gaining valuable insights.

Next Steps

Lionpoint Group and Axxsys, both part of Alpha FMC, provide a broad range of capabilities around Data Warehouse technology and innovation. Our expert consultants possess a unique insight into multiple platforms, including how to integrate them successfully and leverage them to their fullest potential. We are committed to bringing our client’s platforms to the next level, so if you would like to know more about how we can support your data warehouse programmes please contact us by filling out the form below: