WhereScape customization/ parametrization intro

Jakub Jirka
WhereScape Architect
6
minutes to read
April 18, 2023
Data Warehouse
Data Warehouse Automation
Big Data
Business Intelligence
Data Warehouse Solutions

Intro

In the last WhereScape article, I introduced the WhereScape solution to you. In this one, I will focus on the perks of parametrization and what they could bring to you. I will be talking about how to use parameters in your flow from the beginning of your design process up to your deployment.

There are many ways to parametrize your development and deployment. This article will show you how to use different templates and methods in WS RED and 3D. We will start with a look at the different templates in RED up to their parametrization.

When, how, and what to use parameters for

Before you start creating a parameter for everything, think things through a bit more. Not every problem that you will encounter will prove a good candidate for integration into your automatic workflow. Sometimes the customized approach serves for one time only, and the additional costs of bringing it into your parametrized workflow would be very cost ineffective.

Think twice. Don’t blindly implement each exception into your parametrized workflow. There is nothing wrong with a manually coded exception, and later on, you can fully implement it as a fully working customization. With deployment automatization and code generation, redeploying your original customized solution, so that it fits the new standardized way, is a fast process.

Usually, it takes a fair amount of time to create a parametrization that will be usable in a standard way. Therefore, you should always consider how often this new way of usage would be actually used. You don’t want to spend a few days of feature development in 3D and RED just to use it twice. In this case, I think a customized feature that is well documented makes much more sense.

How to parametrize and customize your workflows

Now that we’ve addressed the reasoning behind our customization and the issue of when it makes sense, let’s take a look at different ways of achieving it in WhereScape.

Templating

WS custom templating mechanisms are the bread and butter of customization. It doesn’t matter whether you have a dozen different templates for different use cases or just one for each object, with a very complex structure and way of working. For all the approaches, you will need to orient in templating somewhat. The good thing is that you can start small with the out-of-the-box delivered templates and then move onwards into complex templates and solutions. I won’t be writing about template coding anymore here, because it would take much more space than there is for this article. But maybe in the future, we will follow up with an article focused purely on templates.

I will now describe a few ways how you can customize just by using RED and how to use 3D to move this parametrization even further.

Default templates

In WhereScape RED you can choose default template scripts for each connection.

As you can see, there is an optional template for each object type. You can have multiple connections to the same system, but for different use cases. Based on your development style, you set them up in RED or you can populate via proper usage of target locations from 3D.

This way, you can easily parametrize your workflow without any complex coding.

Extended properties

Extended properties both in 3D and RED are the second option when it comes to enriching your metadata for information, which can help with parametrization, or just add any different value. You can use them purely in RED if you wish to, but combined use in RED and 3D makes an actual difference.

Each object in RED can have its own extended properties. You can set them up in tools -> extended properties.

For each extended property, you can choose the object type that it refers to as well as connection properties. Here you can define the name, label, as well as the description. Be aware that if you want to transfer values of extended properties from 3D or between RED environments, they have to share the exact same variable name.

These extended properties can be easily used in your templates and even scripts. There is easy access using Pebble in templates. Then there are endless options for how to use these properties in your favor. Do you need a different SQL script for some occasion, or do you want different load patterns for each of the systems when both of them have come in similar parquet files? Or maybe you want to have a special type of initialization for just a few objects from a special workflow. Even populating some columns with predefined but not default values is easy and straightforward.

All of this is doable just in RED, but when you bring 3D into the game, it gets much easier and more straightforward.

Custom 3D attributes, entities, and extended properties

As mentioned above, the best approach in my opinion is to create most of your customizations and alterations with the smallest possible effort. That means defining your design methods and conversion rules in a way that means they do most of the job for you.

Let’s briefly discuss additional options in 3D and how to use them to ease your parametrization.

You can use custom attribute types, entity types, extended properties, or even parameters inside of the conversion rules. A smart combination of the given features will ease your process.

Custom entity and attribute types can be set for each model in the settings of the category.

Extended properties can be set on all objects on both the table and column levels. Always try to populate them with a conversion rule, not by hand or one by one.

If there is a need for the manual entrance of some metadata that should be populated into your objects, do it the smart way. There is an option in every conversion rule to set up a parameter that can be filled during the execution of that given conversion rule, even when used in the workflow.

You can later access it in most of the conversion rules. Fill it somewhere or make some conditions or changes based on it.

So, we briefly went through options that can be used for the customization of your scripts, templates, and conversion rules. Let’s wrap up this article with some good practice examples of how some of my clients use it.

Practical usage tips

The first recommendation I would give is to incorporate these changes as early as your design process. In other words, have it developed in 3D in a way that makes it convenient to set up for a person without much hassle. Do you need a different type of script for a specific type of satellite? Create a new attribute type that will create that type of satellite and make it easy for the person who is modeling. The conversion rules make up the rest of the work and then it is simply a case of exporting to RED.

Don’t forget that you can use the conversion rules and extended properties simply for some alterations. You can use them for additional data as well. You can place there the name of the person responsible for a design or a business area that the object belongs to.

There are many more clever ways of how to use the resource, and it can be really challenging to find the ideal approach for your development style. So, find the way that suits you best. I hope that this brief introduction helps you get off to an easier start.

More like this