UD4D — Use data for decisions

Establishing a central data observability platform based on a vendor market tool

We ran a structured, two-phase vendor evaluation and rolled out one enterprise data-observability platform, unifying monitoring across the analytical stack.

Get in touch Download PDF
Establishing a central data observability platform based on a vendor market tool

About the client

Global biopharmaceutical company operating across multiple divisions.

The challenge

The client needed to establish end-to-end data observability across pipelines and data assets while integrating multiple analytical platforms.

  • Fragmented divisional monitoring with inconsistent coverage across teams
  • Limited visibility into pipeline execution, schema drift, and operational metadata
  • Absence of unified observability across seven analytical platforms
  • Requirement for secure hybrid deployment with SSO and SCIM provisioning
  • No standardized onboarding or monitoring process for project teams

The solution

UD4D ran a structured, two-phased vendor evaluation, then delivered an enterprise observability platform integrated across the existing stack.

  • Technical proof-of-concept evaluation of Astro Observe, Monte Carlo, and Acceldata ADOC
  • Selection of ADOC for its hybrid architecture and multi-platform observability capabilities
  • Integration with Databricks, Redshift, AWS Glue, S3, Airflow, dbt Cloud, and Trino for end-to-end monitoring
  • Configuration of SSO and SCIM through Azure Entra ID, with secure connectivity via AWS PrivateLink
  • Definition of onboarding processes, monitoring policies, and technical data-quality rules, with API-based export for custom dashboards

Business impact

  • 11

    projects in production

    Eleven projects onboarded to the new observability platform and running in production within six months, confirming rapid adoption.

  • 7

    platforms under one layer

    Databricks, Redshift, AWS Glue, S3, Airflow, dbt Cloud, and Trino monitored end to end from a single workspace.

  • 3

    observability vendors evaluated

    Astro Observe, Monte Carlo, and Acceldata ADOC benchmarked through technical PoCs, giving an evidence-based platform decision.

  • 2

    complementary quality layers

    ADOC adds technical pipeline observability alongside the existing Collibra DQ coverage, with automated alerting for failures, delays, and anomalies.

Frequently asked questions

What did the client need?

End-to-end data observability across its pipelines and data assets, integrated across multiple analytical platforms.

How is monitoring standardized across teams?

Through standardized onboarding processes, monitoring policies and technical data-quality rules, prepared in bulk for consistent setup across teams, with API-based export into custom dashboards for centralized monitoring.

Which platforms are monitored?

Databricks, Amazon Redshift, AWS Glue, S3, Apache Airflow, dbt Cloud and Trino are monitored from a single workspace, with SSO and SCIM configured through Azure Entra ID and secure connectivity via AWS PrivateLink.

What were the results?

Eleven projects were onboarded and running in production within six months, seven platforms are monitored under one layer, and ADOC’s technical observability complements the existing Collibra DQ coverage.

Related case studies