Custom Node Types in Coalesce: Unlocking Flexibility and Reusability
Nodes are the foundational building blocks in coalesce.io, serving as database objects like tables or views. Each node belongs to a specific type, equipped with a predefined user interface, a create template, and a run template. While coalesce.io provides four standard node types, custom node types allow users to adapt and extend these capabilities for unique requirements.
In this article:
What Are Custom Node Types?
Custom node types enable users to define reusable database object patterns. By specifying a user interface (UI), Data Definition Language (DDL), and Data Manipulation Language (DML), users can create tailored solutions for patterns such as stages, dimensions, facts, hubs, and links. Parameters and macros make these custom types even more adaptable and reusable.
Why Create Custom Node Types?
Custom node types address two key needs:
- Custom Needs: Standard node types may not cover specific use cases.
- Reusability: Custom node types eliminate the redundancy of repeatedly creating similar nodes, saving time and effort.
Build Better Data Platforms
Practical architecture insights for modern data teams. Join 8,000+ data professionals.
Get Free InsightsKey Components of Custom Nodes
Node Definition and UI Configuration
The node definition specifies the UI elements, such as materialization selectors, toggles, dropdowns, and text boxes. These components define how users interact with and configure the custom node.
Create Template
The create template includes SQL logic for generating tables or views. It supports column transformations, comments, clustering keys, and all Snowflake DDL features.
Run Template
The run template defines DML operations, such as inserting data, applying incremental or merge strategies, and performing transformations. These operations are executed exclusively for table-based nodes and utilize all Snowflake DML features.
Get Started with Custom Node Types
Custom node types in coalesce.io empower teams to design reusable, scalable solutions tailored to specific needs. By leveraging their flexibility, you can streamline development, reduce repetitive tasks, and maximize efficiency in your data workflows.
Hi Tim Kirschke,
I need to create customSQL to meet my business requirement. Could you please help me on how to create customSQL node in coalesce?
Hi Sreenivasulu Reddy,
Depending on the kind of custom SQL, you have multiple options to do that.
For column level transformations, you can use the “Transform” section inside a nodes columns: https://docs.coalesce.io/docs/build-your-pipeline/transforms
If you want to do custom JOINs and/or GROUP BYs, you can do it inside a node via the JOIN window: https://docs.coalesce.io/docs/build-your-pipeline/the-build-interface/join-tab
For wrapping re-usable patterns into a custom Node Type, you can create a new node type from your Build Settings and fully customize the whole CREATE & INSERT statement: https://docs.coalesce.io/docs/build-your-pipeline/user-defined-nodes
Best regards
Tim