Registry

In the Canso platform, we begin by registering the data sources and sinks, followed by the registration of features. We support S3 and Kafka as data sources, and S3 and Redis as data sinks. Our platform accommodates raw, derived batch, and streaming features. Once a feature is defined, it must be registered to ensure its metadata, logic, and scheduling details are stored for future reference and reuse.

Steps for Feature Registration

Create Feature Objects

• Define the raw, derived, and streaming features by creating their respective objects with their configurations.

Generate the access token

• To register the feature, generate an access token and store it as an environment variable on your data plane machine.

• Only active tokens can be used. When a token expires, generate a new one and update the environment variable.

• Take a reference from this script below to generate the access token.

Register the Features

• Register each feature object within the platform to persist their details.

• Here the feature names are primary key. You cannot register the feature again with the same name.

• Take a reference from this script to register the features,data sources and sinks.

Tool Tips

• Go to Top ⬆️

• Go back to README.md ⬅️

• Go back to data-sources.md ⬅️

• Go back to data-sinks.md ⬅️

• Go back to raw-feature.md ⬅️

• Go back to derived-feature.md ⬅️

• Go back to streaming-feature.md ⬅️

• Move forward to see deploy-feature.md ➡️