Event Schemas
Togai’s metering system is based on an event driven architecture. You can send raw usage data in the form of an event stream and Togai will process them in real-time to track usage and revenue metrics for your customers.
Basics
Event schemas are used to define the payload structure for the raw usage data you are looking to meter. Event schemas are required when you are looking to use Togai as the primary system to capture your raw usage event data or capture aggregated data periodically before transforming them within Togai’s metering layer.
In Togai, defining an event schema has 3 configurable parts -
-
Attributes - these are numerical values for the usage you are looking to meter. Examples are GB minute data for an heartbeat event, transaction amount for a fintech event, distance for a logistics tracking event. You can send individual event information or aggregated data here depending on your requirements and data model. You can define an attribute name and the unit of measurement for the attribute.
-
Dimensions - these are values used to group and filter your usage data. Examples are region & instance types/sizes for a cloud provider, payment mode for a payments company, SMS operator for a communications platform. Dimensions can be modeled as key value pairs and you define the key name while creating an event schema.
-
Enrichments - enrichments are used to compute additional information based on a formula using the existing event data and/or other variables and augment the event payload with this enriched value. An example is to track read and write rates for a storage provider separately but your pricing is based on total throughput that includes both values. In this case you can create a formula within Togai’s enrichment layer to conver the individual read and write rates into the total throughput metric that should be metered and rated.
When creating an event schema, it is important to understand how you have instrumented your product. Togai is flexible and configurable to create your own schema to match the payload that you already use for instrumenting your product. You can follow 3 basic principles while designing your schema -
- Work with what you already have
- Design with flexibility
- Choose the right idempotency key as “event_id” - You can give a unique idempotency key as the event_id when sending your event. This can be generated in your application. Togai also supports configuring your own configurable custom idempotency key based on variables (attributes and dimensions) defined in the event schema. Togai restricts users to ingest events with same id within a period of 45 days. This restriction is common for /entitled API, /ingest API and /ingestBatch API. i.e, an id used on /ingest API will not be allowed on /ingestBatch or /entitled APIs
Create an event schema
Let’s take an example of a storage company that has pricing based on 2 metrics - Storage and Performance. The unit for Storage tracking is TB-minute and performance is GBps per consumed TB.
In this case, the best practice to instrument your product would be to setup a heartbeat monitor that tracks both metrics every minute and sends that information to your metering system.
In Togai, this can be modeled as a single event schema since the same heartbeat pulse can track both data points. For the performance metric, we need to track read and write rates separately in the schema.
Example of an event schema for a cloud storage provider
Event schema lifecycle
-
Draft - An event schema can be created in a draft mode. Draft mode is meant to be used when you want to create a test schema and edit it as required. You can add/edit/delete attributes and dimensions for schema in a draft state. Schemas in draft mode cannot be associated with a usage meter. You can also archive an event schema that is in draft state. When you save an event schema in the draft state, its status will be shown as ‘Inactive’
-
Active - You can publish an event schema directly to create it in an ‘Active’ state. You can also toggle an ‘Inactive’ schema to an ‘Active’ state and vice versa. Currently, you cannot delete attributes, dimensions or enrichments for an Active schema. However, you can add new attributes, dimensions or enrichments to it.
- Archive - You can archive an Inactive schema. Archived schemas are not displayed in the dashboard but would still be stored within Togai.