Monitoring Pipelines
Tabular provides a range of metrics to help you understand how well File Loader is running for your warehouse or customer. To best facilitate how you can gauge the health of File Loader operations, regardless of the observability tool you use, Tabular adopts the Open Metrics standard.
Open metrics itself is both similar to and compatible with Prometheus.
Tabular’s implementation of Open Metrics is pull-based; your observability tool makes periodic API requests, pulls that data into its own internal metrics, and begins monitoring.
A table in the Tabular database keeps track of state. Metrics include:
the latency of files loading into your table (that is, the time from the moment File Loader first sees the file to the time the event is completed)
how long it’s taking to process files
how many files have been loaded
how many files remain outstanding
Connecting your observability tool to Tabular
Tabular performs authentication and authorization via the OAuth 2 authentication. If the Prometheus client you’re using also supports OAuth 2, it is generally straightforward to connect for that endpoint.
Note
The Tabular endpoint for sending metrics is per warehouse ID – that is, metrics are labeled by warehouse ID. You cannot aggregate multiple warehouses. Also, you can access metrics only for your own warehouse.
Create a Tabular credential.
Assign permissions to access Prometheus to the user or role associated with that credential.
LIST_DATABASES on warehouse
Configure Prometheus to scrape Tabular OpenMetrics endpoint.
global:
scrape_interval: 30s
scrape_timeout: 10s
scrape_configs:
- job_name: tabular
scheme: https
oauth2:
client_id: "${client-id}"
client_secret: "${client-secret}"
token_url: "http://api.tabular.io/ws/v1/oauth/tokens"
metrics_path: /ws/v1/warehouseId/${warehouse-id}/loader/metrics
static_configs:
- targets:
- "api.tabular.io:443"
Tabular provides metric endpoints for both File Loader and change data capture.
ws/v1/warehouseId/{{warehouseId}/loader/metrics
Metric Name | Description |
|---|---|
pending_files | Number of files currently not loaded in yet |
pending_files_older_than_one_hour | Number of files not loaded in that were created over an hour ago |
oldest_pending_file | The timestamp in epoch millis of the oldest file not yet loaded. Not populated if no pending files |
avg_latency_past_hour | Avg number of seconds for files to be loaded |
loaded_files_past_hour | Number of files loaded for past hour |
loaded_bytes_past_hour | Number of bytes loaded for past hour |
ws/v1/warehouseId/{{warehouseId}/cdc/metrics
Metric Name | Description |
tables_unknown_state | Number of CDC tables with an unknown state |
table_update_lag | Time since last row update in milliseconds |
snapshots_behind_source | Number of snapshots behind from source table |
To adjust the granularity of these metrics, use the granularity query parameter.
Query Param Value | Labels |
warehouse (default) |
|
database |
|
table |
|