Tables

All tables that are created in a Tabular warehouse are Apache Iceberg tables using the v2 table format.

By default, they are stored at s3://<warehouse-bucket>/<warehouse-id>/<table-id>.

Table Properties

In addition to the standard Apache Iceberg table properties, Tabular supports additional properties to enable and configure Tabular’s automated services.

Snapshot management properties

PropertyDefaultDescription
history.expire.max-snapshot-age-ms432000000 (5 days)Default max age of snapshots to keep while expiring snapshots
history.expire.min-snapshots-to-keep1Default min number of snapshots to keep while expiring snapshots
history.expire.max-ref-age-msLong.MAX_VALUE (forever)For snapshot references except the main branch, default max age of snapshot references to keep while expiring snapshots. The main branch never expires.

Data lifecycle properties

Delete rows or mask columns from a table once it passes a specified age threshold. Age is based on a user-specified timestamp column.

PropertyDescription
lifecycle.enabledWhether data lifecycle functionality is enabled or not (default false)
lifecycle.data-age-column(Required) Data will be deleted or masked from table based on values in this column
lifecycle.table.max-data-age-msRow-level TTL: Data must be at least this old to be deleted
lifecycle.column.<col_name>.max-data-age-msColumn-level masking: Age at which column masking will apply
lifecycle.column.<col_name>.transformColumn-level masking: Transform function to apply to the column. Currently only supports nullify

Compaction properties

PropertyDefaultDescription
compaction.enabledtrueWhether compaction is enabled on this table or not
compaction.strategybinpackStrategies: binpack and sort (requires a sort_order)
compaction.sort_order*The sort order to use for compaction (* table sort order is used by default)
compaction.options.partial-progress.enabledfalseEnable committing groups of files (see max-file-group-size-bytes) prior to the entire rewrite completing
compaction.options.partial-progress.max-commits10If partial progress is enabled, the max number of commits per compaction
compaction.options.max-file-group-size-bytes100 GBMax size of a commit group
compaction.options.max-concurrent-file-group-rewrites1Number of concurrent groups to rewrite in parallel
compaction.options.target-file-size-bytes*The target file size to produce during compaction (* table property: write.target-file-size-bytes)
compaction.options.rewrite-job-orderNONEbytes-asc, bytes-desc, files-asc, files-desc
compaction.options.delete-file-threshold3If a data file has this number of deletes or more, it will be rewritten regardless of its file size

Rewrite manifests properties

PropertyDefaultDescription
manifest-rewrite.enabledtrueWhether rewriting of manifests is enabled on this table or not
manifest-rewrite.wait-time-min60Minimum number of minutes to wait before triggering another manifest rewrite run

Optimizer properties

PropertyDefaultDescription
optimizer.enabledfalseWhether recommended optimizations are automatically applied to this table or not

File Loader properties

Autoload files dropped into a given path

PropertyDescription
fileloader.enabledtrue/false - determines if we try and match and load new files for this table
fileloader.pathPath we use to match newly created files to table (e.g. ‘s3://bucket-name/path’)
fileloader.file-formatjson, csv, parquet
fileloader.write-modeappend, replace
fileloader.csv.column-delimiterset a custom field/value delimiter – only applicable for file-format = ‘csv’ (default: ‘,’)
fileloader.parse-path-valuestrue/false - if ’true’, any ’name=value’ path parts will be parsed and added as columns/values to the input data
fileloader.file-exclude-glob-filterAny files that match the glob pattern set will not be loaded into the table

Default properties for new tables

These will be re-added on commit if removed. To override, set the value explicitly.

PropertyDefault
write.metadata.compression-codecgzip
write.parquet.compression-codeczstd
write.delete.parquet.compression-codeczstd
write.summary.partition-limit100
write.object-storage.enabledtrue