Core CRUD

Basic dataset creation, retrieval, update, and deletion operations.

GET /api/dataset-id

Generate a unique slug for a dataset from a name query parameter.

Authentication: Required

Query Parameters:

Parameter	Type	Required	Description
name	string	Yes	Desired dataset name

Response:

{
  "id": "sales-analysis-2024"
}

Use Case:

Call this endpoint before creating a dataset to generate a valid, unique slug from a user-provided name.

---

GET /api/datasets

Search and filter datasets with pagination and aggregations.

Authentication: Required

Query Parameters:

Parameter	Type	Default	Description
q	string	-	Full-text search query
type	string	-	Filter by type: datasource, upload, external
datasource	string	-	Filter by datasource ID
sort	string	name	Sort field (prefix with - for descending)
limit	integer	30	Results per page
start	integer	0	Pagination offset

Response:

The response is a paginated HAL collection. Dataset items are returned under _embedded["inf:dataset"], with pagination metadata and aggregations at the top level.

{
  "_links": {
    "self": { "href": "/api/datasets{?sort,limit,start,q,type,datasource}", "templated": true },
    "next": { "href": "/api/datasets?start=30&limit=30" }
  },
  "_embedded": {
    "inf:dataset": [
      {
        "_links": {
          "self": { "href": "/api/datasets/d4f8a2b1-..." },
          "inf:data": { "href": "/api/datasets/analytics:sales-2024/data" },
          "inf:run": { "href": "/api/datasets/analytics:sales-2024/run" },
          "inf:fields": { "href": "/api/datasets/analytics:sales-2024/fields" },
          "inf:filters": { "href": "/api/datasets/analytics:sales-2024/filters" }
        },
        "id": "d4f8a2b1-...",
        "slug": "sales-2024",
        "name": "Sales Data 2024",
        "description": "Quarterly sales analysis",
        "ownerId": "analytics",
        "datasourceId": "a1b2c3d4-...",
        "records": 125000,
        "size": 45678901,
        "shared": false,
        "embedded": false,
        "esIndex": "tenant_abc.sales_2024.12345",
        "esType": "data",
        "createdAt": "2024-01-15T10:00:00.000Z",
        "updatedAt": "2024-02-08T14:30:00.000Z",
        "dataUpdatedAt": "2024-02-08T14:30:00.000Z",
        "datasource": {
          "id": "a1b2c3d4-...",
          "name": "MySQL Production",
          "type": "mysql",
          "embedded": false
        }
      }
    ]
  },
  "start": 0,
  "count": 1,
  "total": 150,
  "aggs": {
    "type": [
      { "id": "datasource", "label": "From a Datasource", "value": 95 },
      { "id": "upload", "label": "From a file", "value": 42 },
      { "id": "external", "label": "Externally provided", "value": 13 }
    ],
    "datasource": [
      { "id": "a1b2c3d4-...", "label": "MySQL Production", "value": 58 },
      { "id": "e5f6a7b8-...", "label": "Postgres Analytics", "value": 37 }
    ],
    "total": 150
  },
  "permissions": {
    "create": true
  }
}

Key Fields:

Field	Description
start	Pagination offset
count	Number of items in this page
total	Total matching datasets
aggs.type	Counts by dataset type (datasource, upload, external)
aggs.datasource	Counts by datasource
aggs.total	Total datasets accessible to the user
permissions	Collection-level permissions (e.g. create)

Each embedded dataset includes a datasource object with id, name, type, and embedded fields. Items also include _links for navigating to sub-resources (data, run, fields, filters, etc.).

Search Optimization

Use the type and datasource parameters to filter results before applying full-text search for better performance.

---

GET /api/datasets-list

Get a complete list of readable datasets with tags, sharing info, and Elasticsearch stats.

Authentication: Required

Response:

The response is a plain JSON array (not HAL-wrapped). Each entry includes nested datasource/query objects, sharing info, tags (as IDs), and the user's permissions. Record counts are recalculated based on the requesting user's applied filters.

[
  {
    "id": "d4f8a2b1-...",
    "slug": "sales-2024",
    "name": "Sales Data 2024",
    "description": "Quarterly sales analysis",
    "ownerId": "analytics",
    "ownerName": "Analytics Team",
    "teamId": "analytics",
    "teamName": "Analytics Team",
    "naturalId": "analytics:sales-2024",
    "datasourceId": "a1b2c3d4-...",
    "folderId": null,
    "records": 125000,
    "size": 45678901,
    "shared": true,
    "favorite": true,
    "datasetFieldReportAccess": false,
    "esType": "data",
    "esIndex": "tenant_abc.sales_2024.12345",
    "createdAt": "2024-01-15T10:00:00.000Z",
    "updatedAt": "2024-02-08T14:30:00.000Z",
    "dataUpdatedAt": "2024-02-08T14:30:00.000Z",
    "datasource": {
      "id": "a1b2c3d4-...",
      "type": "mysql",
      "name": "MySQL Production",
      "permissions": { "edit": true, "write": true }
    },
    "query": {
      "id": "q1r2s3t4-...",
      "language": "sql",
      "datasourceId": "a1b2c3d4-...",
      "defnUpdatedAt": "2024-02-01T12:00:00.000Z",
      "datasource": {
        "id": "a1b2c3d4-...",
        "type": "mysql",
        "name": "MySQL Production",
        "permissions": { "edit": true }
      },
      "permissions": { "edit": true, "write": true }
    },
    "tags": ["tag-uuid-1", "tag-uuid-2"],
    "sharing": [
      { "principalId": "marketing", "accessLevel": 2 }
    ],
    "permissions": {
      "edit": true,
      "write": true,
      "delete": true,
      "share": true,
      "copy": true,
      "refresh": true,
      "export": true,
      "rename": true,
      "add-visual": true,
      "delete-visual": true,
      "change-owner": true,
      "assign-tags": true
    }
  }
]

Key Fields:

Field	Description
naturalId	Composite ID (ownerId:slug) used in URL paths
ownerName	Display name of the owner (user displayName or team name)
favorite	Whether the current user has favorited this dataset
records	Record count (recalculated per-user based on applied filters)
size	Elasticsearch index size in bytes
datasource	Nested datasource object (null if user lacks read access)
query	Nested query object with language and its own datasource
tags	Array of tag UUIDs (not full tag objects)
sharing	Array of share entries with principalId and accessLevel
permissions	User's permissions on this dataset

For user-owned datasets, username and displayName appear instead of teamId/teamName.

Use Case:

This endpoint returns all datasets the user can access with complete metadata, suitable for building dataset picker UIs or dashboards.

---

GET /api/datasets/{id}

Get a single dataset with full details including fields, filters, datasource, and tags.

Authentication: Required

Path Parameters:

Parameter	Type	Description
id	string	Dataset ID or slug

Response:

The response is a full HAL resource with related entities (query, datasource, fields, filters, snapshots) in _embedded, and extensive _links for navigating to sub-resources and actions.

{
  "_links": {
    "self": { "href": "/api/datasets/analytics:sales-2024" },
    "inf:draft": { "href": "/api/datasets/analytics:sales-2024/draft" },
    "inf:run": { "href": "/api/datasets/analytics:sales-2024/run" },
    "inf:refresh": { "href": "/api/datasets/analytics:sales-2024/refresh" },
    "inf:data": { "href": "/api/datasets/analytics:sales-2024/data" },
    "inf:search": { "href": "/api/datasets/analytics:sales-2024/_search" },
    "inf:fields": { "href": "/api/datasets/analytics:sales-2024/fields" },
    "inf:filters": { "href": "/api/datasets/analytics:sales-2024/filters" },
    "inf:visuals": { "href": "/api/datasets/analytics:sales-2024/visuals" },
    "inf:comments": { "href": "/api/datasets/analytics:sales-2024/comments" },
    "inf:dataset-shares": { "href": "/api/datasets/analytics:sales-2024/shares" },
    "inf:exporters": { "href": "/api/datasets/analytics:sales-2024/exporters" },
    "inf:favorite": { "href": "/api/datasets/analytics:sales-2024/favorite" },
    "inf:flow": { "href": "/api/datasets/analytics:sales-2024/flow" },
    "inf:owner": { "href": "/api/datasets/analytics:sales-2024/owner" },
    "inf:mapping": { "href": "/api/datasets/analytics:sales-2024/mapping" },
    "inf:permissions": { "href": "/api/datasets/analytics:sales-2024/permissions" },
    "inf:query": { "href": "/api/queries/q1r2s3t4-..." },
    "edit": { "href": "/api/reports/analytics:sales-2024/edit" }
  },
  "_embedded": {
    "inf:query": {
      "_links": { "self": { "href": "/api/queries/q1r2s3t4-..." } },
      "id": "q1r2s3t4-...",
      "language": "sql",
      "datasourceId": "a1b2c3d4-...",
      "defnUpdatedAt": "2024-02-01T12:00:00.000Z"
    },
    "inf:datasource": {
      "_links": { "self": { "href": "/api/datasources/a1b2c3d4-..." } },
      "id": "a1b2c3d4-...",
      "name": "MySQL Production",
      "type": "mysql"
    },
    "inf:field": [
      {
        "_links": { "self": { "href": "/api/datasets/analytics:sales-2024/fields/order_id" } },
        "name": "order_id",
        "label": "Order ID",
        "dataType": "string",
        "position": 0
      },
      {
        "_links": { "self": { "href": "/api/datasets/analytics:sales-2024/fields/amount" } },
        "name": "amount",
        "label": "Sale Amount",
        "dataType": "number",
        "position": 1
      }
    ],
    "inf:filter": [
      {
        "_links": { "self": { "href": "/api/datasets/analytics:sales-2024/filters/f1a2b3c4-..." } },
        "id": "f1a2b3c4-...",
        "name": "High Value Orders",
        "ownerId": "admin",
        "public": true
      }
    ],
    "inf:snapshot": []
  },
  "id": "d4f8a2b1-...",
  "slug": "sales-2024",
  "name": "Sales Data 2024",
  "description": "Quarterly sales analysis",
  "ownerId": "analytics",
  "datasourceId": "a1b2c3d4-...",
  "embedded": false,
  "esIndex": "tenant_abc.sales_2024.12345",
  "esType": "data",
  "records": 125000,
  "size": 45678901,
  "shared": true,
  "params": {},
  "settings": {},
  "flow": null,
  "dataUpdatedAt": "2024-02-08T14:30:00.000Z",
  "createdAt": "2024-01-15T10:00:00.000Z",
  "updatedAt": "2024-02-08T14:30:00.000Z",
  "tags": ["tag-uuid-1"],
  "permissions": {
    "edit": true,
    "write": true,
    "delete": true,
    "share": true,
    "copy": true,
    "refresh": true,
    "export": true,
    "rename": true,
    "add-visual": true,
    "delete-visual": true,
    "change-owner": true,
    "change-owner-to-team": true,
    "assign-tags": true,
    "modify-flows": true,
    "modify-settings": true,
    "script-fields": true,
    "create-filter": true,
    "edit-filter": true,
    "change-filter-privacy": true,
    "set-index": true,
    "replace-file": false,
    "create-data-view": true
  }
}

Key Details:

Section	Description
_links	~30 HAL links to sub-resources and actions (subset shown above)
_embedded["inf:query"]	The dataset's query definition (language, datasourceId)
_embedded["inf:datasource"]	The query's datasource
_embedded["inf:field"]	Dataset fields sorted by position, then name
_embedded["inf:filter"]	Filters accessible to the current user (public + user-owned)
_embedded["inf:snapshot"]	Child snapshot datasets
tags	Array of tag UUIDs (extracted from tag entities)
records	Record count from Elasticsearch (recalculated per-user)
size	Index size in bytes from Elasticsearch stats
permissions	Full set of permission flags for the current user

Post-Processing:

• Fields are sorted by position, then name
• Record count retrieved from Elasticsearch (reflects user's applied filters)
• Size calculated from index stats
• Filters are filtered to only show public filters and those owned by the current user

---

POST /api/datasets

Create a new dataset with optional data import or upload.

Authentication: Required

Permission: dataset:create

Request Body:

Field	Type	Required	Description
id	string	Yes	Unique dataset slug
name	string	Yes	Dataset display name
description	string	No	Dataset description
type	string	No	Dataset type: datasource, upload, external
datasourceId	string	No	Datasource ID for datasource type
query	string	No	SQL query for datasource type
params	array	No	Query parameters
queryId	string	No	Create from existing query
flow	array	No	Flow transformation steps
data	array	No	Initial data records
upload	string	No	Upload ID for file import
uploadOptions	object	No	Upload parsing options
esIndex	string	No	Custom Elasticsearch index name

Timeout: 15 minutes (900000ms)

Example Request (from datasource):

{
  "id": "customer-orders",
  "name": "Customer Orders",
  "description": "All customer orders from production database",
  "type": "datasource",
  "datasourceId": "mysql-prod",
  "query": "SELECT * FROM orders WHERE created_at >= {{startDate}}",
  "params": [
    {
      "name": "startDate",
      "dataType": "date",
      "defaultValue": "2024-01-01",
      "required": true
    }
  ]
}

Example Request (from upload):

{
  "id": "expense-report",
  "name": "Q1 Expense Report",
  "description": "Employee expenses for Q1 2024",
  "type": "upload",
  "upload": "upload-abc123",
  "uploadOptions": {
    "hasHeaders": true,
    "delimiter": ",",
    "encoding": "utf-8"
  }
}

Example Request (with inline data):

{
  "id": "product-list",
  "name": "Product List",
  "description": "Manual product catalog",
  "data": [
    { "sku": "ABC-001", "name": "Widget A", "price": 19.99 },
    { "sku": "ABC-002", "name": "Widget B", "price": 24.99 }
  ]
}

Response:

Returns the created dataset object with generated fields and initial record count.

Long-Running Operation

Dataset creation with data import can take several minutes for large datasets. Use the progress parameter to track operation status.

---

PUT /api/datasets/{id}

Update dataset properties.

Authentication: Required

Permission: dataset:write

Path Parameters:

Parameter	Type	Description
id	string	Dataset ID or slug

Request Body:

Field	Type	Description
name	string	Dataset display name
description	string	Dataset description
datasourceId	string	Change datasource
query	string	Update SQL query
params	array	Update query parameters
flow	array	Update flow steps

Response:

Returns the updated dataset object.

Example Request:

{
  "name": "Customer Orders (Updated)",
  "description": "Updated description with new date range",
  "query": "SELECT * FROM orders WHERE created_at >= {{startDate}} AND status = {{status}}",
  "params": [
    {
      "name": "startDate",
      "dataType": "date",
      "defaultValue": "2024-01-01",
      "required": true
    },
    {
      "name": "status",
      "dataType": "string",
      "defaultValue": "completed",
      "required": false
    }
  ]
}

Update Scope

Use this endpoint to update metadata and configuration. To modify data or fields, use the specialized endpoints under Data Operations and Fields.

---

DELETE /api/datasets/{id}

Delete a dataset permanently, including all data, fields, filters, visuals, and snapshots.

Authentication: Required

Permission: dataset:write

Path Parameters:

Parameter	Type	Description
id	string	Dataset ID or slug

Response:

204 No Content

Permanent Deletion

This operation cannot be undone. All associated data, fields, filters, visuals, comments, and snapshots will be permanently deleted.

---

PATCH /api/datasets/{id}

Apply JSON Patch operations to a dataset.

Authentication: Required

Permission: dataset:write

Path Parameters:

Parameter	Type	Description
id	string	Dataset ID or slug

Request Body:

JSON Patch document (array of operations).

Example Request:

[
  { "op": "replace", "path": "/name", "value": "New Dataset Name" },
  { "op": "add", "path": "/description", "value": "New description" },
  { "op": "remove", "path": "/query" }
]

Response:

Returns the patched dataset object.

Notes:

• ownerId changes are filtered out for security
• remove operations are converted to add with null value
• Standard JSON Patch operations supported: add, remove, replace, copy, move, test

Use Cases

PATCH is useful for programmatic updates where you want to modify specific fields without sending the entire dataset object.