Administrative Tools

Superuser endpoints for dataset vacuum, index management, and cross-tenant operations.

GET /api/datasets/_vacuum

Get vacuum job configuration for automatic dataset cleanup.

Authentication: Required

Permission: tenant:superuser

Response:

{
  "enabled": true,
  "schedule": "0 2 * * 0",
  "lastRun": "2024-02-04T02:00:00Z",
  "nextRun": "2024-02-11T02:00:00Z",
  "orphanedIndices": 3,
  "totalIndices": 127,
  "config": {
    "dryRun": false,
    "retentionDays": 30,
    "deleteOrphaned": true,
    "deleteEmpty": false
  }
}

Vacuum Job Configuration:

Property	Type	Description
`enabled`	boolean	Vacuum job enabled
`schedule`	string	Cron expression for schedule
`dryRun`	boolean	Preview mode (don't delete)
`retentionDays`	integer	Keep orphaned indices for N days
`deleteOrphaned`	boolean	Delete orphaned indices
`deleteEmpty`	boolean	Delete empty indices

PUT /api/datasets/_vacuum

Update vacuum job schedule and configuration.

Authentication: Required

Permission: tenant:superuser

Request Body:

Field	Type	Description
`enabled`	boolean	Enable/disable vacuum job
`schedule`	string	Cron expression
`config`	object	Vacuum configuration

Example Request:

{
  "enabled": true,
  "schedule": "0 3 * * 0",
  "config": {
    "dryRun": false,
    "retentionDays": 14,
    "deleteOrphaned": true,
    "deleteEmpty": true
  }
}

Response:

Returns the updated vacuum job configuration.

Cron Schedule Examples:

Schedule	Description
`0 2 * * 0`	Weekly on Sunday at 2 AM
`0 3 * * *`	Daily at 3 AM
`0 0 1 * *`	Monthly on the 1st at midnight
`0 /6 * *`	Every 6 hours

Vacuum Impact

Vacuum operations delete Elasticsearch indices permanently. Always test with dryRun: true before enabling automatic deletion.

GET /api/dataset-indices

Get all dataset→esIndex mappings across all tenants.

Authentication: Required

Permission: manager:superuser (cross-tenant)

Response:

{
  "_links": {
    "self": { "href": "/api/dataset-indices" }
  },
  "_embedded": {
    "inf:dataset-index": [
      {
        "tenant": "tenant1",
        "datasetId": "sales-2024",
        "esIndex": "i5_tenant1_sales_2024",
        "records": 125000,
        "size": 45678901,
        "health": "green"
      },
      {
        "tenant": "tenant2",
        "datasetId": "orders",
        "esIndex": "i5_tenant2_orders",
        "records": 87500,
        "size": 32456789,
        "health": "green"
      },
      {
        "tenant": "tenant1",
        "datasetId": null,
        "esIndex": "i5_tenant1_orphaned_index",
        "records": 0,
        "size": 1024,
        "health": "yellow",
        "orphaned": true
      }
    ]
  },
  "start": 0,
  "count": 3,
  "total": 3,
  "summary": {
    "totalIndices": 127,
    "orphanedIndices": 3,
    "totalRecords": 15234567,
    "totalSize": 5678901234
  }
}

Use Case:

System-wide index audit and orphan detection across all tenants.

PUT /api/dataset-indices

Update dataset esIndex mappings via template (bulk operation).

Authentication: Required

Permission: manager:superuser (cross-tenant)

Request Body:

Field	Type	Required	Description
`tenant`	array	Yes	Tenant IDs to update
`dataset`	string	Yes	Dataset slug or pattern
`index`	string	Yes	Index name template

Example Request (rename indices):

{
  "tenant": ["tenant1", "tenant2"],
  "dataset": "sales-*",
  "index": "i5_{{tenant}}_sales_{{year}}"
}

Response:

{
  "updated": 12,
  "datasets": [
    {
      "tenant": "tenant1",
      "datasetId": "sales-2024",
      "oldIndex": "i5_tenant1_sales_old",
      "newIndex": "i5_tenant1_sales_2024"
    },
    {
      "tenant": "tenant2",
      "datasetId": "sales-2024",
      "oldIndex": "i5_tenant2_sales_old",
      "newIndex": "i5_tenant2_sales_2024"
    }
  ]
}

Template Variables:

Variable	Description
`{{tenant}}`	Tenant ID
`{{dataset}}`	Dataset slug
`{{year}}`	Current year
`{{month}}`	Current month
`{{id}}`	Random ID

Cross-Tenant Operation

This endpoint modifies datasets across multiple tenants. Use with extreme caution and test thoroughly before executing.

Vacuum Operation Examples

Enable Weekly Vacuum

await PUT('/api/datasets/_vacuum', {
  enabled: true,
  schedule: '0 2 * * 0',  // Sunday 2 AM
  config: {
    dryRun: false,
    retentionDays: 30,
    deleteOrphaned: true,
    deleteEmpty: false
  }
});

Dry Run Test

// Test vacuum without deleting
await PUT('/api/datasets/_vacuum', {
  enabled: true,
  schedule: '0 3 * * *',
  config: {
    dryRun: true,  // Preview mode
    retentionDays: 7,
    deleteOrphaned: true,
    deleteEmpty: true
  }
});

// Check results
const status = await GET('/api/datasets/_vacuum');
console.log(`Would delete ${status.orphanedIndices} indices`);

Aggressive Cleanup

// Clean up aggressively
await PUT('/api/datasets/_vacuum', {
  enabled: true,
  schedule: '0 3 * * *',
  config: {
    dryRun: false,
    retentionDays: 7,      // Short retention
    deleteOrphaned: true,
    deleteEmpty: true      // Delete empty indices too
  }
});

Index Management Examples

Find Orphaned Indices

const indices = await GET('/api/dataset-indices');

const orphaned = indices.items.filter(idx => idx.orphaned);

console.log(`Found ${orphaned.length} orphaned indices:`);
orphaned.forEach(idx => {
  console.log(`- ${idx.esIndex} (${idx.size} bytes)`);
});

Audit Index Sizes

const indices = await GET('/api/dataset-indices');

// Sort by size
const largest = indices.items
  .sort((a, b) => b.size - a.size)
  .slice(0, 10);

console.log('Top 10 largest indices:');
largest.forEach(idx => {
  console.log(`${idx.esIndex}: ${(idx.size / 1024 / 1024).toFixed(2)} MB`);
});

Bulk Index Rename

// Rename all sales indices across tenants
await PUT('/api/dataset-indices', {
  tenant: ['tenant1', 'tenant2', 'tenant3'],
  dataset: 'sales-2024',
  index: 'i5_{{tenant}}_sales_2024_v2'
});

Best Practices

Vacuum Configuration

Start with dry run - Always test before enabling deletion
Conservative retention - Start with 30+ days retention
Off-peak schedule - Run during low-traffic hours
Monitor results - Check vacuum logs regularly
Alert on failures - Set up notifications for vacuum errors

Index Management

Regular audits - Review indices monthly
Size monitoring - Track index growth trends
Health checks - Monitor cluster health
Backup strategy - Snapshot before major operations
Documentation - Maintain index naming conventions

Cross-Tenant Operations

Test on single tenant - Verify changes work correctly
Backup first - Always backup before cross-tenant changes
Staged rollout - Update tenants in batches
Rollback plan - Have a plan to revert changes
Communication - Notify affected users of changes

Performance

Batch operations - Group related changes
Off-peak execution - Run during maintenance windows
Progress monitoring - Track long-running operations
Resource allocation - Ensure adequate cluster resources
Throttling - Rate-limit operations to prevent overload

Production Safety

All administrative operations should be tested in non-production environments first. Vacuum and bulk index operations are irreversible.

GET /api/datasets/_vacuum​

PUT /api/datasets/_vacuum​

GET /api/dataset-indices​

PUT /api/dataset-indices​

Vacuum Operation Examples​

Enable Weekly Vacuum​

Dry Run Test​

Aggressive Cleanup​

Index Management Examples​

Find Orphaned Indices​

Audit Index Sizes​

Bulk Index Rename​

Best Practices​

Vacuum Configuration​

Index Management​

Cross-Tenant Operations​

Performance​

GET /api/datasets/_vacuum

PUT /api/datasets/_vacuum

GET /api/dataset-indices

PUT /api/dataset-indices

Vacuum Operation Examples

Enable Weekly Vacuum

Dry Run Test

Aggressive Cleanup

Index Management Examples

Find Orphaned Indices

Audit Index Sizes

Bulk Index Rename

Best Practices

Vacuum Configuration

Index Management

Cross-Tenant Operations

Performance