Skip to main content

Managing Multiple Workflows

AI Workflow is designed to handle multiple workflows simultaneously. This guide explains how to create, manage, and coordinate multiple workflows.

Overview​

AI Workflow supports running multiple independent workflows concurrently. Each workflow operates in its own isolated context with its own configuration, state, and execution history.

Creating Multiple Workflows​

Basic Workflow Creation​

import { AIWorkflow } from 'ai-workflow';

const workflow = new AIWorkflow({
  apiKey: process.env.AI_WORKFLOW_API_KEY
});

// Create multiple workflows
const workflow1 = await workflow.create({
  name: 'Data Processing Workflow',
  description: 'Processes incoming data'
});

const workflow2 = await workflow.create({
  name: 'Content Generation Workflow',
  description: 'Generates content using AI'
});

const workflow3 = await workflow.create({
  name: 'Analysis Workflow',
  description: 'Analyzes data and generates reports'
});

Batch Workflow Creation​

// Create multiple workflows at once
const workflows = await Promise.all([
  workflow.create({ name: 'Workflow 1', description: 'First workflow' }),
  workflow.create({ name: 'Workflow 2', description: 'Second workflow' }),
  workflow.create({ name: 'Workflow 3', description: 'Third workflow' })
]);

console.log(`Created ${workflows.length} workflows`);

Listing and Filtering Workflows​

List All Workflows​

// Get all workflows
const allWorkflows = await workflow.list();

console.log(`Total workflows: ${allWorkflows.length}`);

Filter Workflows​

// Filter by status
const activeWorkflows = await workflow.list({
  status: 'active'
});

// Filter by name
const dataWorkflows = await workflow.list({
  name: 'Data'
});

// Filter by date range
const recentWorkflows = await workflow.list({
  createdAfter: '2024-01-01',
  createdBefore: '2024-12-31'
});

Workflow Organization​

Tagging Workflows​

// Add tags to organize workflows
await workflow.update(workflow1.id, {
  tags: ['data-processing', 'production']
});

await workflow.update(workflow2.id, {
  tags: ['content-generation', 'staging']
});

Grouping Workflows​

// Group workflows by category
const dataWorkflows = await workflow.list({ tags: ['data-processing'] });
const contentWorkflows = await workflow.list({ tags: ['content-generation'] });
const analysisWorkflows = await workflow.list({ tags: ['analysis'] });

Executing Multiple Workflows​

Sequential Execution​

// Execute workflows one after another
const result1 = await workflow.execute(workflow1.id, { input: data1 });
const result2 = await workflow.execute(workflow2.id, { input: data2 });
const result3 = await workflow.execute(workflow3.id, { input: data3 });

Parallel Execution​

// Execute multiple workflows in parallel
const results = await Promise.all([
  workflow.execute(workflow1.id, { input: data1 }),
  workflow.execute(workflow2.id, { input: data2 }),
  workflow.execute(workflow3.id, { input: data3 })
]);

console.log('All workflows completed:', results);

Conditional Execution​

// Execute workflows based on conditions
const data = await fetchData();

if (data.type === 'text') {
  await workflow.execute(workflow1.id, { input: data });
} else if (data.type === 'image') {
  await workflow.execute(workflow2.id, { input: data });
} else {
  await workflow.execute(workflow3.id, { input: data });
}

Workflow Dependencies​

Chaining Workflows​

// Execute workflows in sequence where output feeds into next
const result1 = await workflow.execute(workflow1.id, { input: initialData });
const result2 = await workflow.execute(workflow2.id, { input: result1.output });
const result3 = await workflow.execute(workflow3.id, { input: result2.output });

Workflow Pipelines​

// Create a pipeline of workflows
async function executePipeline(data) {
  const pipeline = [workflow1.id, workflow2.id, workflow3.id];
  let currentData = data;
  
  for (const workflowId of pipeline) {
    const result = await workflow.execute(workflowId, { input: currentData });
    currentData = result.output;
  }
  
  return currentData;
}

Monitoring Multiple Workflows​

Track All Executions​

// Monitor all workflow executions
const executions = await workflow.getExecutions({
  workflowIds: [workflow1.id, workflow2.id, workflow3.id]
});

executions.forEach(exec => {
  console.log(`Workflow ${exec.workflowId}: ${exec.status}`);
});

Real-time Monitoring​

// Set up real-time monitoring
workflow.on('execution:started', (execution) => {
  console.log(`Workflow ${execution.workflowId} started`);
});

workflow.on('execution:completed', (execution) => {
  console.log(`Workflow ${execution.workflowId} completed`);
});

workflow.on('execution:failed', (execution) => {
  console.error(`Workflow ${execution.workflowId} failed:`, execution.error);
});

Best Practices​

1. Naming Conventions​

Use consistent naming for workflows:

// Good naming
await workflow.create({ name: 'data-processing-production' });
await workflow.create({ name: 'data-processing-staging' });
await workflow.create({ name: 'content-generation-production' });

2. Resource Management​

Limit concurrent executions:

// Limit to 5 concurrent executions
const maxConcurrent = 5;
const workflowQueue = [workflow1, workflow2, workflow3, workflow4, workflow5];

for (let i = 0; i < workflowQueue.length; i += maxConcurrent) {
  const batch = workflowQueue.slice(i, i + maxConcurrent);
  await Promise.all(batch.map(w => workflow.execute(w.id, { input: data })));
}

3. Error Handling​

Handle errors for each workflow:

const workflows = [workflow1, workflow2, workflow3];

for (const wf of workflows) {
  try {
    await workflow.execute(wf.id, { input: data });
  } catch (error) {
    console.error(`Workflow ${wf.id} failed:`, error);
    // Continue with other workflows
  }
}

4. Cleanup​

Remove unused workflows:

// Delete old or unused workflows
const oldWorkflows = await workflow.list({
  updatedBefore: '2024-01-01'
});

for (const wf of oldWorkflows) {
  await workflow.delete(wf.id);
}

Example: Multi-Workflow System​

Here's a complete example of managing multiple workflows:

import { AIWorkflow } from 'ai-workflow';

const workflow = new AIWorkflow({
  apiKey: process.env.AI_WORKFLOW_API_KEY
});

// 1. Create workflows
const workflows = await Promise.all([
  workflow.create({ name: 'Data Ingestion', tags: ['ingestion'] }),
  workflow.create({ name: 'Data Processing', tags: ['processing'] }),
  workflow.create({ name: 'Data Analysis', tags: ['analysis'] }),
  workflow.create({ name: 'Report Generation', tags: ['reporting'] })
]);

// 2. Execute in pipeline
async function processData(data) {
  const results = [];
  let currentData = data;
  
  for (const wf of workflows) {
    const result = await workflow.execute(wf.id, { input: currentData });
    results.push(result);
    currentData = result.output;
  }
  
  return results;
}

// 3. Monitor executions
workflow.on('execution:completed', (execution) => {
  console.log(`✅ ${execution.workflowId} completed`);
});

// 4. Execute
const finalResults = await processData(initialData);

Next Steps​