# FlowDrop - Drupal Workflow Editor

_This project is under active development. Expect breaking changes every hour._


- For most part this README is acting like a wish list.
- It may get out of sync as various ideas are being tried on a daily basis.
- The project ultimately strives to solve the need for a UI similar to
  Langflow/n8n/Flowise.
- Since the AI ecosystem is moving at a pace which is hard to keep
  up with, I have decided to implement some custom functionality
  within the module that would be replaced with appropriate candidate from
  Drupal's ecosystem in future (but no promises)

A powerful visual workflow editor for Drupal that enables drag-and-drop creation of complex workflows with AI integration, data processing, and automation capabilities.

## 🚀 Features

- **Visual Workflow Editor**: Drag-and-drop interface for building complex workflows

## 📁 Project Structure

```
flowdrop/
├── src/                              # Core PHP classes
│   ├── Attribute/                    # Plugin discovery attributes
│   │   └── FlowDropNodeProcessor.php # Node processor attribute
│   ├── Controller/                   # Admin controllers
│   ├── DTO/                          # Data Transfer Objects
│   │   ├── ParameterBag.php          # Parameter handling
│   │   └── Output.php                # Output handling
│   ├── Hook/                         # Drupal hooks
│   ├── Plugin/FlowDropNodeProcessor/ # Base plugin classes
│   │   ├── AbstractFlowDropNodeProcessor.php
│   │   ├── FlowDropNodeProcessorInterface.php
│   │   └── NodeExecutorInterface.php
│   └── Service/                      # Core services
│       └── FlowDropNodeProcessorPluginManager.php
├── modules/                          # Drupal sub-modules
│   ├── flowdrop_workflow/            # Workflow entity management
│   ├── flowdrop_ui/                  # Svelte frontend application
│   ├── flowdrop_node_type/           # Node type definitions
│   ├── flowdrop_node_category/       # Node categorization
│   ├── flowdrop_node_processor/      # Node processor plugins
│   │   └── src/Plugin/FlowDropNodeProcessor/  # 25+ node processors
│   ├── flowdrop_pipeline/            # Pipeline management
│   ├── flowdrop_job/                 # Job execution system
│   ├── flowdrop_runtime/             # Workflow execution engine
│   └── flowdrop_trigger/             # Trigger system
├── js/                               # Drupal JavaScript integration
└── config/                           # Drupal configuration
```

## 🏗️ Architecture

### Core Components

1. **Plugin System**: Attribute-based discovery of node processors
2. **Node Runtime Service**: Executes individual workflow nodes
3. **Workflow Engine**: Drupal entity-based workflow management
4. **Pipeline Management**: Pipeline and job orchestration
5. **Runtime Orchestrators**: Synchronous and asynchronous execution engines
6. **Frontend**: Svelte-based visual editor
7. **API Layer**: RESTful API for workflow operations

### Node Categories

- **AI**: AI model integrations (Chat models, embeddings)
- **Agents**: Agent-based workflows
- **Bundles**: Grouped node configurations
- **Data**: Data manipulation and transformation
- **Embeddings**: Vector embeddings operations
- **Helpers**: Utility helper nodes
- **Inputs**: Input nodes (Text Input, etc.)
- **Logic**: Conditional nodes (If/Else, Switch Gateway, Boolean Gateway)
- **Memories**: Memory and context management
- **Models**: Model configurations
- **Outputs**: Output nodes (Text Output, Chat Output)
- **Processing**: Data processing operations
- **Prompts**: Prompt templates and management
- **Tools**: External tool integrations
- **Vector Stores**: Vector database operations

### Available Node Processors

| Category | Nodes |
|----------|-------|
| **Processing** | Calculator, DataframeOperations, DataOperations, DataExtractor, DataShaper, DataToDataframe, DataToJson, JsonToData, MessageToData |
| **AI/Models** | ChatModel, PromptTemplate |
| **Inputs** | TextInput |
| **Outputs** | TextOutput, ChatOutput |
| **Logic** | IfElse, AorB, BooleanGateway, SwitchGateway |
| **Helpers** | DateTime, RegexExtractor, SplitText, HttpRequest, Nop, Note |
| **Triggers** | AbstractTrigger |

## 🛠️ Installation

### Prerequisites

- Drupal 11
- PHP 8.3
- Node.js 18+ (for frontend development)

### Installation Steps

1. **Install the module**:
   ```bash
   # Copy to your Drupal modules directory
   cp -r flowdrop /path/to/drupal/web/modules/custom/
   ```

2. **Enable the module**:
   ```bash
   ddev drush en flowdrop flowdrop_workflow flowdrop_ui flowdrop_runtime
   ```

3. **Build the frontend**:
   ```bash
   cd modules/flowdrop_ui/app/flowdrop
   npm install
   npm run build:iife
   ```

4. **Clear Drupal cache**:
   ```bash
   ddev drush cr
   ```

## 🎯 Usage

### Accessing the Workflow Editor

1. Navigate to `/admin/structure/flowdrop-workflow`
2. Create a new workflow or edit an existing one
3. Use the visual editor to build your workflow

### Creating Workflows

1. **Add Nodes**: Drag nodes from the palette to the canvas
2. **Configure Nodes**: Set parameters and configuration for each node
3. **Connect Nodes**: Create connections between node inputs and outputs
4. **Test Execution**: Use the execution panel to test your workflow
5. **Save & Deploy**: Save your workflow for production use


## 🔧 Development

### Adding New Node Types

1. **Create Node Class** in `modules/flowdrop_node_processor/src/Plugin/FlowDropNodeProcessor/`:
   ```php
   <?php

   declare(strict_types=1);

   namespace Drupal\flowdrop_node_processor\Plugin\FlowDropNodeProcessor;

   use Drupal\Core\Logger\LoggerChannelInterface;
   use Drupal\Core\Logger\LoggerChannelFactoryInterface;
   use Drupal\flowdrop\Attribute\FlowDropNodeProcessor;
   use Drupal\flowdrop\Plugin\FlowDropNodeProcessor\AbstractFlowDropNodeProcessor;
   use Drupal\flowdrop\DTO\ParameterBagInterface;
   use Symfony\Component\DependencyInjection\ContainerInterface;

   /**
    * Custom node processor.
    */
   #[FlowDropNodeProcessor(
     id: "my_custom_node",
     label: new \Drupal\Core\StringTranslation\TranslatableMarkup("My Custom Node"),
     description: "Description of what this node does",
     version: "1.0.0"
   )]
   class MyCustomNode extends AbstractFlowDropNodeProcessor {

     public function __construct(
       array $configuration,
       $plugin_id,
       $plugin_definition,
       protected LoggerChannelFactoryInterface $loggerFactory,
     ) {
       parent::__construct($configuration, $plugin_id, $plugin_definition);
     }

     /**
      * {@inheritdoc}
      */
     public static function create(ContainerInterface $container, array $configuration, $plugin_id, $plugin_definition) {
       return new static(
         $configuration,
         $plugin_id,
         $plugin_definition,
         $container->get("logger.factory")
       );
     }

     /**
      * {@inheritdoc}
      */
     protected function getLogger(): LoggerChannelInterface {
       return $this->loggerFactory->get("flowdrop_node_processor");
     }

     /**
      * {@inheritdoc}
      */
     public function process(ParameterBagInterface $params): array {
       // Your node logic here
       $mySetting = $params->get("mySetting", "default");
       return ["result" => "processed data"];
     }

     /**
      * {@inheritdoc}
      */
     public function getParameterSchema(): array {
       return [
         "type" => "object",
         "properties" => [
           "mySetting" => [
             "type" => "string",
             "title" => "My Setting",
             "description" => "Description of this setting",
           ],
         ],
       ];
     }

     /**
      * {@inheritdoc}
      */
     public function getOutputSchema(): array {
       return [
         "type" => "object",
         "properties" => [
           "result" => [
             "type" => "string",
             "description" => "The processed result",
           ],
         ],
       ];
     }

     /**
      * {@inheritdoc}
      */
     public function validateParams(array $params): bool {
       // Custom validation logic
       return TRUE;
     }
   }
   ```

2. **Implement Required Methods**:
   - `process(ParameterBagInterface $params)`: Main node logic
   - `getParameterSchema()`: Unified parameter schema (inputs + config)
   - `getOutputSchema()`: Output schema
   - `validateParams()`: Parameter validation
   - `getLogger()`: Return logger channel

3. **Note**: Category, tags, and visual type are managed by the config entity, not by the plugin attribute.

### Frontend Development

The frontend is a Svelte application located in `modules/flowdrop_ui/app/flowdrop/`.

```bash
cd modules/flowdrop_ui/app/flowdrop
npm run dev          # Development server
npm run build:iife   # Build for Drupal integration
npm run test         # Run tests
```

## 📚 API Documentation

### REST API Endpoints

#### Workflows
| Method | Endpoint | Description | Status |
|--------|----------|-------------|--------|
| GET | `/api/flowdrop/workflows` | List workflows | ✅ Implemented |
| POST | `/api/flowdrop/workflows` | Create workflow | ✅ Implemented |
| GET | `/api/flowdrop/workflows/{id}` | Get workflow | ✅ Implemented |
| PUT | `/api/flowdrop/workflows/{id}` | Update workflow | ✅ Implemented |
| DELETE | `/api/flowdrop/workflows/{id}` | Delete workflow | ✅ Implemented |
| POST | `/api/flowdrop/workflows/{id}/execute` | Execute workflow | ⏳ Pending |

#### Nodes
| Method | Endpoint | Description | Status |
|--------|----------|-------------|--------|
| GET | `/api/flowdrop/nodes` | List available nodes | ✅ Implemented |
| GET | `/api/flowdrop/nodes/{category}` | Filter by category | ✅ Implemented |
| GET | `/api/flowdrop/nodes/{plugin_id}/metadata` | Get node metadata | ✅ Implemented |
| POST | `/api/flowdrop/nodes/{plugin_id}/validate` | Validate configuration | ✅ Implemented |
| GET | `/api/flowdrop/port-config` | Get port configuration | ✅ Implemented |

#### Pipelines
| Method | Endpoint | Description | Status |
|--------|----------|-------------|--------|
| GET | `/api/flowdrop/workflow/{workflow_id}/pipelines` | Get workflow pipelines | ✅ Implemented |
| GET | `/api/flowdrop/pipeline/{pipeline_id}` | Get pipeline | ✅ Implemented |
| GET | `/api/flowdrop/pipeline/{pipeline_id}/status` | Get pipeline status | ✅ Implemented |
| GET | `/api/flowdrop/pipeline/{pipeline_id}/logs` | Get pipeline logs | ✅ Implemented |

#### Runtime
| Method | Endpoint | Description | Status |
|--------|----------|-------------|--------|
| GET | `/api/flowdrop-runtime/status/{executionId}` | Get execution status | ✅ Implemented |
| GET | `/api/flowdrop-runtime/status/{executionId}/nodes` | Get node statuses | ✅ Implemented |
| GET | `/api/flowdrop-runtime/status/{executionId}/details` | Get execution details | ✅ Implemented |
| GET | `/api/flowdrop-runtime/health` | Health check | ✅ Implemented |
| GET | `/api/flowdrop-runtime/metrics` | System metrics | ✅ Implemented |

#### Executions (Pending Implementation)
| Method | Endpoint | Description | Status |
|--------|----------|-------------|--------|
| GET | `/api/flowdrop/executions/active` | Active executions | ⏳ Pending |
| GET | `/api/flowdrop/workflows/{id}/executions/{execution_id}/state` | Execution status | ⏳ Pending |

### Node Configuration Schema

Each node type defines its configuration schema:

```json
{
  "type": "object",
  "properties": {
    "setting1": {
      "type": "string",
      "title": "Setting 1",
      "description": "Description of setting"
    },
    "setting2": {
      "type": "number",
      "default": 0
    }
  }
}
```

## 📝 Logging

Workflow execution is logged through Drupal's logging system:

```php
$this->loggerFactory->get("flowdrop")->info("Workflow executed", [
  "workflow_id" => $workflowId,
  "execution_time" => $executionTime,
]);
```

## 🧪 Testing

Run tests:

```bash
# PHPUnit tests
vendor/bin/phpunit modules/flowdrop_runtime/tests/

# Drupal test runner
drush phpunit modules/flowdrop_runtime/tests/Integration/
```

## 🤝 Contributing

Not accepting Contribution until the module stabilizes. Stay tuned.
