Compare commits

...

5 Commits

Author SHA1 Message Date
CodingOnStar
47724ec764 feat: enhance E2E testing setup with Cloudflare Access support and improved authentication handling
- Added support for Cloudflare Access headers in Playwright configuration and teardown.
- Updated global setup for E2E tests to validate authentication credentials and handle login more robustly.
- Enhanced README with authentication configuration details and supported methods.
- Updated Playwright reporter configuration to include JSON output for test results.
2025-12-16 14:15:09 +08:00
CodingOnStar
3863894072 Merge remote-tracking branch 'origin/main' into feat/e2e-testing 2025-12-16 10:04:19 +08:00
CodingOnStar
cf20e9fd38 Merge remote-tracking branch 'origin/main' into feat/e2e-testing 2025-12-11 15:47:59 +08:00
CodingOnStar
a8a0f2c900 Merge remote-tracking branch 'origin/main' into feat/e2e-testing 2025-12-10 14:44:51 +08:00
CodingOnStar
7b968c6c2e feat: add Playwright E2E testing framework and initial test setup
- Introduced Playwright for end-to-end testing, including configuration in .
- Created global setup and teardown scripts for authentication and cleanup.
- Added page object models for key application pages (Apps, SignIn, Workflow).
- Implemented utility functions for API interactions and common test helpers.
- Updated  to exclude Playwright test results and auth files.
- Added initial E2E tests for the Apps page.
- Updated  with new test scripts for E2E testing.
2025-12-09 18:14:57 +08:00
17 changed files with 2150 additions and 2 deletions

7
web/.gitignore vendored
View File

@@ -8,6 +8,13 @@
# testing
/coverage
# playwright e2e
/e2e/.auth/
/e2e/test-results/
/playwright-report/
/blob-report/
/test-results/
# next.js
/.next/
/out/

324
web/e2e/README.md Normal file
View File

@@ -0,0 +1,324 @@
# E2E Testing Guide
This directory contains End-to-End (E2E) tests for the Dify web application using [Playwright](https://playwright.dev/).
## Quick Start
### 1. Setup
```bash
# Install dependencies (if not already done)
pnpm install
# Install Playwright browsers
pnpm exec playwright install chromium
```
### 2. Configure Environment (Optional)
Add E2E test configuration to your `web/.env.local` file:
```env
# E2E Test Configuration
# Base URL of the frontend (optional, defaults to http://localhost:3000)
E2E_BASE_URL=https://test.example.com
# Skip starting dev server (use existing deployed server)
E2E_SKIP_WEB_SERVER=true
# API URL (optional, defaults to http://localhost:5001/console/api)
NEXT_PUBLIC_API_PREFIX=http://localhost:5001/console/api
# Authentication Configuration
# Test user credentials
NEXT_PUBLIC_E2E_USER_EMAIL=test@example.com
NEXT_PUBLIC_E2E_USER_PASSWORD=your-password
```
### Authentication Methods
Dify supports multiple login methods, but not all are suitable for E2E testing:
| Method | E2E Support | Configuration |
|--------|-------------|---------------|
| **Email + Password** | ✅ Recommended | Set `NEXT_PUBLIC_E2E_USER_EMAIL` and `NEXT_PUBLIC_E2E_USER_PASSWORD` |
#### Email + Password (Default)
The most reliable method for E2E testing. Simply set the credentials:
```env
NEXT_PUBLIC_E2E_USER_EMAIL=test@example.com
NEXT_PUBLIC_E2E_USER_PASSWORD=your-password
```
### 3. Run Tests
```bash
# Run all E2E tests
pnpm test:e2e
# Run tests with UI (interactive mode)
pnpm test:e2e:ui
# Run tests with browser visible
pnpm test:e2e:headed
# Run tests in debug mode
pnpm test:e2e:debug
# View test report
pnpm test:e2e:report
```
## Project Structure
```
web/
├── .env.local # Environment config (includes E2E variables)
├── playwright.config.ts # Playwright configuration
└── e2e/
├── fixtures/ # Test fixtures (extended test objects)
│ └── index.ts # Main fixtures with page objects
├── pages/ # Page Object Models (POM)
│ ├── base.page.ts # Base class for all page objects
│ ├── signin.page.ts # Sign-in page interactions
│ ├── apps.page.ts # Apps listing page interactions
│ ├── workflow.page.ts # Workflow editor interactions
│ └── index.ts # Page objects export
├── tests/ # Test files (*.spec.ts)
├── utils/ # Test utilities
│ ├── index.ts # Utils export
│ ├── test-helpers.ts # Common helper functions
│ └── api-helpers.ts # API-level test helpers
├── .auth/ # Authentication state (gitignored)
├── global.setup.ts # Authentication setup
├── global.teardown.ts # Cleanup after tests
└── README.md # This file
```
## Writing Tests
### Using Page Objects
```typescript
import { test, expect } from '../fixtures'
test('create a new app', async ({ appsPage }) => {
await appsPage.goto()
await appsPage.createApp({
name: 'My Test App',
type: 'chatbot',
})
await appsPage.expectAppExists('My Test App')
})
```
### Using Test Helpers
```typescript
import { test, expect } from '../fixtures'
import { generateTestId, waitForNetworkIdle } from '../utils/test-helpers'
test('search functionality', async ({ appsPage }) => {
const uniqueName = generateTestId('app')
// ... test logic
})
```
### Test Data Cleanup
Always clean up test data to avoid polluting the database:
```typescript
test('create and delete app', async ({ appsPage }) => {
const appName = generateTestId('test-app')
// Create
await appsPage.createApp({ name: appName, type: 'chatbot' })
// Test assertions
await appsPage.expectAppExists(appName)
// Cleanup
await appsPage.deleteApp(appName)
})
```
### Skipping Authentication
For tests that need to verify unauthenticated behavior:
```typescript
test.describe('unauthenticated tests', () => {
test.use({ storageState: { cookies: [], origins: [] } })
test('redirects to login', async ({ page }) => {
await page.goto('/apps')
await expect(page).toHaveURL(/\/signin/)
})
})
```
## Best Practices
### 1. Use Page Object Model (POM)
- Encapsulate page interactions in page objects
- Makes tests more readable and maintainable
- Changes to selectors only need to be updated in one place
### 2. Use Meaningful Test Names
```typescript
// Good
test('should display error message for invalid email format', ...)
// Bad
test('test1', ...)
```
### 3. Use Data-TestId Attributes
When adding elements to the application, use `data-testid` attributes:
```tsx
// In React component
<button data-testid="create-app-button">Create App</button>
// In test
await page.getByTestId('create-app-button').click()
```
### 4. Generate Unique Test Data
```typescript
import { generateTestId } from '../utils/test-helpers'
const appName = generateTestId('my-app') // e.g., "my-app-1732567890123-abc123"
```
### 5. Handle Async Operations
```typescript
// Wait for element
await expect(element).toBeVisible({ timeout: 10000 })
// Wait for navigation
await page.waitForURL(/\/apps/)
// Wait for network
await page.waitForLoadState('networkidle')
```
## Creating New Page Objects
1. Create a new file in `e2e/pages/`:
```typescript
// e2e/pages/my-feature.page.ts
import type { Page, Locator } from '@playwright/test'
import { BasePage } from './base.page'
export class MyFeaturePage extends BasePage {
readonly myElement: Locator
constructor(page: Page) {
super(page)
this.myElement = page.getByTestId('my-element')
}
get path(): string {
return '/my-feature'
}
async doSomething(): Promise<void> {
await this.myElement.click()
}
}
```
2. Export from `e2e/pages/index.ts`:
```typescript
export { MyFeaturePage } from './my-feature.page'
```
3. Add to fixtures in `e2e/fixtures/index.ts`:
```typescript
import { MyFeaturePage } from '../pages/my-feature.page'
type DifyFixtures = {
// ... existing fixtures
myFeaturePage: MyFeaturePage
}
export const test = base.extend<DifyFixtures>({
// ... existing fixtures
myFeaturePage: async ({ page }, use) => {
await use(new MyFeaturePage(page))
},
})
```
## Debugging
### Visual Debugging
```bash
# Open Playwright UI
pnpm test:e2e:ui
# Run with visible browser
pnpm test:e2e:headed
# Debug mode with inspector
pnpm test:e2e:debug
```
### Traces and Screenshots
Failed tests automatically capture:
- Screenshots
- Video recordings
- Trace files
View them:
```bash
pnpm test:e2e:report
```
### Manual Trace Viewing
```bash
pnpm exec playwright show-trace e2e/test-results/path-to-trace.zip
```
## Troubleshooting
### Tests timeout waiting for elements
1. Check if selectors are correct
2. Increase timeout: `{ timeout: 30000 }`
3. Add explicit waits: `await page.waitForSelector(...)`
### Authentication issues
1. Make sure global.setup.ts has completed successfully
2. For deployed environments, ensure E2E_BASE_URL matches your cookie domain
3. Clear auth state: `rm -rf e2e/.auth/`
### Flaky tests
1. Add explicit waits for async operations
2. Use `test.slow()` for inherently slow tests
3. Add retry logic for unstable operations
## Resources
- [Playwright Documentation](https://playwright.dev/docs/intro)
- [Page Object Model Pattern](https://playwright.dev/docs/pom)
- [Best Practices](https://playwright.dev/docs/best-practices)
- [Debugging Guide](https://playwright.dev/docs/debug)

55
web/e2e/fixtures/index.ts Normal file
View File

@@ -0,0 +1,55 @@
import { test as base, expect } from '@playwright/test'
import { AppsPage } from '../pages/apps.page'
import { SignInPage } from '../pages/signin.page'
import { WorkflowPage } from '../pages/workflow.page'
/**
* Extended test fixtures for Dify E2E tests
*
* This module provides custom fixtures that inject page objects
* into tests, making it easier to write maintainable tests.
*
* @example
* ```typescript
* import { test, expect } from '@/e2e/fixtures'
*
* test('can create new app', async ({ appsPage }) => {
* await appsPage.goto()
* await appsPage.createApp('My Test App')
* await expect(appsPage.appCard('My Test App')).toBeVisible()
* })
* ```
*/
// Define custom fixtures type
type DifyFixtures = {
appsPage: AppsPage
signInPage: SignInPage
workflowPage: WorkflowPage
}
/**
* Extended test object with Dify-specific fixtures
*/
export const test = base.extend<DifyFixtures>({
// Apps page fixture
appsPage: async ({ page }, run) => {
const appsPage = new AppsPage(page)
await run(appsPage)
},
// Sign in page fixture
signInPage: async ({ page }, run) => {
const signInPage = new SignInPage(page)
await run(signInPage)
},
// Workflow page fixture
workflowPage: async ({ page }, run) => {
const workflowPage = new WorkflowPage(page)
await run(workflowPage)
},
})
// Re-export expect for convenience
export { expect }

127
web/e2e/global.setup.ts Normal file
View File

@@ -0,0 +1,127 @@
import { expect, test as setup } from '@playwright/test'
import fs from 'node:fs'
import path from 'node:path'
const authFile = path.join(__dirname, '.auth/user.json')
/**
* Supported authentication methods for E2E tests
* - password: Email + Password login (default, recommended)
*
* OAuth (GitHub/Google) and SSO are not supported in E2E tests
* as they require third-party authentication which cannot be reliably automated.
*/
/**
* Global setup for E2E tests
*
* This runs before all tests and handles authentication.
* The authenticated state is saved and reused across all tests.
*
* Environment variables:
* - NEXT_PUBLIC_E2E_USER_EMAIL: Test user email (required)
* - NEXT_PUBLIC_E2E_USER_PASSWORD: Test user password (required for 'password' method)
*/
setup('authenticate', async ({ page }) => {
const email = process.env.NEXT_PUBLIC_E2E_USER_EMAIL
const password = process.env.NEXT_PUBLIC_E2E_USER_PASSWORD
// Validate required credentials based on auth method
if (!email) {
console.warn(
'⚠️ NEXT_PUBLIC_E2E_USER_EMAIL not set.',
'Creating empty auth state. Tests requiring auth will fail.',
)
await saveEmptyAuthState(page)
return
}
if (!password) {
console.warn(
'⚠️ NEXT_PUBLIC_E2E_USER_PASSWORD not set for password auth method.',
'Creating empty auth state. Tests requiring auth will fail.',
)
await saveEmptyAuthState(page)
return
}
// Navigate to login page
await page.goto('/signin')
await page.waitForLoadState('networkidle')
// Execute login
await loginWithPassword(page, email, password!)
// Wait for successful redirect to /apps
await expect(page).toHaveURL(/\/apps/, { timeout: 30000 })
// Save authenticated state
await page.context().storageState({ path: authFile })
console.log('✅ Authentication successful, state saved.')
})
/**
* Save empty auth state when credentials are not available
*/
async function saveEmptyAuthState(page: import('@playwright/test').Page): Promise<void> {
const authDir = path.dirname(authFile)
if (!fs.existsSync(authDir))
fs.mkdirSync(authDir, { recursive: true })
await page.context().storageState({ path: authFile })
}
/**
* Login using email and password
* Based on: web/app/signin/components/mail-and-password-auth.tsx
*/
async function loginWithPassword(
page: import('@playwright/test').Page,
email: string,
password: string,
): Promise<void> {
console.log('📧 Logging in with email and password...')
// Fill in login form
// Email input has id="email"
await page.locator('#email').fill(email)
// Password input has id="password"
await page.locator('#password').fill(password)
// Wait for button to be enabled (form validation passes)
const signInButton = page.getByRole('button', { name: /sign in/i })
await expect(signInButton).toBeEnabled({ timeout: 5000 })
// Click login button and wait for navigation or API response
// The app uses ky library which follows redirects automatically
// Some environments may have WAF/CDN that adds extra redirects
// So we use a more flexible approach: wait for either URL change or API response
const responsePromise = page.waitForResponse(
resp => resp.url().includes('login') && resp.request().method() === 'POST',
{ timeout: 15000 },
).catch(() => null) // Don't fail if we can't catch the response
await signInButton.click()
// Try to get the response, but don't fail if we can't
const response = await responsePromise
if (response) {
const status = response.status()
console.log(`📡 Login API response status: ${status}`)
// 200 = success, 302 = redirect (some WAF/CDN setups)
if (status !== 200 && status !== 302) {
// Try to get error details
try {
const body = await response.json()
console.error('❌ Login failed:', body)
}
catch {
console.error(`❌ Login failed with status ${status}`)
}
}
}
else {
console.log('⚠️ Could not capture login API response, will verify via URL redirect')
}
console.log('✅ Password login request sent')
}

214
web/e2e/global.teardown.ts Normal file
View File

@@ -0,0 +1,214 @@
import { request, test as teardown } from '@playwright/test'
/**
* Global teardown for E2E tests
*
* This runs after all tests complete.
* Cleans up test data created during E2E tests.
*
* Environment variables:
* - NEXT_PUBLIC_API_PREFIX: API URL (default: http://localhost:5001/console/api)
*
* Based on Dify API:
* - GET /apps - list all apps
* - DELETE /apps/{id} - delete an app
* - GET /datasets - list all datasets
* - DELETE /datasets/{id} - delete a dataset
*/
// API base URL with fallback for local development
// Ensure baseURL ends with '/' for proper path concatenation
const API_BASE_URL = (process.env.NEXT_PUBLIC_API_PREFIX || 'http://localhost:5001/console/api').replace(/\/?$/, '/')
// Cloudflare Access headers (for protected environments).
// Prefer environment variables to avoid hardcoding secrets in repo.
const CF_ACCESS_CLIENT_ID = process.env.CF_ACCESS_CLIENT_ID
const CF_ACCESS_CLIENT_SECRET = process.env.CF_ACCESS_CLIENT_SECRET
const cfAccessHeaders: Record<string, string> = {}
if (CF_ACCESS_CLIENT_ID && CF_ACCESS_CLIENT_SECRET) {
cfAccessHeaders['CF-Access-Client-Id'] = CF_ACCESS_CLIENT_ID
cfAccessHeaders['CF-Access-Client-Secret'] = CF_ACCESS_CLIENT_SECRET
}
// Test data prefixes - used to identify test-created data
// Should match the prefix used in generateTestId()
const TEST_DATA_PREFIXES = ['e2e-', 'test-']
/**
* Check if a name matches test data pattern
*/
function isTestData(name: string): boolean {
return TEST_DATA_PREFIXES.some(prefix => name.toLowerCase().startsWith(prefix))
}
/**
* Delete a single app by ID
*/
async function deleteApp(
context: Awaited<ReturnType<typeof request.newContext>>,
app: { id: string, name: string },
): Promise<boolean> {
try {
const response = await context.delete(`apps/${app.id}`)
return response.ok()
}
catch {
console.warn(` Failed to delete app "${app.name}"`)
return false
}
}
/**
* Delete a single dataset by ID
*/
async function deleteDataset(
context: Awaited<ReturnType<typeof request.newContext>>,
dataset: { id: string, name: string },
): Promise<boolean> {
try {
const response = await context.delete(`datasets/${dataset.id}`)
return response.ok()
}
catch {
console.warn(` Failed to delete dataset "${dataset.name}"`)
return false
}
}
teardown('cleanup test data', async () => {
console.log('🧹 Starting global teardown...')
const fs = await import('node:fs')
const authPath = 'e2e/.auth/user.json'
// Check if auth state file exists and has cookies
if (!fs.existsSync(authPath)) {
console.warn('⚠️ Auth state file not found, skipping cleanup')
console.log('🧹 Global teardown complete.')
return
}
let csrfToken = ''
try {
const authState = JSON.parse(fs.readFileSync(authPath, 'utf-8'))
if (!authState.cookies || authState.cookies.length === 0) {
console.warn('⚠️ Auth state is empty (no cookies), skipping cleanup')
console.log('🧹 Global teardown complete.')
return
}
// Extract CSRF token from cookies for API requests
// Cookie name may be 'csrf_token' or '__Host-csrf_token' depending on environment
const csrfCookie = authState.cookies.find((c: { name: string }) =>
c.name === 'csrf_token' || c.name === '__Host-csrf_token',
)
csrfToken = csrfCookie?.value || ''
}
catch {
console.warn('⚠️ Failed to read auth state, skipping cleanup')
console.log('🧹 Global teardown complete.')
return
}
try {
// Create API request context with auth state and CSRF header
const context = await request.newContext({
baseURL: API_BASE_URL,
storageState: authPath,
extraHTTPHeaders: {
'X-CSRF-Token': csrfToken,
...cfAccessHeaders,
},
})
// Clean up test apps
const appsDeleted = await cleanupTestApps(context)
console.log(` 📱 Deleted ${appsDeleted} test apps`)
// Clean up test datasets
const datasetsDeleted = await cleanupTestDatasets(context)
console.log(` 📚 Deleted ${datasetsDeleted} test datasets`)
await context.dispose()
}
catch (error) {
// Don't fail teardown if cleanup fails - just log the error
console.warn('⚠️ Teardown cleanup encountered errors:', error)
}
// Clean up auth state file in CI environment for security
// In local development, keep it for faster iteration (skip re-login)
if (process.env.CI) {
try {
fs.unlinkSync(authPath)
console.log(' 🔐 Auth state file deleted (CI mode)')
}
catch {
// Ignore if file doesn't exist or can't be deleted
}
}
console.log('🧹 Global teardown complete.')
})
/**
* Clean up test apps
* Deletes all apps with names starting with test prefixes
*/
async function cleanupTestApps(context: Awaited<ReturnType<typeof request.newContext>>): Promise<number> {
try {
// Fetch all apps - API: GET /apps
const response = await context.get('apps', {
params: { page: 1, limit: 100 },
})
if (!response.ok()) {
console.warn(' Failed to fetch apps list:', response.status(), response.url())
return 0
}
const data = await response.json()
const apps: Array<{ id: string, name: string }> = data.data || []
// Filter test apps and delete them
const testApps = apps.filter(app => isTestData(app.name))
const results = await Promise.all(testApps.map(app => deleteApp(context, app)))
return results.filter(Boolean).length
}
catch (error) {
console.warn(' Error cleaning up apps:', error)
return 0
}
}
/**
* Clean up test datasets (knowledge bases)
* Deletes all datasets with names starting with test prefixes
*/
async function cleanupTestDatasets(context: Awaited<ReturnType<typeof request.newContext>>): Promise<number> {
try {
// Fetch all datasets - API: GET /datasets
const response = await context.get('datasets', {
params: { page: 1, limit: 100 },
})
if (!response.ok()) {
console.warn(' Failed to fetch datasets list:', response.status(), response.url())
return 0
}
const data = await response.json()
const datasets: Array<{ id: string, name: string }> = data.data || []
// Filter test datasets and delete them
const testDatasets = datasets.filter(dataset => isTestData(dataset.name))
const results = await Promise.all(testDatasets.map(dataset => deleteDataset(context, dataset)))
return results.filter(Boolean).length
}
catch (error) {
console.warn(' Error cleaning up datasets:', error)
return 0
}
}

243
web/e2e/pages/apps.page.ts Normal file
View File

@@ -0,0 +1,243 @@
import type { Locator, Page } from '@playwright/test'
import { expect } from '@playwright/test'
import { BasePage } from './base.page'
/**
* Apps (Studio) Page Object Model
*
* Handles interactions with the main apps listing page.
* Based on: web/app/components/apps/list.tsx
* web/app/components/apps/new-app-card.tsx
* web/app/components/apps/app-card.tsx
*/
export class AppsPage extends BasePage {
// Main page elements
readonly createFromBlankButton: Locator
readonly createFromTemplateButton: Locator
readonly importDSLButton: Locator
readonly searchInput: Locator
readonly appGrid: Locator
// Create app modal elements (from create-app-modal/index.tsx)
readonly createAppModal: Locator
readonly appNameInput: Locator
readonly appDescriptionInput: Locator
readonly createButton: Locator
readonly cancelButton: Locator
// App type selectors in create modal
readonly chatbotType: Locator
readonly completionType: Locator
readonly workflowType: Locator
readonly agentType: Locator
readonly chatflowType: Locator
// Delete confirmation
readonly deleteConfirmButton: Locator
constructor(page: Page) {
super(page)
// Create app card buttons (from new-app-card.tsx)
// t('app.newApp.startFromBlank') = "Create from Blank"
this.createFromBlankButton = page.getByRole('button', { name: 'Create from Blank' })
// t('app.newApp.startFromTemplate') = "Create from Template"
this.createFromTemplateButton = page.getByRole('button', { name: 'Create from Template' })
// t('app.importDSL') = "Import DSL file"
this.importDSLButton = page.getByRole('button', { name: /Import DSL/i })
// Search input (from list.tsx)
this.searchInput = page.getByPlaceholder(/search/i)
// App grid container
this.appGrid = page.locator('.grid').first()
// Create app modal
this.createAppModal = page.locator('[class*="fullscreen-modal"]').or(page.getByRole('dialog'))
// App name input - placeholder: t('app.newApp.appNamePlaceholder') = "Give your app a name"
this.appNameInput = page.getByPlaceholder('Give your app a name')
// Description input - placeholder: t('app.newApp.appDescriptionPlaceholder') = "Enter the description of the app"
this.appDescriptionInput = page.getByPlaceholder('Enter the description of the app')
// Create button - t('app.newApp.Create') = "Create"
this.createButton = page.getByRole('button', { name: 'Create', exact: true })
this.cancelButton = page.getByRole('button', { name: 'Cancel' })
// App type selectors (from create-app-modal)
// These are displayed as clickable cards/buttons
this.chatbotType = page.getByText('Chatbot', { exact: true })
this.completionType = page.getByText('Completion', { exact: true }).or(page.getByText('Text Generator'))
this.workflowType = page.getByText('Workflow', { exact: true })
this.agentType = page.getByText('Agent', { exact: true })
this.chatflowType = page.getByText('Chatflow', { exact: true })
// Delete confirmation button
this.deleteConfirmButton = page.getByRole('button', { name: /confirm|delete/i }).last()
}
get path(): string {
return '/apps'
}
/**
* Get app card by name
* App cards use AppIcon and display the app name
*/
appCard(name: string): Locator {
return this.appGrid.locator(`div:has-text("${name}")`).first()
}
/**
* Get app card's more menu button (three dots)
*/
appCardMenu(name: string): Locator {
return this.appCard(name).locator('svg[class*="ri-more"]').or(
this.appCard(name).locator('button:has(svg)').last(),
)
}
/**
* Click "Create from Blank" button
*/
async clickCreateFromBlank(): Promise<void> {
await this.createFromBlankButton.click()
await expect(this.createAppModal).toBeVisible({ timeout: 10000 })
}
/**
* Click "Create from Template" button
*/
async clickCreateFromTemplate(): Promise<void> {
await this.createFromTemplateButton.click()
}
/**
* Select app type in create modal
*/
async selectAppType(type: 'chatbot' | 'completion' | 'workflow' | 'agent' | 'chatflow'): Promise<void> {
const typeMap: Record<string, Locator> = {
chatbot: this.chatbotType,
completion: this.completionType,
workflow: this.workflowType,
agent: this.agentType,
chatflow: this.chatflowType,
}
await typeMap[type].click()
}
/**
* Fill app name
*/
async fillAppName(name: string): Promise<void> {
await this.appNameInput.fill(name)
}
/**
* Fill app description
*/
async fillAppDescription(description: string): Promise<void> {
await this.appDescriptionInput.fill(description)
}
/**
* Confirm app creation
*/
async confirmCreate(): Promise<void> {
await this.createButton.click()
}
/**
* Create a new app with full flow
*/
async createApp(options: {
name: string
type?: 'chatbot' | 'completion' | 'workflow' | 'agent' | 'chatflow'
description?: string
}): Promise<void> {
const { name, type = 'chatbot', description } = options
await this.clickCreateFromBlank()
await this.selectAppType(type)
await this.fillAppName(name)
if (description)
await this.fillAppDescription(description)
await this.confirmCreate()
// Wait for navigation to new app or modal to close
await this.page.waitForURL(/\/app\//, { timeout: 30000 })
}
/**
* Search for an app
*/
async searchApp(query: string): Promise<void> {
await this.searchInput.fill(query)
await this.page.waitForTimeout(500) // Debounce
}
/**
* Open an app by clicking its card
*/
async openApp(name: string): Promise<void> {
await this.appCard(name).click()
await this.waitForNavigation()
}
/**
* Delete an app by name
*/
async deleteApp(name: string): Promise<void> {
// Hover on app card to show menu
await this.appCard(name).hover()
// Click more menu (three dots icon)
await this.appCardMenu(name).click()
// Click delete in menu
// t('common.operation.delete') = "Delete"
await this.page.getByRole('menuitem', { name: 'Delete' })
.or(this.page.getByText('Delete').last())
.click()
// Confirm deletion
await this.deleteConfirmButton.click()
// Wait for app to be removed
await expect(this.appCard(name)).toBeHidden({ timeout: 10000 })
}
/**
* Get count of visible apps
*/
async getAppCount(): Promise<number> {
// Each app card has the app icon and name
return this.appGrid.locator('[class*="app-card"], [class*="rounded-xl"]').count()
}
/**
* Check if apps list is empty
*/
async isEmpty(): Promise<boolean> {
// Empty state component is shown when no apps
const emptyState = this.page.locator('[class*="empty"]')
return emptyState.isVisible()
}
/**
* Verify app exists
*/
async expectAppExists(name: string): Promise<void> {
await expect(this.page.getByText(name).first()).toBeVisible({ timeout: 10000 })
}
/**
* Verify app does not exist
*/
async expectAppNotExists(name: string): Promise<void> {
await expect(this.page.getByText(name).first()).toBeHidden({ timeout: 10000 })
}
}

144
web/e2e/pages/base.page.ts Normal file
View File

@@ -0,0 +1,144 @@
import type { Locator, Page } from '@playwright/test'
import { expect } from '@playwright/test'
/**
* Base Page Object Model class
*
* All page objects should extend this class.
* Provides common functionality and patterns for page objects.
*/
export abstract class BasePage {
readonly page: Page
// Common elements that exist across multiple pages
protected readonly loadingSpinner: Locator
constructor(page: Page) {
this.page = page
// Loading spinner - based on web/app/components/base/loading/index.tsx
// Uses SVG with .spin-animation class
this.loadingSpinner = page.locator('.spin-animation')
}
/**
* Abstract method - each page must define its URL path
*/
abstract get path(): string
/**
* Navigate to this page
*/
async goto(): Promise<void> {
await this.page.goto(this.path)
await this.waitForPageLoad()
}
/**
* Wait for page to finish loading
*/
async waitForPageLoad(): Promise<void> {
await this.page.waitForLoadState('networkidle')
// Wait for any loading spinners to disappear
if (await this.loadingSpinner.isVisible())
await this.loadingSpinner.waitFor({ state: 'hidden', timeout: 30000 })
}
/**
* Check if page is currently visible
*/
async isVisible(): Promise<boolean> {
return this.page.url().includes(this.path)
}
/**
* Wait for and verify a toast notification
* Toast text is in .system-sm-semibold class
*/
async expectToast(text: string | RegExp): Promise<void> {
const toast = this.page.locator('.system-sm-semibold').filter({ hasText: text })
await expect(toast).toBeVisible({ timeout: 10000 })
}
/**
* Wait for a successful operation toast
* Success toast has bg-toast-success-bg background and RiCheckboxCircleFill icon
*/
async expectSuccessToast(text?: string | RegExp): Promise<void> {
// Success toast contains .text-text-success class (green checkmark icon)
const successIndicator = this.page.locator('.text-text-success')
await expect(successIndicator).toBeVisible({ timeout: 10000 })
if (text) {
const toastText = this.page.locator('.system-sm-semibold').filter({ hasText: text })
await expect(toastText).toBeVisible({ timeout: 10000 })
}
}
/**
* Wait for an error toast
* Error toast has bg-toast-error-bg background and RiErrorWarningFill icon
*/
async expectErrorToast(text?: string | RegExp): Promise<void> {
// Error toast contains .text-text-destructive class (red warning icon)
const errorIndicator = this.page.locator('.text-text-destructive')
await expect(errorIndicator).toBeVisible({ timeout: 10000 })
if (text) {
const toastText = this.page.locator('.system-sm-semibold').filter({ hasText: text })
await expect(toastText).toBeVisible({ timeout: 10000 })
}
}
/**
* Wait for a warning toast
* Warning toast has bg-toast-warning-bg background
*/
async expectWarningToast(text?: string | RegExp): Promise<void> {
const warningIndicator = this.page.locator('.text-text-warning-secondary')
await expect(warningIndicator).toBeVisible({ timeout: 10000 })
if (text) {
const toastText = this.page.locator('.system-sm-semibold').filter({ hasText: text })
await expect(toastText).toBeVisible({ timeout: 10000 })
}
}
/**
* Get the current page title
*/
async getTitle(): Promise<string> {
return this.page.title()
}
/**
* Take a screenshot of the current page
*/
async screenshot(name: string): Promise<void> {
await this.page.screenshot({
path: `e2e/test-results/screenshots/${name}.png`,
fullPage: true,
})
}
/**
* Wait for navigation to complete
*/
async waitForNavigation(options?: { timeout?: number }): Promise<void> {
await this.page.waitForLoadState('networkidle', options)
}
/**
* Press keyboard shortcut
*/
async pressShortcut(shortcut: string): Promise<void> {
await this.page.keyboard.press(shortcut)
}
/**
* Get element by test id (data-testid attribute)
*/
getByTestId(testId: string): Locator {
return this.page.getByTestId(testId)
}
}

10
web/e2e/pages/index.ts Normal file
View File

@@ -0,0 +1,10 @@
/**
* Page Object Models Index
*
* Export all page objects from a single entry point.
*/
export { BasePage } from './base.page'
export { SignInPage } from './signin.page'
export { AppsPage } from './apps.page'
export { WorkflowPage } from './workflow.page'

View File

@@ -0,0 +1,112 @@
import type { Locator, Page } from '@playwright/test'
import { expect } from '@playwright/test'
import { BasePage } from './base.page'
/**
* Sign In Page Object Model
*
* Handles all interactions with the login/sign-in page.
* Based on: web/app/signin/components/mail-and-password-auth.tsx
*/
export class SignInPage extends BasePage {
readonly emailInput: Locator
readonly passwordInput: Locator
readonly signInButton: Locator
readonly forgotPasswordLink: Locator
readonly errorMessage: Locator
constructor(page: Page) {
super(page)
// Selectors based on actual signin page
// See: web/app/signin/components/mail-and-password-auth.tsx
this.emailInput = page.locator('#email') // id="email"
this.passwordInput = page.locator('#password') // id="password"
this.signInButton = page.getByRole('button', { name: 'Sign in' }) // t('login.signBtn')
this.forgotPasswordLink = page.getByRole('link', { name: /forgot/i })
this.errorMessage = page.locator('[class*="toast"]').or(page.getByRole('alert'))
}
get path(): string {
return '/signin'
}
/**
* Fill in email address
*/
async fillEmail(email: string): Promise<void> {
await this.emailInput.fill(email)
}
/**
* Fill in password
*/
async fillPassword(password: string): Promise<void> {
await this.passwordInput.fill(password)
}
/**
* Click sign in button
*/
async clickSignIn(): Promise<void> {
await this.signInButton.click()
}
/**
* Complete login flow
*/
async login(email: string, password: string): Promise<void> {
await this.fillEmail(email)
await this.fillPassword(password)
await this.clickSignIn()
}
/**
* Login and wait for redirect to dashboard/apps
*/
async loginAndWaitForRedirect(email: string, password: string): Promise<void> {
await this.login(email, password)
// After successful login, Dify redirects to /apps
await expect(this.page).toHaveURL(/\/apps/, { timeout: 30000 })
}
/**
* Verify invalid credentials error is shown
* Error message: t('login.error.invalidEmailOrPassword') = "Invalid email or password."
*/
async expectInvalidCredentialsError(): Promise<void> {
await expect(this.errorMessage.filter({ hasText: /invalid|incorrect|wrong/i }))
.toBeVisible({ timeout: 10000 })
}
/**
* Verify email validation error
* Error message: t('login.error.emailInValid') = "Please enter a valid email address"
*/
async expectEmailValidationError(): Promise<void> {
await expect(this.errorMessage.filter({ hasText: /valid email/i }))
.toBeVisible({ timeout: 10000 })
}
/**
* Verify password empty error
* Error message: t('login.error.passwordEmpty') = "Password is required"
*/
async expectPasswordEmptyError(): Promise<void> {
await expect(this.errorMessage.filter({ hasText: /password.*required/i }))
.toBeVisible({ timeout: 10000 })
}
/**
* Check if user is already logged in (auto-redirected)
*/
async isRedirectedToApps(timeout = 5000): Promise<boolean> {
try {
await this.page.waitForURL(/\/apps/, { timeout })
return true
}
catch {
return false
}
}
}

View File

@@ -0,0 +1,353 @@
import type { Locator, Page } from '@playwright/test'
import { expect } from '@playwright/test'
import { BasePage } from './base.page'
/**
* Workflow Editor Page Object Model
*
* Handles interactions with the Dify workflow/canvas editor.
* Based on: web/app/components/workflow/
*
* Key components:
* - ReactFlow canvas: web/app/components/workflow/index.tsx
* - Run button: web/app/components/workflow/header/run-mode.tsx
* - Publish button: web/app/components/workflow/header/index.tsx
* - Zoom controls: web/app/components/workflow/operator/zoom-in-out.tsx
* - Node panel: web/app/components/workflow/panel/index.tsx
* - Block selector: web/app/components/workflow/block-selector/
*/
export class WorkflowPage extends BasePage {
// Canvas elements - ReactFlow based (web/app/components/workflow/index.tsx)
readonly canvas: Locator
readonly minimap: Locator
// Header action buttons (web/app/components/workflow/header/)
readonly runButton: Locator
readonly stopButton: Locator
readonly publishButton: Locator
readonly undoButton: Locator
readonly redoButton: Locator
readonly historyButton: Locator
readonly checklistButton: Locator
// Zoom controls (web/app/components/workflow/operator/zoom-in-out.tsx)
readonly zoomInButton: Locator
readonly zoomOutButton: Locator
readonly zoomPercentage: Locator
// Node panel - appears when node is selected (web/app/components/workflow/panel/)
readonly nodeConfigPanel: Locator
readonly envPanel: Locator
readonly versionHistoryPanel: Locator
// Debug and preview panel (web/app/components/workflow/panel/debug-and-preview/)
readonly debugPreviewPanel: Locator
readonly chatInput: Locator
// Block selector - for adding nodes (web/app/components/workflow/block-selector/)
readonly blockSelector: Locator
readonly blockSearchInput: Locator
constructor(page: Page) {
super(page)
// Canvas - ReactFlow renders with these classes
this.canvas = page.locator('.react-flow')
this.minimap = page.locator('.react-flow__minimap')
// Run button - shows "Test Run" text with play icon (run-mode.tsx)
// When running, shows "Running" with loading spinner
this.runButton = page.locator('.flex.items-center').filter({ hasText: /Test Run|Running|Listening/ }).first()
this.stopButton = page.locator('button').filter({ has: page.locator('svg.text-text-accent') }).filter({ hasText: '' }).last()
// Publish button in header (header/index.tsx)
this.publishButton = page.getByRole('button', { name: /Publish|Update/ })
// Undo/Redo buttons (header/undo-redo.tsx)
this.undoButton = page.locator('[class*="undo"]').or(page.getByRole('button', { name: 'Undo' }))
this.redoButton = page.locator('[class*="redo"]').or(page.getByRole('button', { name: 'Redo' }))
// History and checklist buttons (header/run-and-history.tsx)
this.historyButton = page.getByRole('button', { name: /history/i })
this.checklistButton = page.locator('[class*="checklist"]')
// Zoom controls at bottom (operator/zoom-in-out.tsx)
// Uses RiZoomInLine and RiZoomOutLine icons
this.zoomInButton = page.locator('.react-flow').locator('..').locator('button').filter({ has: page.locator('[class*="zoom-in"]') }).first()
.or(page.locator('svg[class*="RiZoomInLine"]').locator('..'))
this.zoomOutButton = page.locator('.react-flow').locator('..').locator('button').filter({ has: page.locator('[class*="zoom-out"]') }).first()
.or(page.locator('svg[class*="RiZoomOutLine"]').locator('..'))
this.zoomPercentage = page.locator('.system-sm-medium').filter({ hasText: /%$/ })
// Node config panel - appears on right when node selected (panel/index.tsx)
this.nodeConfigPanel = page.locator('.absolute.bottom-1.right-0.top-14')
this.envPanel = page.locator('[class*="env-panel"]')
this.versionHistoryPanel = page.locator('[class*="version-history"]')
// Debug preview panel (panel/debug-and-preview/)
this.debugPreviewPanel = page.locator('[class*="debug"], [class*="preview-panel"]')
this.chatInput = page.locator('textarea[placeholder*="Enter"], textarea[placeholder*="input"]')
// Block selector popup (block-selector/)
this.blockSelector = page.locator('[class*="block-selector"], [role="dialog"]').filter({ hasText: /LLM|Code|HTTP|IF/ })
this.blockSearchInput = page.getByPlaceholder(/search/i)
}
get path(): string {
// Dynamic path - will be set when navigating to specific workflow
return '/app'
}
/**
* Navigate to a specific workflow app by ID
*/
async gotoWorkflow(appId: string): Promise<void> {
await this.page.goto(`/app/${appId}/workflow`)
await this.waitForPageLoad()
await this.waitForCanvasReady()
}
/**
* Wait for ReactFlow canvas to be fully loaded
*/
async waitForCanvasReady(): Promise<void> {
await expect(this.canvas).toBeVisible({ timeout: 30000 })
// Wait for nodes to render (ReactFlow needs time to initialize)
await this.page.waitForSelector('.react-flow__node', { timeout: 30000 })
await this.page.waitForTimeout(500) // Allow animation to complete
}
/**
* Get a node by its displayed title/name
* Dify nodes use .react-flow__node class with title text inside
*/
node(name: string): Locator {
return this.canvas.locator('.react-flow__node').filter({ hasText: name })
}
/**
* Get start node (entry point of workflow)
*/
get startNode(): Locator {
return this.canvas.locator('.react-flow__node').filter({ hasText: /Start|开始/ }).first()
}
/**
* Get end node
*/
get endNode(): Locator {
return this.canvas.locator('.react-flow__node').filter({ hasText: /End|结束/ }).first()
}
/**
* Add a new node by clicking on canvas edge and selecting from block selector
* @param nodeType - Node type like 'LLM', 'Code', 'HTTP Request', 'IF/ELSE', etc.
*/
async addNode(nodeType: string): Promise<void> {
// Click the + button on a node's edge to open block selector
const addButton = this.canvas.locator('.react-flow__node').first()
.locator('[class*="handle"], [class*="add"]')
await addButton.click()
// Wait for block selector to appear
await expect(this.blockSelector).toBeVisible({ timeout: 5000 })
// Search for node type if search is available
if (await this.blockSearchInput.isVisible())
await this.blockSearchInput.fill(nodeType)
// Click on the node type option
await this.blockSelector.getByText(nodeType, { exact: false }).first().click()
await this.waitForCanvasReady()
}
/**
* Select a node on the canvas (opens config panel on right)
*/
async selectNode(name: string): Promise<void> {
await this.node(name).click()
// Config panel should appear
await expect(this.nodeConfigPanel).toBeVisible({ timeout: 5000 })
}
/**
* Delete the currently selected node using keyboard
*/
async deleteSelectedNode(): Promise<void> {
await this.page.keyboard.press('Delete')
// Or Backspace
// await this.page.keyboard.press('Backspace')
}
/**
* Delete a node by name using context menu
*/
async deleteNode(name: string): Promise<void> {
await this.node(name).click({ button: 'right' })
await this.page.getByRole('menuitem', { name: /delete|删除/i }).click()
}
/**
* Connect two nodes by dragging from source handle to target handle
*/
async connectNodes(fromNode: string, toNode: string): Promise<void> {
// ReactFlow uses data-handlepos for handle positions
const sourceHandle = this.node(fromNode).locator('.react-flow__handle-right, [data-handlepos="right"]')
const targetHandle = this.node(toNode).locator('.react-flow__handle-left, [data-handlepos="left"]')
await sourceHandle.dragTo(targetHandle)
}
/**
* Run/test the workflow (click Test Run button)
*/
async runWorkflow(): Promise<void> {
await this.runButton.click()
}
/**
* Stop a running workflow
*/
async stopWorkflow(): Promise<void> {
await this.stopButton.click()
}
/**
* Check if workflow is currently running
*/
async isRunning(): Promise<boolean> {
const text = await this.runButton.textContent()
return text?.includes('Running') || text?.includes('Listening') || false
}
/**
* Publish the workflow
*/
async publishWorkflow(): Promise<void> {
await this.publishButton.click()
// Handle confirmation dialog if it appears
const confirmButton = this.page.getByRole('button', { name: /confirm|确认/i })
if (await confirmButton.isVisible({ timeout: 2000 }))
await confirmButton.click()
await this.expectSuccessToast()
}
/**
* Wait for workflow run to complete (success or failure)
*/
async waitForRunComplete(timeout = 60000): Promise<void> {
// Wait until the "Running" state ends
await expect(async () => {
const isStillRunning = await this.isRunning()
expect(isStillRunning).toBe(false)
}).toPass({ timeout })
}
/**
* Verify workflow run completed successfully
*/
async expectRunSuccess(): Promise<void> {
await this.waitForRunComplete()
// Check for success indicators in the debug panel or toast
const successIndicator = this.page.locator(':text("Succeeded"), :text("success"), :text("成功")')
await expect(successIndicator).toBeVisible({ timeout: 10000 })
}
/**
* Get the count of nodes on canvas
*/
async getNodeCount(): Promise<number> {
return this.canvas.locator('.react-flow__node').count()
}
/**
* Verify a specific node exists on canvas
*/
async expectNodeExists(name: string): Promise<void> {
await expect(this.node(name)).toBeVisible()
}
/**
* Verify a specific node does not exist on canvas
*/
async expectNodeNotExists(name: string): Promise<void> {
await expect(this.node(name)).not.toBeVisible()
}
/**
* Zoom in the canvas
*/
async zoomIn(): Promise<void> {
// Use keyboard shortcut Ctrl++
await this.page.keyboard.press('Control++')
}
/**
* Zoom out the canvas
*/
async zoomOut(): Promise<void> {
// Use keyboard shortcut Ctrl+-
await this.page.keyboard.press('Control+-')
}
/**
* Fit view to show all nodes (keyboard shortcut Ctrl+1)
*/
async fitView(): Promise<void> {
await this.page.keyboard.press('Control+1')
}
/**
* Get current zoom percentage
*/
async getZoomPercentage(): Promise<number> {
const text = await this.zoomPercentage.textContent()
return Number.parseInt(text?.replace('%', '') || '100')
}
/**
* Undo last action (Ctrl+Z)
*/
async undo(): Promise<void> {
await this.page.keyboard.press('Control+z')
}
/**
* Redo last undone action (Ctrl+Shift+Z)
*/
async redo(): Promise<void> {
await this.page.keyboard.press('Control+Shift+z')
}
/**
* Open version history panel
*/
async openVersionHistory(): Promise<void> {
await this.historyButton.click()
await expect(this.versionHistoryPanel).toBeVisible()
}
/**
* Duplicate selected node (Ctrl+D)
*/
async duplicateSelectedNode(): Promise<void> {
await this.page.keyboard.press('Control+d')
}
/**
* Copy selected node (Ctrl+C)
*/
async copySelectedNode(): Promise<void> {
await this.page.keyboard.press('Control+c')
}
/**
* Paste node (Ctrl+V)
*/
async pasteNode(): Promise<void> {
await this.page.keyboard.press('Control+v')
}
}

View File

@@ -0,0 +1,25 @@
import { expect, test } from '../fixtures'
/**
* Apps page E2E tests
*
* These tests verify the apps listing and creation functionality.
*/
test.describe('Apps Page', () => {
test('should display apps page after authentication', async ({ page }) => {
// Navigate to apps page
await page.goto('/apps')
// Verify we're on the apps page (not redirected to signin)
await expect(page).toHaveURL(/\/apps/)
// Wait for the page to fully load
await page.waitForLoadState('networkidle')
// Take a screenshot for debugging
await page.screenshot({ path: 'e2e/test-results/apps-page.png' })
console.log('✅ Apps page loaded successfully')
})
})

View File

@@ -0,0 +1,165 @@
import type { APIRequestContext } from '@playwright/test'
/**
* API helper utilities for test setup and cleanup
*
* Use these helpers to set up test data via API before tests run,
* or to clean up data after tests complete.
*
* Environment variables:
* - NEXT_PUBLIC_API_PREFIX: API URL (default: http://localhost:5001/console/api)
*
* Based on Dify API configuration:
* @see web/config/index.ts - API_PREFIX
* @see web/types/app.ts - AppModeEnum
*/
// API base URL with fallback for local development
const API_BASE_URL = process.env.NEXT_PUBLIC_API_PREFIX || 'http://localhost:5001/console/api'
/**
* Dify App mode types
* @see web/types/app.ts - AppModeEnum
*/
export type AppMode = 'chat' | 'completion' | 'workflow'
/**
* Create a new app via API
*
* @param request - Playwright API request context
* @param data - App data
* @param data.name - App name
* @param data.mode - App mode: chat (Chatbot), completion (Text Generator),
* workflow, advanced-chat (Chatflow), agent-chat (Agent)
* @param data.description - Optional description
* @param data.icon - Optional icon
* @param data.iconBackground - Optional icon background color
*/
export async function createAppViaApi(
request: APIRequestContext,
data: {
name: string
mode: AppMode
description?: string
icon?: string
iconBackground?: string
},
): Promise<{ id: string, name: string }> {
const response = await request.post(`${API_BASE_URL}/apps`, {
data: {
name: data.name,
mode: data.mode,
description: data.description || '',
icon: data.icon || 'default',
icon_background: data.iconBackground || '#FFFFFF',
},
})
if (!response.ok()) {
const error = await response.text()
throw new Error(`Failed to create app: ${error}`)
}
return response.json()
}
/**
* Delete an app via API
*/
export async function deleteAppViaApi(
request: APIRequestContext,
appId: string,
): Promise<void> {
const response = await request.delete(`${API_BASE_URL}/apps/${appId}`)
if (!response.ok() && response.status() !== 404) {
const error = await response.text()
throw new Error(`Failed to delete app: ${error}`)
}
}
/**
* Create a dataset/knowledge base via API
*/
export async function createDatasetViaApi(
request: APIRequestContext,
data: {
name: string
description?: string
},
): Promise<{ id: string, name: string }> {
const response = await request.post(`${API_BASE_URL}/datasets`, {
data: {
name: data.name,
description: data.description || '',
},
})
if (!response.ok()) {
const error = await response.text()
throw new Error(`Failed to create dataset: ${error}`)
}
return response.json()
}
/**
* Delete a dataset via API
*/
export async function deleteDatasetViaApi(
request: APIRequestContext,
datasetId: string,
): Promise<void> {
const response = await request.delete(`${API_BASE_URL}/datasets/${datasetId}`)
if (!response.ok() && response.status() !== 404) {
const error = await response.text()
throw new Error(`Failed to delete dataset: ${error}`)
}
}
/**
* Get current user info via API
*/
export async function getCurrentUserViaApi(
request: APIRequestContext,
): Promise<{ id: string, email: string, name: string }> {
const response = await request.get(`${API_BASE_URL}/account/profile`)
if (!response.ok()) {
const error = await response.text()
throw new Error(`Failed to get user info: ${error}`)
}
return response.json()
}
/**
* Cleanup helper - delete all test apps by name pattern
*/
export async function cleanupTestApps(
request: APIRequestContext,
namePattern: RegExp,
): Promise<number> {
const response = await request.get(`${API_BASE_URL}/apps`)
if (!response.ok())
return 0
const { data: apps } = await response.json() as { data: Array<{ id: string, name: string }> }
const testApps = apps.filter(app => namePattern.test(app.name))
let deletedCount = 0
for (const app of testApps) {
try {
await deleteAppViaApi(request, app.id)
deletedCount++
}
catch {
// Ignore deletion errors during cleanup
}
}
return deletedCount
}

8
web/e2e/utils/index.ts Normal file
View File

@@ -0,0 +1,8 @@
/**
* Test Utilities Index
*
* Export all utility functions from a single entry point.
*/
export * from './test-helpers'
export * from './api-helpers'

View File

@@ -0,0 +1,174 @@
import type { Locator, Page } from '@playwright/test'
/**
* Common test helper utilities for E2E tests
*/
/**
* Wait for network to be idle with a custom timeout
*/
export async function waitForNetworkIdle(page: Page, timeout = 5000): Promise<void> {
await page.waitForLoadState('networkidle', { timeout })
}
/**
* Wait for an element to be stable (not moving/resizing)
*/
export async function waitForStable(locator: Locator, timeout = 5000): Promise<void> {
await locator.waitFor({ state: 'visible', timeout })
// Additional wait for animations to complete
await locator.evaluate(el => new Promise<void>((resolve) => {
const observer = new MutationObserver(() => {
// Observer callback - intentionally empty, just watching for changes
})
observer.observe(el, { attributes: true, subtree: true })
setTimeout(() => {
observer.disconnect()
resolve()
}, 100)
}))
}
/**
* Safely click an element with retry logic
*/
export async function safeClick(
locator: Locator,
options?: { timeout?: number, force?: boolean },
): Promise<void> {
const { timeout = 10000, force = false } = options || {}
await locator.waitFor({ state: 'visible', timeout })
await locator.click({ force, timeout })
}
/**
* Fill input with clear first
*/
export async function fillInput(
locator: Locator,
value: string,
options?: { clear?: boolean },
): Promise<void> {
const { clear = true } = options || {}
if (clear)
await locator.clear()
await locator.fill(value)
}
/**
* Select option from dropdown/select element
*/
export async function selectOption(
trigger: Locator,
optionText: string,
page: Page,
): Promise<void> {
await trigger.click()
await page.getByRole('option', { name: optionText }).click()
}
/**
* Wait for toast notification and verify its content
*
* Based on Dify toast implementation:
* @see web/app/components/base/toast/index.tsx
*
* Toast structure:
* - Container: .fixed.z-[9999] with rounded-xl
* - Type background classes: bg-toast-success-bg, bg-toast-error-bg, etc.
* - Type icon classes: text-text-success, text-text-destructive, etc.
*/
export async function waitForToast(
page: Page,
expectedText: string | RegExp,
type?: 'success' | 'error' | 'warning' | 'info',
): Promise<Locator> {
// Dify toast uses fixed positioning with z-[9999]
const toastContainer = page.locator('.fixed.z-\\[9999\\]')
// Filter by type if specified
let toast: Locator
if (type) {
// Each type has specific background class
const typeClassMap: Record<string, string> = {
success: '.bg-toast-success-bg',
error: '.bg-toast-error-bg',
warning: '.bg-toast-warning-bg',
info: '.bg-toast-info-bg',
}
toast = toastContainer.filter({ has: page.locator(typeClassMap[type]) })
.filter({ hasText: expectedText })
}
else {
toast = toastContainer.filter({ hasText: expectedText })
}
await toast.waitFor({ state: 'visible', timeout: 10000 })
return toast
}
/**
* Dismiss any visible modals
*/
export async function dismissModal(page: Page): Promise<void> {
const modal = page.locator('[role="dialog"]')
if (await modal.isVisible()) {
// Try clicking close button or backdrop
const closeButton = modal.locator('button[aria-label*="close"], button:has-text("Cancel")')
if (await closeButton.isVisible())
await closeButton.click()
else
await page.keyboard.press('Escape')
await modal.waitFor({ state: 'hidden', timeout: 5000 })
}
}
/**
* Generate a unique test identifier
*/
export function generateTestId(prefix = 'test'): string {
const timestamp = Date.now()
const random = Math.random().toString(36).substring(2, 8)
return `${prefix}-${timestamp}-${random}`
}
/**
* Take a screenshot with a descriptive name
*/
export async function takeDebugScreenshot(
page: Page,
name: string,
): Promise<void> {
const timestamp = new Date().toISOString().replace(/[:.]/g, '-')
await page.screenshot({
path: `e2e/test-results/debug-${name}-${timestamp}.png`,
fullPage: true,
})
}
/**
* Retry an action with exponential backoff
*/
export async function retryAction<T>(
action: () => Promise<T>,
options?: { maxAttempts?: number, baseDelay?: number },
): Promise<T> {
const { maxAttempts = 3, baseDelay = 1000 } = options || {}
for (let attempt = 1; attempt <= maxAttempts; attempt++) {
try {
return await action()
}
catch (error) {
if (attempt === maxAttempts)
throw error
const delay = baseDelay * 2 ** (attempt - 1)
await new Promise(resolve => setTimeout(resolve, delay))
}
}
throw new Error('Unreachable')
}

View File

@@ -40,6 +40,11 @@
"test": "jest",
"test:watch": "jest --watch",
"analyze-component": "node testing/analyze-component.js",
"test:e2e": "playwright test",
"test:e2e:ui": "playwright test --ui",
"test:e2e:headed": "playwright test --headed",
"test:e2e:debug": "playwright test --debug",
"test:e2e:report": "playwright show-report",
"storybook": "storybook dev -p 6006",
"build-storybook": "storybook build",
"preinstall": "npx only-allow pnpm",
@@ -158,6 +163,7 @@
"@next/bundle-analyzer": "15.5.9",
"@next/eslint-plugin-next": "15.5.9",
"@next/mdx": "15.5.9",
"@playwright/test": "^1.56.1",
"@rgrove/parse-xml": "^4.2.0",
"@storybook/addon-docs": "9.1.13",
"@storybook/addon-links": "9.1.13",
@@ -191,6 +197,7 @@
"bing-translate-api": "^4.1.0",
"code-inspector-plugin": "1.2.9",
"cross-env": "^10.1.0",
"dotenv": "^17.2.3",
"eslint": "^9.38.0",
"eslint-plugin-oxlint": "^1.23.0",
"eslint-plugin-react-hooks": "^5.2.0",

170
web/playwright.config.ts Normal file
View File

@@ -0,0 +1,170 @@
import path from 'node:path'
import { defineConfig, devices } from '@playwright/test'
/**
* Playwright Configuration for Dify E2E Tests
*
* Environment variables are loaded from web/.env.local
*
* E2E specific variables:
* - E2E_BASE_URL: Base URL for tests (default: http://localhost:3000)
* - E2E_SKIP_WEB_SERVER: Set to 'true' to skip starting dev server (for CI with deployed env)
* @see https://playwright.dev/docs/test-configuration
*/
// Load environment variables from web/.env.local
require('dotenv').config({ path: path.resolve(__dirname, '.env.local') })
// Base URL for the frontend application
// - Local development: http://localhost:3000
// - CI/CD with deployed env: set E2E_BASE_URL to the deployed URL
const BASE_URL = process.env.E2E_BASE_URL || 'http://localhost:3000'
// Whether to skip starting the web server
// - Local development: false (start dev server)
// - CI/CD with deployed env: true (use existing server)
const SKIP_WEB_SERVER = process.env.E2E_SKIP_WEB_SERVER === 'true'
// Cloudflare Access headers (for protected environments).
// Prefer environment variables to avoid hardcoding secrets in repo.
const CF_ACCESS_CLIENT_ID = process.env.CF_ACCESS_CLIENT_ID
const CF_ACCESS_CLIENT_SECRET = process.env.CF_ACCESS_CLIENT_SECRET
const cfAccessHeaders: Record<string, string> = {}
if (CF_ACCESS_CLIENT_ID && CF_ACCESS_CLIENT_SECRET) {
cfAccessHeaders['CF-Access-Client-Id'] = CF_ACCESS_CLIENT_ID
cfAccessHeaders['CF-Access-Client-Secret'] = CF_ACCESS_CLIENT_SECRET
}
export default defineConfig({
// Directory containing test files
testDir: './e2e/tests',
// Run tests in files in parallel
fullyParallel: true,
// Fail the build on CI if you accidentally left test.only in the source code
forbidOnly: !!process.env.CI,
// Retry on CI only
retries: process.env.CI ? 2 : 0,
// Opt out of parallel tests on CI for stability
workers: process.env.CI ? 1 : undefined,
// Reporter to use
reporter: process.env.CI
? [['html', { open: 'never', outputFolder: 'playwright-report' }], ['github'], ['json', { outputFile: 'e2e/test-results/results.json' }]]
: [['html', { open: 'on-failure' }], ['list']],
// Shared settings for all the projects below
use: {
// Base URL for all page.goto() calls
baseURL: BASE_URL,
// Extra headers for all requests made by the browser context.
extraHTTPHeaders: cfAccessHeaders,
// Bypass Content Security Policy to allow test automation
// This is needed when testing against environments with strict CSP headers
bypassCSP: true,
// Collect trace when retrying the failed test
trace: 'on-first-retry',
// Take screenshot on failure
screenshot: 'only-on-failure',
// Record video on failure
video: 'on-first-retry',
// Default timeout for actions
actionTimeout: 10000,
// Default timeout for navigation
navigationTimeout: 30000,
},
// Global timeout for each test
timeout: 60000,
// Expect timeout
expect: {
timeout: 10000,
},
// Configure projects for major browsers
projects: [
// Setup project - runs before all tests to handle authentication
{
name: 'setup',
testDir: './e2e',
testMatch: /global\.setup\.ts/,
teardown: 'teardown',
},
{
name: 'teardown',
testDir: './e2e',
testMatch: /global\.teardown\.ts/,
},
// Main test project - uses authenticated state
{
name: 'chromium',
use: {
...devices['Desktop Chrome'],
// Use prepared auth state
storageState: 'e2e/.auth/user.json',
},
dependencies: ['setup'],
},
// Test in Firefox (optional, uncomment when needed)
// {
// name: 'firefox',
// use: {
// ...devices['Desktop Firefox'],
// storageState: 'e2e/.auth/user.json',
// },
// dependencies: ['setup'],
// },
// Test in WebKit (optional, uncomment when needed)
// {
// name: 'webkit',
// use: {
// ...devices['Desktop Safari'],
// storageState: 'e2e/.auth/user.json',
// },
// dependencies: ['setup'],
// },
// Test against mobile viewports (optional)
// {
// name: 'mobile-chrome',
// use: {
// ...devices['Pixel 5'],
// storageState: 'e2e/.auth/user.json',
// },
// dependencies: ['setup'],
// },
],
// Output folder for test artifacts
outputDir: 'e2e/test-results',
// Run your local dev server before starting the tests
// - Local: starts dev server automatically
// - CI with deployed env: set E2E_SKIP_WEB_SERVER=true to skip
...(SKIP_WEB_SERVER
? {}
: {
webServer: {
command: 'pnpm dev',
url: BASE_URL,
// Reuse existing server in local dev, start fresh in CI
reuseExistingServer: !process.env.CI,
timeout: 120000,
},
}),
})

14
web/pnpm-lock.yaml generated
View File

@@ -386,6 +386,9 @@ importers:
'@next/mdx':
specifier: 15.5.9
version: 15.5.9(@mdx-js/loader@3.1.1(webpack@5.103.0(esbuild@0.25.0)(uglify-js@3.19.3)))(@mdx-js/react@3.1.1(@types/react@19.2.7)(react@19.2.3))
'@playwright/test':
specifier: ^1.56.1
version: 1.57.0
'@rgrove/parse-xml':
specifier: ^4.2.0
version: 4.2.0
@@ -485,6 +488,9 @@ importers:
cross-env:
specifier: ^10.1.0
version: 10.1.0
dotenv:
specifier: ^17.2.3
version: 17.2.3
eslint:
specifier: ^9.38.0
version: 9.39.1(jiti@1.21.7)
@@ -4994,6 +5000,10 @@ packages:
resolution: {integrity: sha512-uBq4egWHTcTt33a72vpSG0z3HnPuIl6NqYcTrKEg2azoEyl2hpW0zqlxysq2pK9HlDIHyHyakeYaYnSAwd8bow==}
engines: {node: '>=12'}
dotenv@17.2.3:
resolution: {integrity: sha512-JVUnt+DUIzu87TABbhPmNfVdBDt18BLOWjMUFJMSi/Qqg7NTYtabbvSNJGOJ7afbRuv9D/lngizHtP7QyLQ+9w==}
engines: {node: '>=12'}
duplexer@0.1.2:
resolution: {integrity: sha512-jtD6YG370ZCIi/9GTaJKQxWTZD045+4R4hTk/x1UyoqadyJ9x9CgSi1RlVDQF8U2sxLLSnFkCaMihqljHIWgMg==}
@@ -6892,7 +6902,6 @@ packages:
next@15.5.9:
resolution: {integrity: sha512-agNLK89seZEtC5zUHwtut0+tNrc0Xw4FT/Dg+B/VLEo9pAcS9rtTKpek3V6kVcVwsB2YlqMaHdfZL4eLEVYuCg==}
engines: {node: ^18.18.0 || ^19.8.0 || >= 20.0.0}
deprecated: This version has a security vulnerability. Please upgrade to a patched version. See https://nextjs.org/blog/security-update-2025-12-11 for more details.
hasBin: true
peerDependencies:
'@opentelemetry/api': ^1.1.0
@@ -11456,7 +11465,6 @@ snapshots:
'@playwright/test@1.57.0':
dependencies:
playwright: 1.57.0
optional: true
'@pmmmwh/react-refresh-webpack-plugin@0.5.17(react-refresh@0.14.2)(type-fest@4.2.0)(webpack-hot-middleware@2.26.1)(webpack@5.103.0(esbuild@0.25.0)(uglify-js@3.19.3))':
dependencies:
@@ -13913,6 +13921,8 @@ snapshots:
dotenv@16.6.1: {}
dotenv@17.2.3: {}
duplexer@0.1.2: {}
echarts-for-react@3.0.5(echarts@5.6.0)(react@19.2.3):