-
Notifications
You must be signed in to change notification settings - Fork 207
docs: add Comfy Agent guides #1651
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
14 commits
Select commit
Hold shift + click to select a range
eb092c9
Prepare Comfy Agent beta docs for cross-functional review
stale2000 8a8edbd
Make Comfy Agent beta docs easier to read
stale2000 b61d8e0
Describe Comfy Agent in product terms
stale2000 f07553a
Focus Comfy Agent docs on user tasks
stale2000 253b2c0
Ground Comfy Agent guides in shipped capabilities
stale2000 b4ae19d
Use plain language in Comfy Agent guides
stale2000 a8bd14c
Help readers complete their first Agent workflow
stale2000 9d31f3d
Give readers actionable next steps after their first image
stale2000 a5f1065
Replace generic Agent resource lists with task links
stale2000 369f554
Keep privacy guidance beside the relevant Agent explanation
stale2000 f4b7043
Restore Comfy Agent feedback link
stale2000 ce9f6bb
Restore Comfy Agent waitlist entry
stale2000 fff7fb8
Align setup instructions with the run permission label
stale2000 32b473b
Preserve private-alpha access guidance during the Agent expansion
stale2000 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,46 @@ | ||
| --- | ||
| title: "Comfy Agent Data and Privacy" | ||
| sidebarTitle: "Data and privacy" | ||
| description: "Understand what Comfy Agent uses to answer a request and how local and Cloud use differ." | ||
| icon: "shield" | ||
| --- | ||
|
|
||
| ## The short version | ||
|
|
||
| Comfy Agent uses your message, selected workflow, selected nodes, and attachments. When you work with a local ComfyUI installation, chat history is stored on that machine. Your message and the information needed to answer it are sent through Comfy's online service. | ||
|
|
||
| ## What the Agent uses | ||
|
|
||
| | Data | How it helps | | ||
| | --- | --- | | ||
| | Your message | Tells the Agent what you want to do. | | ||
| | Selected workflow and nodes | Show which workflow and nodes you want explained or changed. | | ||
| | Image attachment | Lets the Agent inspect the image and use it in a workflow. | | ||
| | Video or audio attachment | Lets the Agent read media properties and extract or convert content. | | ||
| | Workflow and execution errors | Help the Agent explain and fix a failed workflow. | | ||
| | Personal skill | Supplies reusable instructions. | | ||
|
|
||
| The [Privacy Policy](https://www.comfy.org/privacy-policy?utm_source=docs) and [Terms of Service](https://www.comfy.org/terms-of-service?utm_source=docs) govern your use of Comfy. | ||
|
|
||
| ## Cloud and local data flow | ||
|
|
||
| | Where you use Comfy Agent | What happens | | ||
| | --- | --- | | ||
| | **Comfy Cloud** | Comfy Cloud stores conversations, workflows, and assets. Your messages are processed by Comfy's online service, and workflows run on Cloud GPUs. | | ||
|
Check warning on line 29 in agent-tools/in-app-agent-data-privacy.mdx
|
||
| | **Local ComfyUI in Desktop** | The local installation stores chat history, workflows, and files. Your messages and relevant workflow information are processed by Comfy's online service. Local workflow nodes run on your hardware; partner nodes use their online providers. | | ||
|
|
||
| ## Chat history and sync | ||
|
|
||
| Conversations with the Cloud Agent are stored with Comfy Cloud. Conversations with the local Agent are stored on the machine running ComfyUI. The two histories do not sync. This depends on whether you are using Cloud or a local installation, not which application window you open it in. | ||
|
|
||
| ## Skills and guidance | ||
|
|
||
| Personal skills contain instructions you create for the Agent. They are saved to your account and used when your request matches them or you name them. | ||
|
|
||
| Built-in Agent guidance and skills for external coding agents are separate from your personal Agent skills. | ||
|
|
||
| ## Retention and deletion | ||
|
|
||
| For retention details, see the [Privacy Policy](https://www.comfy.org/privacy-policy?utm_source=docs) and [Data Retention](/support/data-retention). | ||
|
|
||
| For account or data-deletion requests, contact [support@comfy.org](mailto:support@comfy.org). | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,72 @@ | ||
| --- | ||
| title: "Get Started with Comfy Agent" | ||
| sidebarTitle: "Get started" | ||
| description: "Open Comfy Agent in Comfy Cloud or Comfy Desktop." | ||
| icon: "play" | ||
| --- | ||
|
|
||
| ## Choose your setup | ||
|
|
||
| You need a Comfy account with Agent access. If you do not have access, [join the private alpha waitlist](https://links.comfy.org/agentalpha). | ||
|
|
||
| - **Comfy Cloud:** [Sign in to Cloud](https://cloud.comfy.org). No local installation is needed. | ||
| - **Comfy Desktop:** Follow the [Desktop installation guide](/installation/desktop/overview), then sign in. For an existing installation, follow [Update ComfyUI](/installation/update_comfyui). | ||
|
|
||
| ## Make your first image | ||
|
|
||
| ### Open the Agent | ||
|
|
||
| Open a blank workflow, then select **Ask Comfy Agent** in the workflow bar. Read and accept the consent notice if prompted. Use **Choose a workflow** in the Agent panel to select the blank workflow. | ||
|
|
||
| ### Build a workflow | ||
|
|
||
| Send this message: | ||
|
|
||
| > Build a text-to-image workflow for a red sneaker on a neutral gray background. Use an available model and include a Save Image node. Check the workflow for errors, but don't run it yet. | ||
|
|
||
| The Agent builds the workflow on the canvas, fills in the prompt and model settings, and checks the node connections and required inputs. You should see a connected workflow ending in a **Save Image** node. | ||
|
|
||
| If the Agent reports a missing model or file, supply it or ask for an alternative available in your installation. Ask it to check the workflow again before continuing. | ||
|
|
||
| ### Generate the image | ||
|
|
||
| Open **Run permissions** next to the message box, choose **Ask**, and select **Save changes** if you changed the setting. | ||
|
|
||
| Send “Run this workflow.” When the Agent asks for approval, select **Run**. Select **Cancel** instead if you do not want to start that run. | ||
|
|
||
| When generation finishes, the Agent returns the output in the conversation. The image also appears in the **Save Image** node. Right-click the image there to save it to your computer. | ||
|
|
||
| ### Make a change | ||
|
|
||
| Send “Change the background to white and keep the model and other settings.” The Agent updates the existing workflow. Ask it to run again to compare the result. | ||
|
|
||
| ## Troubleshooting | ||
|
|
||
| ### The Agent button is missing | ||
|
|
||
| Sign in to your Comfy account. In Comfy Desktop, [update the application](/installation/update_comfyui) and restart it. If the button is still missing, check your account's Agent access with [support](/support/contact-support). | ||
|
|
||
| ### The Agent cannot edit the workflow I expected | ||
|
|
||
| Choose the intended workflow in the Agent panel, then send your message again. See [Choose which workflow to edit](/agent-tools/in-app-agent#choose-which-workflow-to-edit). | ||
|
|
||
| ### The Agent asks for an input | ||
|
|
||
| Select **Attach a file** or drag the requested file into the message box, then ask the Agent to continue. | ||
|
|
||
| ### I cannot find a local conversation in Cloud | ||
|
|
||
| Local and Cloud history are separate. See [Data and privacy](/agent-tools/in-app-agent-data-privacy#chat-history-and-sync). | ||
|
|
||
| ## Next steps | ||
|
|
||
| <CardGroup cols={2}> | ||
| <Card title="Edit an existing workflow" icon="pen" href="/agent-tools/in-app-agent#build-and-edit-workflows"> | ||
| Change prompts, models, or processing steps. | ||
| </Card> | ||
| <Card title="Save a reusable skill" icon="wand-magic-sparkles" href="/agent-tools/in-app-agent-skills#create-a-skill-in-chat"> | ||
| Remember instructions for future requests. | ||
| </Card> | ||
| </CardGroup> | ||
|
|
||
| See [Data and privacy](/agent-tools/in-app-agent-data-privacy) to learn where messages, files, and conversations are stored. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,52 @@ | ||
| --- | ||
| title: "Comfy Agent Skills" | ||
| sidebarTitle: "Agent skills" | ||
| description: "Use personal skills to give Comfy Agent reusable instructions for your work." | ||
| icon: "wand-magic-sparkles" | ||
| --- | ||
|
|
||
| ## What is a personal skill? | ||
|
|
||
| A personal skill is a reusable set of instructions for Comfy Agent. It includes a name, a one-line description of when to use it, and the instructions themselves. | ||
|
|
||
| Use personal skills to repeat instructions such as “make square images,” “use a white background,” or “check node connections before running.” | ||
|
|
||
| Personal skills are saved to your account. The Agent loads a skill when its description matches your request or when you ask for it by name. | ||
|
|
||
| ## Create a skill in chat | ||
|
|
||
| Tell the Agent what to save, what to call it, and when it should be used: | ||
|
|
||
| > Save this as a skill called product-shot: When I ask for a product image, use a square 1024 × 1024 canvas, a neutral gray background, and a Save Image node. Validate the workflow before running it. | ||
|
|
||
| The Agent saves the skill for future messages. A new or updated skill takes effect with your next message. | ||
|
|
||
| ## Use a skill | ||
|
|
||
| Ask for a skill by name: | ||
|
|
||
| > Use my product-shot skill to create a studio photo workflow for a red sneaker. | ||
|
|
||
| The Agent can also select a skill automatically when your request matches its description. | ||
|
|
||
| ## Update or delete a skill | ||
|
|
||
| Use the same skill name to replace its instructions: | ||
|
|
||
| > Update my product-shot skill to use a white background and a 1536 × 1536 canvas. | ||
|
|
||
| Delete a skill by name: | ||
|
|
||
| > Delete my product-shot skill. | ||
|
|
||
| ## Save an existing skill file | ||
|
|
||
| Paste the contents of your `SKILL.md` file into chat and ask the Agent to save it as a personal skill. The Agent reads its name and description from the header at the top of the file. The saved skill is available from your next message. | ||
|
|
||
| ## Built-in guidance and coding-agent skills | ||
|
|
||
| Comfy Agent includes guidance for building workflows and choosing models. Personal skills add your own instructions for repeated tasks. | ||
|
|
||
| [Skills for coding agents](/agent-tools/skills) are installed separately in tools such as Claude Code. Installing one there does not add it to Comfy Agent. | ||
|
|
||
| Keep API keys, passwords, and access tokens out of skill instructions. For data handling, see [Data and privacy](/agent-tools/in-app-agent-data-privacy#skills-and-guidance). |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,39 +1,121 @@ | ||
| --- | ||
| title: "Comfy In-App Agent" | ||
| sidebarTitle: "In-App Agent" | ||
| description: "Join the private alpha waitlist for Comfy In-App Agent on Comfy Cloud." | ||
| title: "Comfy Agent" | ||
| sidebarTitle: "Overview" | ||
| description: "Use Comfy Agent to understand, build, edit, and run ComfyUI workflows. Join the private alpha waitlist for access." | ||
| icon: "comments" | ||
| --- | ||
|
|
||
| <Note> | ||
| **Private alpha.** Comfy In-App Agent is rolling out to a limited set of users. Public docs and website pages are not available yet. Join the waitlist to request access. | ||
| **Private alpha.** Comfy Agent is rolling out to a limited set of users. Join the waitlist to request access. | ||
| </Note> | ||
|
|
||
| **Comfy In-App Agent** is the agent experience inside Comfy Cloud: prompt in chat, and the agent can build or edit workflows on your graph. | ||
| Comfy Agent is an assistant inside ComfyUI. It can build and edit workflows, explain nodes, inspect media, fix validation errors, and run the finished workflow. | ||
|
|
||
| ## Join the waitlist | ||
|
|
||
| <Card title="Join the alpha waitlist" icon="list" href="https://links.comfy.org/agentalpha"> | ||
| Request access to the Comfy In-App Agent private alpha. | ||
| Request access to the Comfy Agent private alpha. | ||
| </Card> | ||
|
|
||
| ## Feedback | ||
| ## What it can do | ||
|
|
||
| Already in the alpha? Share bugs and impressions: | ||
| - Find templates, models, and nodes available in your ComfyUI environment. | ||
| - Build a workflow from a template and customize its prompts and settings. | ||
| - Change part of the current workflow without rebuilding the rest. | ||
| - Add, connect, configure, and remove nodes. | ||
| - Validate a workflow before running it. | ||
| - Inspect images and read video or audio properties. | ||
| - Run workflows and return their outputs. | ||
|
|
||
| <Card title="Send feedback" icon="message" href="https://links.comfy.org/agentalphafeedback"> | ||
| Report bugs or share impressions from the alpha. | ||
| </Card> | ||
| ## Where you can use it | ||
|
|
||
| | Where you use Comfy Agent | What to expect | | ||
| | --- | --- | | ||
| | **Comfy Cloud** | The Agent, workflows, and assets use your Comfy Cloud account. There is no local package to install. | | ||
| | **Comfy Desktop** | The Agent works with the ComfyUI installation on your machine, including its workflows, models, and custom nodes. | | ||
| | **Network connection** | Agent requests use Comfy services. See [Data and privacy](/agent-tools/in-app-agent-data-privacy). | | ||
|
|
||
| Comfy Agent is separate from [Comfy MCP](/agent-tools/mcp) and [Comfy CLI](/agent-tools/cli). MCP connects an external AI client to Comfy. The CLI is for terminal and script workflows. Comfy Agent works in the ComfyUI interface. | ||
|
|
||
| To make your first image, follow [Get started with Comfy Agent](/agent-tools/in-app-agent-installation#make-your-first-image). | ||
|
|
||
| ## Choose which workflow to edit | ||
|
|
||
| Use the workflow selector in the Agent panel to choose the workflow to edit, then send your message. Switching to another tab while the Agent is responding does not redirect its edits. | ||
|
|
||
| The Agent cannot create or switch tabs itself. To work elsewhere, open the other workflow, choose it in the Agent panel, and send a new message. | ||
|
|
||
| The Agent can read another open or saved workflow for comparison without loading it into the current tab. For example: | ||
|
|
||
| > Compare this workflow with my portrait workflow and use the same sampler settings here. | ||
|
|
||
| ## Build and edit workflows | ||
|
|
||
| For a new workflow, open a blank workflow tab and choose it in the Agent panel. The Agent searches for a matching template before building from individual nodes. A template includes a connected graph and an output node. | ||
|
|
||
| For changes to an existing workflow, name the part you want changed and what should stay the same: | ||
|
|
||
| - “Change the prompt to a watercolor landscape. Keep the model and sampler settings.” | ||
| - “Add an upscaling stage after image generation.” | ||
| - “Switch this workflow to Flux using models installed here.” | ||
|
|
||
| ## Validate before running | ||
|
|
||
| The Agent checks node connections, model names, required values, and output nodes before a run. It separates workflow errors from inputs that only you can provide. | ||
|
|
||
| If the workflow is valid but still needs an image, video, or other input, attach the file and ask the Agent to continue. | ||
|
|
||
| ## Work with media | ||
|
|
||
| ## Related | ||
| The Agent can inspect the visible contents and dimensions of a still image. For video, it can read the duration, dimensions, frame rate, and format. For audio, it can read the duration, format, and audio tracks. | ||
|
|
||
| Looking for agent tools that work from Claude, Cursor, Codex, and other MCP clients (outside the ComfyUI canvas)? | ||
| Use **Attach a file** in the message box to add an image or file. You can also drag media into the conversation. | ||
|
|
||
| | Add to the conversation | Ask | Result | | ||
| | --- | --- | --- | | ||
| | An image | “Describe the lighting in this image.” | A description you can use to refine your prompt. | | ||
| | A video | “Trim this to the first five seconds.” | A shortened video file. | | ||
| | Two video clips | “Join clip A followed by clip B.” | One video containing both clips in that order. | | ||
| | A video and an audio file | “Add this audio to the video.” | A video with the supplied audio. | | ||
|
|
||
| Media editing requires the media tools to be available in your installation. In Cloud, edited files are saved to your asset library. The Agent returns a preview or file link in the conversation. | ||
|
|
||
| ## Run permissions | ||
|
|
||
| Choose a run permission next to the message box: | ||
|
|
||
| - **Ask** requests approval before each workflow run. | ||
| - **Auto** runs workflows without asking for each run. | ||
|
|
||
| Select **Save changes** to apply your choice. In Ask mode, select **Run** in the approval message to start the workflow, or **Cancel** to decline that run. | ||
|
|
||
| Ask and Auto control workflow runs. The Agent can edit the selected workflow in either mode. | ||
|
|
||
| ## What the Agent can use | ||
|
|
||
| The Agent uses your message, selected workflow, selected nodes, attachments, and workflow errors. It can also read another workflow when you ask it to compare or reuse settings. | ||
|
|
||
| For Cloud and local data flow, chat history, and privacy controls, see [Comfy Agent data and privacy](/agent-tools/in-app-agent-data-privacy). | ||
|
|
||
| ## Stop or cancel | ||
|
|
||
| Select **Stop** in the message box to stop the Agent's current response. Workflow execution has separate controls in the [run and queue area](/interface/overview); use those to manage a workflow that has already been submitted. | ||
|
|
||
| ## Next steps | ||
|
|
||
| <CardGroup cols={2}> | ||
| <Card title="Comfy MCP" icon="bolt" href="/agent-tools/mcp"> | ||
| Connect external MCP clients to ComfyUI for generation, search, and workflows. | ||
| <Card title="Make your first image" icon="play" href="/agent-tools/in-app-agent-installation#make-your-first-image"> | ||
| Build, run, and revise an image workflow. | ||
| </Card> | ||
| <Card title="Agent Tools overview" icon="robot" href="/agent-tools"> | ||
| Compare the cloud connection, the local connection, and the In-App Agent. | ||
| <Card title="Agent skills" icon="wand-magic-sparkles" href="/agent-tools/in-app-agent-skills"> | ||
| Create and use reusable Agent instructions. | ||
| </Card> | ||
| </CardGroup> | ||
|
|
||
| ## Feedback | ||
|
|
||
| Already in the alpha? Share bugs and impressions: | ||
|
|
||
| <Card title="Send feedback" icon="message" href="https://links.comfy.org/agentalphafeedback"> | ||
| Report bugs or share impressions from the alpha. | ||
| </Card> | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.