Skip to content

Fix clip_skip being ignored for negative prompt embeddings in SDXL pipelines#13973

Open
Dev-X25874 wants to merge 4 commits into
huggingface:mainfrom
Dev-X25874:fix/sdxl-clip-skip-negative-prompt-embeds
Open

Fix clip_skip being ignored for negative prompt embeddings in SDXL pipelines#13973
Dev-X25874 wants to merge 4 commits into
huggingface:mainfrom
Dev-X25874:fix/sdxl-clip-skip-negative-prompt-embeds

Conversation

@Dev-X25874

Copy link
Copy Markdown
Contributor

What does this PR do?

clip_skip was silently ignored when computing negative prompt embeddings
in all three SDXL encode_prompt methods. The positive prompt path
correctly branches on clip_skip:

if clip_skip is None:
    prompt_embeds = prompt_embeds.hidden_states[-2]
else:
    prompt_embeds = prompt_embeds.hidden_states[-(clip_skip + 2)]

but the negative prompt path unconditionally used hidden_states[-2]
regardless of the clip_skip value. This means that whenever clip_skip
is set, positive and negative embeddings are extracted from different hidden
layers, breaking the symmetry that classifier-free guidance relies on.

Fixed by mirroring the existing conditional to the negative path in:

  • pipeline_stable_diffusion_xl.py
  • pipeline_stable_diffusion_xl_img2img.py
  • pipeline_stable_diffusion_xl_inpaint.py

A test is added to test_stable_diffusion_xl.py that calls encode_prompt
with clip_skip=1 vs clip_skip=None and asserts that negative embeddings
differ between the two calls (i.e. clip_skip now actually affects them).

Fixes # (issue)

Before submitting

Who can review?

@yiyixuxu @asomoza

@github-actions github-actions Bot added tests pipelines size/M PR with diff < 200 LOC labels Jun 17, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Hi @Dev-X25874, thanks for the PR! It does not appear to link an issue it fixes. If this PR addresses an existing issue, please add a closing keyword (e.g. Fixes #1234) to the PR description so the issue is linked. See the contribution guide for more details. If this PR intentionally does not fix a tracked issue, a maintainer can add the no-issue-needed label to silence this reminder.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pipelines size/M PR with diff < 200 LOC tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant