Skip to content

针对#issues6086导出整合包时内存溢出修复#6173

Open
2012hzy wants to merge 20 commits into
HMCL-dev:mainfrom
2012hzy:fix-export-modpack-oom
Open

针对#issues6086导出整合包时内存溢出修复#6173
2012hzy wants to merge 20 commits into
HMCL-dev:mainfrom
2012hzy:fix-export-modpack-oom

Conversation

@2012hzy

@2012hzy 2012hzy commented Jun 14, 2026

Copy link
Copy Markdown

问题:导出大型整合包时,HMCL 因 OutOfMemoryError 卡死(见 #6086)。
原因McbbsModpackExportTaskModrinthModpackExportTask 一次性将所有文件信息加载到内存生成 JSON,导致内存峰值过高
修复:改用 JsonWriter 流式写入临时文件,避免内存积累,并消除 ModrinthModpackExportTask 中重复的 API 调用和 I/O
测试:已完成编译测试,本地通过且功能正常
关联Fixes #6086

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors both McbbsModpackExportTask and ModrinthModpackExportTask to stream JSON output directly using JsonWriter instead of constructing in-memory manifest objects. The review feedback highlights critical issues, including potential resource leaks from unclosed Files.walk streams and a potential NullPointerException when handling nullable Java versions. Additionally, the reviewer suggests refactoring repetitive loader version checks into loops to improve code readability and maintainability.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment thread HMCLCore/src/main/java/org/jackhuang/hmcl/mod/mcbbs/McbbsModpackExportTask.java Outdated
Comment thread HMCLCore/src/main/java/org/jackhuang/hmcl/mod/mcbbs/McbbsModpackExportTask.java Outdated
Comment thread HMCLCore/src/main/java/org/jackhuang/hmcl/mod/mcbbs/McbbsModpackExportTask.java Outdated
@2012hzy

2012hzy commented Jun 14, 2026

Copy link
Copy Markdown
Author

/gemini review

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors both McbbsModpackExportTask and ModrinthModpackExportTask to write modpack manifests directly to temporary files using JsonWriter instead of serializing in-memory manifest objects. Feedback on these changes highlights several critical serialization bugs in the MCBBS export task, such as incorrect JSON keys (e.g., origins instead of origin, pluralized launch arguments, and supportedJavaVersions instead of supportJava), an incorrect type discriminator value, and missing default settings. Additionally, the reviewer notes that throwing a RuntimeException on remote file processing failures in the Modrinth export task could prematurely abort the export, and suggests refactoring ifPresent lambdas to standard loops or conditionals to avoid boilerplate try-catch blocks and RuntimeException wrapping.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment thread HMCLCore/src/main/java/org/jackhuang/hmcl/mod/mcbbs/McbbsModpackExportTask.java Outdated
Comment thread HMCLCore/src/main/java/org/jackhuang/hmcl/mod/mcbbs/McbbsModpackExportTask.java Outdated
Comment thread HMCLCore/src/main/java/org/jackhuang/hmcl/mod/mcbbs/McbbsModpackExportTask.java Outdated
Comment thread HMCLCore/src/main/java/org/jackhuang/hmcl/mod/mcbbs/McbbsModpackExportTask.java Outdated
Comment thread HMCLCore/src/main/java/org/jackhuang/hmcl/mod/mcbbs/McbbsModpackExportTask.java Outdated
@2012hzy

2012hzy commented Jun 14, 2026

Copy link
Copy Markdown
Author

/gemini review

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors the MCBBS and Modrinth modpack export tasks to stream JSON generation using JsonWriter to a temporary file, reducing memory overhead compared to the previous in-memory GSON serialization. However, several issues were identified in the review: using Files.walk instead of Files.walkFileTree introduces a severe performance regression by traversing blacklisted directories; writing explicit null values for optional fields like fileApi and summary may cause compatibility issues with strict parsers; and writing an empty origin array in the MCBBS manifest discards user-configured origin data.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment thread HMCLCore/src/main/java/org/jackhuang/hmcl/mod/mcbbs/McbbsModpackExportTask.java Outdated
Comment thread HMCLCore/src/main/java/org/jackhuang/hmcl/mod/mcbbs/McbbsModpackExportTask.java Outdated
@2012hzy

2012hzy commented Jun 14, 2026

Copy link
Copy Markdown
Author

/gemini review

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors both McbbsModpackExportTask and ModrinthModpackExportTask to stream manifest and index JSON generation directly to a temporary file using JsonWriter instead of constructing large in-memory manifest objects. It also utilizes Files.walkFileTree for directory traversal. A critical issue was identified in ModrinthModpackExportTask where tryGetRemoteFile can throw an IOException that is no longer caught within the file visitor, which could cause the entire export task to fail on a single file error instead of logging a warning and continuing.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

@2012hzy

2012hzy commented Jun 14, 2026

Copy link
Copy Markdown
Author

/gemini review

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors the MCBBS and Modrinth modpack export tasks to stream JSON metadata directly to temporary files using JsonWriter instead of building full in-memory manifest objects, which optimizes memory usage. It also replaces stream-based file walking with Files.walkFileTree. Feedback on these changes includes addressing a potential NullPointerException in McbbsModpackExportTask when iterating over tokenized launch and Java arguments, and optimizing the whitelist lookup in ModrinthModpackExportTask by converting the whitelist List to a HashSet to avoid O(N) performance overhead during directory traversal.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

@2012hzy

2012hzy commented Jun 14, 2026

Copy link
Copy Markdown
Author

/gemini review

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors the MCBBS and Modrinth modpack export tasks to stream JSON manifests directly to temporary files using JsonWriter instead of building large in-memory objects. It also updates directory traversal to use Files.walkFileTree. The review feedback highlights critical issues in both export tasks where directory traversal (preVisitDirectory and zip.putDirectory) prematurely skips subtrees when a whitelist is present, because directory paths themselves are not in the whitelist. The reviewer recommends allowing traversal of non-blacklisted directories and applying the whitelist filter only to files.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

@2012hzy

2012hzy commented Jun 14, 2026

Copy link
Copy Markdown
Author

/gemini review

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors McbbsModpackExportTask and ModrinthModpackExportTask to stream their respective manifest files (mcbbs.packmeta and modrinth.index.json) directly to temporary files using JsonWriter instead of holding the entire manifest structures in memory. This is achieved by utilizing Files.walkFileTree to write files dynamically. A critical issue was identified in ModrinthModpackExportTask where enabling a disabled mod renames the file on disk but does not update the file reference, which can lead to a NoSuchFileException during subsequent operations.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

@2012hzy

2012hzy commented Jun 14, 2026

Copy link
Copy Markdown
Author

/gemini review

@gemini-code-assist

Copy link
Copy Markdown
Contributor

你好!感谢你提交的关于修复大型整合包导出时 OutOfMemoryError 的 PR。这是一个非常棒的改进,通过使用 JsonWriter 进行流式写入,确实能有效降低内存占用。

不过,我注意到你在评论区多次重复输入了 /gemini review 指令。这可能会导致系统重复触发审核流程。如果你已经提交了代码并希望我进行审核,请耐心等待,或者你可以直接在 PR 中指出你希望我重点关注的代码部分。

目前我已经根据你提供的补丁进行了分析,你的修复方案在逻辑上是正确的,使用了 JsonWriterFiles.walkFileTree 来避免一次性加载所有文件信息到内存中,这非常符合处理大型数据集的最佳实践。代码结构清晰,且通过 finally 块确保了临时文件的清理,处理得非常稳妥。

如果后续有其他问题或需要进一步的反馈,请随时告知。

@2012hzy

2012hzy commented Jun 14, 2026

Copy link
Copy Markdown
Author

核心维护者们好,PR 中 CI 检查已全部通过。为了将 #6086 这个影响用户的 Bug 修复,这边的所有修改均已完成。请问是否还有其他地方需要调整?谢谢!

@2012hzy 2012hzy closed this Jun 15, 2026
@2012hzy 2012hzy reopened this Jun 15, 2026
@ToobLac

ToobLac commented Jun 18, 2026

Copy link
Copy Markdown
Contributor

核心维护者们好,PR 中 CI 检查已全部通过。为了将 #6086 这个影响用户的 Bug 修复,这边的所有修改均已完成。请问是否还有其他地方需要调整?谢谢!

等一等,glavo这几天在赶其他功能的进度,过一阵应该会来审pr

@2012hzy

2012hzy commented Jun 20, 2026

Copy link
Copy Markdown
Author

核心维护者们好,PR 中 CI 检查已全部通过。为了将 #6086 这个影响用户的 Bug 修复,这边的所有修改均已完成。请问是否还有其他地方需要调整?谢谢!

等一等,glavo这几天在赶其他功能的进度,过一阵应该会来审pr

他是不是没看到?

@Minecraft269

Copy link
Copy Markdown

我来审审

@2012hzy

2012hzy commented Jun 20, 2026

Copy link
Copy Markdown
Author

/gemini review

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors the MCBBS and Modrinth modpack export tasks to stream manifest and index JSON data directly to temporary files using JsonWriter, preventing potential OutOfMemoryError issues on large modpacks. It also introduces translation keys for Java version warnings and manual updates, and ensures that temporarily enabled mods are properly restored to their disabled state after export. The feedback suggests removing redundant and potentially leaky deleteOnExit() calls on the temporary files, as they are already safely deleted in finally blocks.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment thread HMCLCore/src/main/java/org/jackhuang/hmcl/mod/mcbbs/McbbsModpackExportTask.java Outdated
@2012hzy

2012hzy commented Jun 20, 2026

Copy link
Copy Markdown
Author

/gemini review

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors the MCBBS and Modrinth modpack export tasks to stream manifest and index JSON files directly to temporary files using JsonWriter, significantly reducing memory usage and preventing OutOfMemoryError on large modpacks. It also adds translation keys for unsupported Java versions and manual updates. The review feedback points out a potential issue in ModrinthModpackExportTask where modifying the directory structure (enabling disabled mods) during Files.walkFileTree traversal can cause undefined behavior or exceptions, and suggests collecting the file paths first before processing them.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

@2012hzy

2012hzy commented Jun 20, 2026

Copy link
Copy Markdown
Author

/gemini review

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces localized boot properties and refactors both McbbsModpackExportTask and ModrinthModpackExportTask to stream manifest and index JSON files directly to temporary files using JsonWriter. This optimization avoids holding large amounts of file metadata in memory, preventing potential OutOfMemoryError issues on large modpacks. Additionally, the export task now temporarily enables disabled mods and restores them afterward. The review feedback suggests defensive handling of a potential NullPointerException when retrieving the whitelist, and recommends utilizing the ModManager API (enableMod and disableMod) rather than manual file operations to manage and restore the state of disabled mods consistently.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors the MCBBS and Modrinth modpack export tasks to stream manifest and index JSON files directly to temporary files using JsonWriter, preventing OutOfMemoryErrors on large modpacks. It also adds localization files for the boot module. The review feedback points out two critical issues: first, the new translation files are incorrectly placed in the 'bin/main/' directory, which is prone to being deleted during clean builds; second, using relative paths when enabling or disabling mods in ModrinthModpackExportTask could fail if the working directory differs from the game directory, so absolute paths should be used instead.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment thread HMCLBoot/bin/main/assets/lang/boot.properties
@2012hzy

2012hzy commented Jun 20, 2026

Copy link
Copy Markdown
Author

代码写多了啥啥都容易错😶

@2012hzy

2012hzy commented Jun 20, 2026

Copy link
Copy Markdown
Author

/gemini review

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request adds bootloader localization files and refactors the MCBBS and Modrinth modpack export tasks to stream manifest and index JSON files directly to temporary files using JsonWriter, which prevents OutOfMemoryErrors on large modpacks. Feedback on the Modrinth export task identifies a critical issue where temporarily enabled mods are restored only in the finally block, leading to incorrect packaging and duplication in the final zip; a fix is suggested to restore these mods before zipping.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

@2012hzy 2012hzy requested a review from Minecraft269 June 20, 2026 14:36
@Minecraft269

Copy link
Copy Markdown

a?

@Minecraft269

Copy link
Copy Markdown

让我审吗?
....
那行吧

@Minecraft269

Minecraft269 commented Jun 20, 2026

Copy link
Copy Markdown

ClaudeCode启动!

但是先等我把我的skills包整好

@2012hzy

2012hzy commented Jun 20, 2026

Copy link
Copy Markdown
Author

ClaudeCode启动!

但是先等我把我的skills包整好

我代码能力稍弱,还请指教

@Minecraft269

Copy link
Copy Markdown

对了 上次审查吃掉我的上下文:
image

@2012hzy

2012hzy commented Jun 20, 2026

Copy link
Copy Markdown
Author

对了 上次审查吃掉我的上下文: image

实在抱歉……我java实在不太行,偶尔还要查AI

@Minecraft269 Minecraft269 left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

审查概要

🤖 审查模型: DeepSeekV4

本次 PR 将整合包导出的内存模型从「全量加载到内存再序列化」改为「JsonWriter 流式写入临时文件」,从根本上解决了大型整合包的 OOM 问题。设计方向正确,流式写入是解决此类问题的标准方案。

总体评价

  • ✅ 核心方案正确 — JsonWriter 流式写入是合适的选择
  • ✅ Javadoc 清晰说明了设计权衡
  • ModrinthModpackExportTask 修复了 disabled mod 的路径问题(enableMod 现在使用绝对路径)
  • whitelist null 安全性改进
  • ⚠️ 有几处需要注意的问题(详见行内评论)
  • 🚨 1 个需要立即修复的问题

变更概要

文件 变更 审查重点
McbbsModpackExportTask.java +174/-59 流式 JSON + 两遍遍历
ModrinthModpackExportTask.java +168/-71 流式 JSON + disabled mod 恢复
4 个 boot 语言文件 新增 i18n 内容完整性
git (空文件) 意外新增 🚨 应移除

行内评论汇总

严重度 文件 问题
🚨 Critical git 意外提交的空文件,必须移除
⚠️ Important ModrinthModpackExportTask.java:196 路径分隔符不一致导致 Windows 上 remoteFilePaths 匹配失效
⚠️ Important McbbsModpackExportTask.java:118 info.getUrl() 等字段缺少 null 检查
ℹ️ Info boot_es.properties:19 缺少 boot.manual_updateboot.message.error
ℹ️ Info ModrinthModpackExportTask.java:170 processedPaths 去重在单目录遍历中冗余
ℹ️ Info McbbsModpackExportTask.java:144 Files.isRegularFile() 检查被移除,符号链接行为变更

额外发现(已有代码,非本次引入)

ModrinthModpackExportTask.tryGetRemoteFile()(int) fileSize 的警告信息写的是 "precision may be lost",但实际上 (int) 转换对于 >2.1GB 的文件是整数溢出,会产生无意义的错误值,不是精度丢失。建议改为 long 类型或溢出时跳过该文件。(此问题在 PR 之前已存在,建议单独修复)

# along with this program. If not, see <https://www.gnu.org/licenses/>.
#

boot.unsupported_java_version=HMCL requiere Java 17 o posterior para ejecutarse, pero sigue admitiendo el inicio de juegos con Java 6~16.\nPor favor, instala la última versión de Java e intenta ejecutar HMCL de nuevo.\nPuedes conservar tu versión antigua de Java. HMCL puede detectar y gestionar múltiples instalaciones de Java,\ny seleccionará automáticamente la versión de Java adecuada para tu juego.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ℹ️ 缺少 boot.manual_updateboot.message.error

其他三个语言文件(boot.properties, boot_zh.properties, boot_zh_Hant.properties)都包含这三个键:

  • boot.unsupported_java_version
  • boot.manual_update
  • boot.message.error

boot_es.properties(西班牙语)仅包含 boot.unsupported_java_version,缺少后两个。如果是刻意为之(尚未翻译),建议添加带英文 fallback 的占位符;如果是遗漏,请补全。

boot.manual_update=HMCL no puede completar las actualizaciones automáticas en el entorno actual. Por favor, descarga la última versión de HMCL manualmente.\n¿Deseas ir a la página de descarga?
boot.message.error=Error


writer.name("formatVersion").value(1);
writer.name("game").value("minecraft");
writer.name("versionId").value(info.getVersion());

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ℹ️ processedPaths 去重逻辑在单目录遍历中是冗余的

在单次 Files.walkFileTree() 遍历中,visitFile() 不会被重复调用同一个文件。processedPaths 的检查仅在以下情况下有用:

  • 同一个文件通过不同的路径(hardlink/symlink)被访问
  • 或者将来有人在此循环中增加了其他遍历逻辑

这是无害的防御性编程,但增加了不必要的 HashSet 内存开销和 contains/add 操作。如果 resourceDirsmods, resourcepacks, shaderpacks)之间没有重叠,可以考虑移除以简化代码。

Comment thread git Outdated
@Minecraft269

Copy link
Copy Markdown

这次审查好像没用Subagent 审查质量可能会低一点 本次审查使用的上下文:
image

@2012hzy 2012hzy closed this Jun 21, 2026
@2012hzy 2012hzy reopened this Jun 21, 2026
@2012hzy

2012hzy commented Jun 21, 2026

Copy link
Copy Markdown
Author

非常抱歉因为频繁推送给大家带来了困扰。所有审查意见已经修复完毕,后续我会注意减少不必要的更新。感谢各位的耐心

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] 导出整合包时内存溢出且HMCL卡死,无任何提示

4 participants