Skip to content

Refactor OpenVINO backend: move and clean up parameter handling#240

Open
zhaixuejun1993 wants to merge 6 commits into
ravi9:dev_backend_openvinofrom
zhaixuejun1993:xuejun/refacter-v0
Open

Refactor OpenVINO backend: move and clean up parameter handling#240
zhaixuejun1993 wants to merge 6 commits into
ravi9:dev_backend_openvinofrom
zhaixuejun1993:xuejun/refacter-v0

Conversation

@zhaixuejun1993

Copy link
Copy Markdown
Collaborator

This pull request refactors how model inputs and extra inputs are handled in the GGML-to-OpenVINO translation layer, improving type safety and code clarity. Instead of storing model inputs and extra inputs as generic OpenVINO nodes, the code now uses explicit structures (ModelInputInfo and ModelExtraInputInfo) to describe their properties. This change also centralizes the logic for creating OpenVINO parameters and constants, and updates related interfaces and usages throughout the codebase. Additionally, the codebase is cleaned up with improved documentation and removal of unused utilities.

Key changes:

Refactoring of Model Input Handling:

  • Replaces storage of model inputs and extra inputs from std::shared_ptr<ov::Node> to new structures ModelInputInfo and ModelExtraInputInfo, providing explicit type and shape information. (ggml-decoder.h, openvino/decoder.h, ggml-decoder.cpp, [1] [2] [3] [4] [5] [6] [7]

  • Updates the get_model_inputs() and get_model_extra_inputs() interfaces to return the new info structures, and removes the now-unnecessary get_model_extra_input_values() method. (ggml-decoder.h, openvino/decoder.h, [1] [2]

Centralized Parameter and Constant Creation:

  • Introduces helper functions create_parameter and create_extra_input to consistently create OpenVINO parameter and constant nodes from the new info structures. Updates the translation session to use these helpers when constructing the OpenVINO model. (openvino/translate_session.cpp, [1] [2]

API and Utility Improvements:

  • Updates get_ov_input_tensor to use the new ModelExtraInputInfo structure for constructing input tensors. (utils.cpp, ggml/src/ggml-openvino/utils.cppL840-R843)
  • Adds or improves documentation for several utility functions and classes, making the codebase easier to understand and maintain. (openvino/translate_session.h, openvino/utils.h, [1] [2] [3]

Cleanup and Removal of Unused Code:

  • Removes unused or redundant utility functions such as getCurrentTime, non_cont_dim, and various template utilities from utils.cpp and utils.h. (utils.cpp, utils.h, [1] [2]

Minor Fixes:

Additional information

Requirements

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant