Skip to content

Correctly track session elements in dynamic build plan - #2104

Open
abderrahim wants to merge 1 commit into
masterfrom
abderrahim/dynamic-session-elements
Open

Correctly track session elements in dynamic build plan#2104
abderrahim wants to merge 1 commit into
masterfrom
abderrahim/dynamic-session-elements

Conversation

@abderrahim

Copy link
Copy Markdown
Contributor

When using a dynamic build plan, all elements are passed to the queue to be enqueued, but are only actually enqueued when they become required.

This moves the tracking of session elements to the first queue rather than just take everything that we pass to the queue (which would be all elements).

This is only used in the UI AFAICT, and doesn't actually affect the build.

Comment thread src/buildstream/_stream.py
@abderrahim
abderrahim force-pushed the abderrahim/dynamic-session-elements branch from bc72d34 to 4f18986 Compare August 4, 2026 11:39
if not track and not self.queues:
# First non-track queue
queue.set_required_element_check()
queue.set_session_elements(self.session_elements)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is never executed for bst source track, resulting in the frontend indicating 0 session elements.

query_cache marks the CacheQueryQueue (with storage-service) also as track=True. Need to make sure that fixing bst source track won't make the counter wrong in sessions with a CacheQueryQueue.

#
def _enqueue_element(self, element):
if self.session_elements is not None:
self.session_elements.append(element)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

_enqueue_element seems to be called also for retried jobs. I haven't tested it but this might increase the counter too much. Maybe session_elements should be a set instead? (May need to use OrderedDict as the order may be significant).

@abderrahim
abderrahim force-pushed the abderrahim/dynamic-session-elements branch from 4f18986 to bf69297 Compare August 4, 2026 16:51
When using a dynamic build plan, all elements are passed to the queue
to be enqueued, but are only actually enqueued when they become
required.

This moves the tracking of session elements to the first queue rather
than just take everything that we pass to the queue (which would be
all elements).

This also makes session_elements a set rather than a list since an
element can be enqueued more than once (e.g. if it is retried).

This is only used in the UI, and doesn't actually affect the build.
@abderrahim
abderrahim force-pushed the abderrahim/dynamic-session-elements branch from bf69297 to e14c936 Compare August 4, 2026 18:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants