feat(ccwidgets): Implementing Real Time Assistance feature within cc widgets - #722
Conversation
…x#668) Co-authored-by: Shreyas Sharma <shreyassharma9912@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
…ed response feature
|
This pull request is automatically being deployed by Amplify Hosting (learn more). |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 65e4090194
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| @@ -0,0 +1,70 @@ | |||
| { | |||
| "name": "@webex/cc-ai-assistant", | |||
There was a problem hiding this comment.
This introduces a new workspace package/widget, but the change contains no ai-assistant module spec or standing-doc/router update, so the new public surface cannot be routed or validated by the repo's SDD docs. Please add the module spec/standing doc updates in the same change as the code.
AGENTS.md reference: AGENTS.md:L68-L68
Useful? React with 👍 / 👎.
| ); | ||
| } | ||
|
|
||
| if (status === 'listening' || hasFiredInitialRequest) { |
There was a problem hiding this comment.
When the first getRealTimeAssistance call rejects, the hook has already set hasFiredInitialRequest to true and then sets status to error; this branch runs before the error branch, so the panel keeps showing “Listening for information” and never exposes the retry/error message. Please check status === 'error' before the hasFiredInitialRequest listening fallback.
Useful? React with 👍 / 👎.
| oninput={(e: CustomEvent<{value: string}> & {target: HTMLInputElement}) => | ||
| onChange(e.detail?.value ?? e.target?.value ?? '') | ||
| } |
There was a problem hiding this comment.
For the Momentum React Input, the repo's existing usage is the camel-cased onInput; this lowercase oninput prop is not handled by the React wrapper, so typing in the “Add context” field never calls onChange and the controlled value stays empty. That blocks agents from submitting extra context after starting a suggestion session.
Useful? React with 👍 / 👎.
| onClose: 'function', | ||
| onClearChat: 'function', | ||
| onFullScreenToggle: 'function', | ||
| onSuggestionReceived: 'function', |
There was a problem hiding this comment.
For Web Component consumers this is the only arrival callback exposed, but the AI assistant hook only reads and invokes onRealTimeAssistReceived, never onSuggestionReceived. As a result <widget-cc-ai-assistant onSuggestionReceived=...> will not fire when a suggestion arrives; please either expose onRealTimeAssistReceived here or bridge the deprecated prop to the new callback.
Useful? React with 👍 / 👎.
| <label key={widget}> | ||
| <input | ||
| type="checkbox" | ||
| name={widget} | ||
| checked={selectedWidgets[widget]} | ||
| onChange={handleCheckboxChange} | ||
| data-testid={`samples:widget-${widget}`} | ||
| /> |
There was a problem hiding this comment.
Each widget now renders this second label in addition to the original one above it, producing duplicate checkboxes with the same name and data-testid. In WebRTC-locked mode the duplicate also omits the disabled guard, so users can still toggle widgets that the first checkbox correctly disables, and tests querying by test id become ambiguous.
Useful? React with 👍 / 👎.
Shreyas281299
left a comment
Shreyas281299
left a comment
There was a problem hiding this comment.
There are no UTs and SnapshotTesting for new components in cc-components.
| /> | ||
| | ||
| {formatWidgetName(widget)} | ||
| {widget === 'outdialCall' && ( |
There was a problem hiding this comment.
I think this is redundant
| @@ -1,5 +1,16 @@ | |||
| import {Profile} from './store.types'; | |||
|
|
|||
| export const AI_FEATURE_SUGGESTED_RESPONSES_KEY = 'isSuggestedResponsesEnabled'; | |||
There was a problem hiding this comment.
This should be in the constants file
| export const AI_FEATURE_SUGGESTED_RESPONSES_KEY = 'isSuggestedResponsesEnabled'; | ||
|
|
||
| const getValueAtPath = (obj: unknown, path: string): unknown => { | ||
| return path.split('.').reduce<unknown>((acc, segment) => { | ||
| if (acc && typeof acc === 'object' && segment in (acc as Record<string, unknown>)) { | ||
| return (acc as Record<string, unknown>)[segment]; | ||
| } | ||
| return undefined; | ||
| }, obj); | ||
| }; |
There was a problem hiding this comment.
Do we not have correct types for this? we are typecasting and using unknown alot
| const TASK_MULTI_LOGIN_HYDRATE = 'task:multiLoginHydrate'; | ||
| // Mirrored from the SDK's CC_TASK_EVENTS — importing the runtime const breaks | ||
| // our jest transform setup. | ||
| const SUGGESTED_RESPONSE_EVENT = 'SUGGESTED_RESPONSE'; |
There was a problem hiding this comment.
These shoudl be in the constants file
There was a problem hiding this comment.
Moved. I have moved MULTI_LOGIN_HYDRATE as well which is not part of this PR
| task.on(TASK_EVENTS.TASK_PARTICIPANT_LEFT_FAILED, this.refreshTaskList); | ||
| task.on(TASK_EVENTS.TASK_CONFERENCE_TRANSFERRED, this.refreshTaskList); | ||
| task.on(TASK_EVENTS.TASK_CONFERENCE_TRANSFER_FAILED, this.refreshTaskList); | ||
| task.on(TASK_EVENTS.TASK_POST_CALL_ACTIVITY, this.refreshTaskList); |
There was a problem hiding this comment.
Why are we removing this?
There was a problem hiding this comment.
It was already present, please check before task.on(TASK_EVENTS.TASK_CONFERENCE_ESTABLISHING, this.refreshTaskList);
| const getRealTimeAssistance = jest.fn().mockResolvedValue({}); | ||
| return { | ||
| __esModule: true, | ||
| default: { | ||
| cc: { | ||
| apiAIAssistant: { | ||
| getRealTimeAssistance, | ||
| }, | ||
| }, | ||
| clearRealTimeAssist, | ||
| }, | ||
| }; | ||
| }); |
There was a problem hiding this comment.
we should be using mocks from test-fixtures. And use the complete mock cc object
| import {useAiAssistant} from '../src/helper'; | ||
| import store from '@webex/cc-store'; | ||
|
|
||
| jest.mock('@webex/cc-store', () => { |
There was a problem hiding this comment.
[nitpick ]If we can avoid mocking store that would be better.
| onFullScreenToggle, | ||
| onRealTimeAssistReceived, | ||
| }: UseAiAssistantInput) => { | ||
| const [chrome, setChrome] = useState<AIAssistantChromeState>('closed'); |
There was a problem hiding this comment.
Is this chrome googleChrome?? or something new?
github-actions
Bot
removed
the
validated
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 710b6684f2
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| const [pendingRequest, setPendingRequest] = useState(false); | ||
| // ADD_SUGGESTIONS_EXTRA_CONTEXT only makes sense after a GET_SUGGESTIONS has fired. | ||
| const [hasFiredInitialRequest, setHasFiredInitialRequest] = useState(false); | ||
| const [userMessages, setUserMessages] = useState<UserMessage[]>([]); |
There was a problem hiding this comment.
When the mounted widget moves from one task to another, interactionId changes but this hook retains userMessages, contextDraft, hasFiredInitialRequest, and the request/error state from the previous task. Consequently an agent opening the assistant for the next customer can see context entered for the prior interaction and can submit additional context without starting the new session; reset the local session state whenever interactionId changes.
Useful? React with 👍 / 👎.
|
|
||
| const hasEntries = chatEntries.length > 0; | ||
|
|
||
| if (hasEntries) { |
There was a problem hiding this comment.
When any request rejects after hasFiredInitialRequest becomes true, chatEntries already contains the seeded greeting, so this hasEntries branch returns before the error UI below can run. This means even the first failed request still shows only the greeting, and failed follow-up context requests likewise provide no error or retry control. Although the earlier status-ordering issue was changed, the fresh greeting-backed hasEntries path still hides the error.
Useful? React with 👍 / 👎.
| contextDraft, | ||
| hasFiredInitialRequest, | ||
| chatEntries, | ||
| clearChat, |
There was a problem hiding this comment.
The hook returns clearChat, but the top-level component never consumes it and neither the header nor body exposes a control that invokes it. Therefore React and Web Component consumers cannot trigger a reset, onClearChat is a dead public callback, and accumulated suggestions/context can only disappear when the task is removed; pass this action into the presentation layer and wire it to a clear/reset control.
Useful? React with 👍 / 👎.
| </div> | ||
| ); | ||
| })} | ||
| {hasFiredInitialRequest && status !== 'error' ? ( |
There was a problem hiding this comment.
hasFiredInitialRequest && status !== 'error' keeps “Listening for information” visible when status === 'ready'. Please consider showing this only when status === 'listening' (or while pendingRequest is true).
|
|
||
| const hasEntries = chatEntries.length > 0; | ||
|
|
||
| if (hasEntries) { |
There was a problem hiding this comment.
Once the greeting is added, hasEntries stays true, so the error UI never shows. Check status === 'error' before the hasEntries branch, or show the error inside the chat view.
| ); | ||
| return; | ||
| } | ||
| setRequestStatus('listening'); |
There was a problem hiding this comment.
Double-clicking “Get Suggestions” or sending context quickly can start multiple getRealTimeAssistance calls because there’s no in-flight guard. Fix: add if (pendingRequest) return; before setRequestStatus('listening'), or disable Get Suggestions / Send while status === 'listening'.
| node.scrollTop = node.scrollHeight; | ||
| }, [chatEntries.length, status]); | ||
|
|
||
| const contextDisabled = !isFeatureEnabled || !hasFiredInitialRequest; |
There was a problem hiding this comment.
Context input stays enabled while a request is in flight. Consider disabling the input and Send button when status === 'listening'.
There was a problem hiding this comment.
This is addressed in latest changes
| this.store.acceptedCampaignIds = new Set(); | ||
| this.realtimeTranscriptionListeners = {}; | ||
| this.setLastConsultDestination(null); | ||
| this.setLastConsultDestination(null); |
There was a problem hiding this comment.
Duplicate line — please remove one.
| }, 1500); | ||
| } | ||
| } catch (err) { | ||
| console.warn('[AIAssistant] copy to clipboard failed', err); |
There was a problem hiding this comment.
Clipboard copy failure uses console.warn. Per repo logging conventions, please use store.logger (with module/method metadata) instead of console in widget code.
| (event: AIAssistantFeedbackEvent, assist: RealTimeAssistPayload) => { | ||
| const api = store.cc?.apiAIAssistant; | ||
| const adaptiveCardId = assist?.data?.adaptiveCardId; | ||
| if (!interactionId || !agentId || !adaptiveCardId || !api?.sendRealTimeAssistanceUserAction) return; |
There was a problem hiding this comment.
If adaptiveCardId is missing from the payload, the handler returns immediately and the agent gets no indication that like/dislike/copy was not sent. Please log a warning via store.logger or show a small inline error so failed feedback isn’t silent.
| onFeedbackRef.current?.({type: 'copy', actionId}); | ||
| return; | ||
| } | ||
| if (label.includes('dislike')) { |
There was a problem hiding this comment.
Like/dislike icons toggle before sendRealTimeAssistanceUserAction completes. If the SDK call fails, the UI still shows liked/disliked. Consider updating icons only after success, or revert on failure.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 3cea779598
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if (taskId && !this.realTimeAssistListeners[taskId]) { | ||
| const listener = (payload: RealTimeAssistPayload) => this.handleRealTimeAssist(taskId, payload); | ||
| this.realTimeAssistListeners[taskId] = listener; | ||
| task.on(SUGGESTED_RESPONSE_EVENT, listener); |
There was a problem hiding this comment.
When TASK_HYDRATE, TASK_MERGED, or multi-login hydration supplies a new task object for an interaction already present in realTimeAssistListeners, this ID-only guard skips attaching SUGGESTED_RESPONSE to the replacement object even though registerTaskEventListeners attaches the other handlers to it. Suggestions emitted by the active replacement task are therefore silently lost, and removal attempts to detach the callback from the wrong object; track the bound task identity and rebind when it changes.
Useful? React with 👍 / 👎.
| if (len <= lastSeenCountRef.current) return; | ||
| lastSeenCountRef.current = len; | ||
|
|
||
| const latest = realTimeAssist[len - 1]; |
There was a problem hiding this comment.
When multiple assistance events are appended before React commits a render, len can increase by more than one, but this effect advances lastSeenCountRef directly to the new length and invokes onRealTimeAssistReceived only for the final element. The callbacks for the other newly arrived payloads are permanently lost; process every item between the previous count and len before advancing the cursor.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c2c17b0dbd
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| ], | ||
| type: 'asset/resource', | ||
| generator: { | ||
| filename: 'images/[name][ext][query]', |
There was a problem hiding this comment.
In published builds, cc-components is compiled before @webex/cc-ai-assistant and @webex/cc-widgets, so these asset/resource imports have already become runtime strings such as images/like-regular.svg inside cc-components/dist/index.js. The downstream webpack builds consume that prebuilt JavaScript and therefore never see or copy the original SVG modules into their own dist directories; hosts using either published widget package will request page-relative images/*.svg URLs that are absent, leaving the like/dislike/copy and source controls without visible icons. Inline these assets or explicitly propagate the emitted image directory through the downstream package builds.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c2e29cf926
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| actionTimeStamp: sentAt, | ||
| ...(context ? {context} : {}), | ||
| }); | ||
| setHasInitialRequestSucceeded(true); |
There was a problem hiding this comment.
When interactionId changes while this request is awaiting the SDK, the reset effect clears the new session, but the old promise can subsequently execute this setter—or the error setters in catch—against that new session. The next customer can therefore see the previous request's greeting/error and gain the context controls without starting their own request; additionally, the old finally can clear the in-flight guard for a newer request. Capture an interaction/request generation and discard completions that no longer match it.
Useful? React with 👍 / 👎.
| {name: 'widget-cc-call-control-cad', component: WebCallControlCAD}, | ||
| {name: 'widget-cc-realtime-transcript', component: WebRealTimeTranscript}, | ||
| {name: 'widget-cc-digital-channels', component: WebDigitalChannels}, | ||
| {name: 'widget-cc-ai-assistant', component: WebAIAssistant}, |
There was a problem hiding this comment.
Although the new AI-assistant module spec and router entry were added, this public custom element remains absent from ai-docs/CONTRACTS.md, while cc-widgets-spec.md still enumerates only the previous exports/tags and cc-components-spec.md:124 still asserts exactly 11 exported components. This leaves the repository's canonical public-surface and touched-module standing docs inconsistent with the code; update those documents in this same change.
AGENTS.md reference: AGENTS.md:L68-L68
Useful? React with 👍 / 👎.
| } | ||
|
|
||
| const FeatureItem: React.FC<FeatureItemProps> = ({description, icon, title}) => ( | ||
| <li className="ai-assistant__landing-feature"> |
There was a problem hiding this comment.
This should be List from momentum-design
| @@ -0,0 +1,9 @@ | |||
| declare module '*.svg' { | |||
There was a problem hiding this comment.
Why is this required?
There was a problem hiding this comment.
WE are getting an erro as below:
error TS2307: Cannot find module '@momentum-design/icons/dist/svg/arrow-down-regular.svg'
or its corresponding type declarations.
This is becasue how we are importing icons from momentum-design.
I looked into using IconNext as well but that was not possible hence this change is needed for now
github-actions
Bot
removed
the
validated
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c76032a087
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| setIsAIAssistantFullScreen(isFs); | ||
| console.log('AIAssistant fullScreen', isFs); | ||
| }} | ||
| onRealTimeAssistReceived={(payload) => console.log('AIAssistant suggestion', payload)} |
There was a problem hiding this comment.
When the sample enables AI Assistant, every received payload is written to the browser console; these payloads contain suggestion text, adaptive-card content, and potentially customer statements derived from the active interaction. This exposes contact-center PII through logs and also teaches integrators an unsafe callback pattern, so log only a non-sensitive event marker or omit this callback.
AGENTS.md reference: AGENTS.md:L67-L67
Useful? React with 👍 / 👎.
| useEffect(() => { | ||
| lastSeenCountRef.current = 0; |
There was a problem hiding this comment.
When the selected task changes A → B → A, this resets the single cursor to zero even though store.realTimeAssist[A] still contains A's previously delivered payloads. The response effect then treats the entire retained array as unseen and invokes onRealTimeAssistReceived again without a new SDK event, causing hosts that mirror the callback to duplicate suggestions; retain counts by interaction or initialize the cursor so previously delivered entries are not replayed.
Useful? React with 👍 / 👎.
| const close = useCallback(() => { | ||
| setChrome('closed'); | ||
| setIsFullScreen(false); | ||
| onClose?.(); |
There was a problem hiding this comment.
If a host controls its fullscreen layout from onFullScreenToggle, closing the widget while fullscreen silently resets the internal flag but never emits false. The host therefore remains in fullscreen layout while the widget has returned to its launcher, and reopening leaves the host and widget states inconsistent unless every consumer independently duplicates the sample app's onClose workaround; emit the fullscreen exit when close clears this state.
Useful? React with 👍 / 👎.
Kesari3008
added
the
run_e2e
Thank you for your contribution! |
5 participants
COMPLETES #https://jira-eng-sjc12.cisco.com/jira/browse/CAI-8037
This pull request addresses
Implementing Real Time Assistant Widget within existing cc widgets
by making the following changes
Added new widget: AI Assiatnt under which real time assistance feature is implemented
Change Type
The following scenarios were tested
https://app.vidcast.io/share/7db25f78-68f5-42a5-94cc-360b4e0aebc3
The GAI Coding Policy And Copyright Annotation Best Practices
Checklist before merging
Make sure to have followed the contributing guidelines before submitting.