ImplicitServer emits an exclusive Array.end, breaking multi-chunk implicit transfers

[chrislupp]

Summary

ImplicitServer writes Array.end as an exclusive index, while ExplicitServer and both clients treat it as inclusive. Any implicit variable large enough to span more than one chunk fails to decode — including Python client → Python server.

Reproduction

Python-only, no other implementation involved. A 5-element variable with num_double=2 (3 chunks):

from concurrent import futures
import numpy as np, grpc, philote_mdo.general as pmdo
import philote_mdo.generated.data_pb2 as data

class VecImplicit(pmdo.ImplicitDiscipline):
    def setup(self):
        self.add_input("a", shape=(5,))
        self.add_output("x", shape=(5,))
    def compute_residuals(self, inputs, outputs, residuals):
        residuals["x"] = outputs["x"] - inputs["a"]
    def solve_residuals(self, inputs, outputs):
        outputs["x"] = inputs["a"]

server = grpc.server(futures.ThreadPoolExecutor(max_workers=4))
pmdo.ImplicitServer(discipline=VecImplicit()).attach_to_server(server)
server.add_insecure_port("[::]:50077"); server.start()

client = pmdo.ImplicitClient(channel=grpc.insecure_channel("localhost:50077"))
client._stream_options = data.StreamOptions(num_double=2)   # 5 values -> 3 chunks
client.send_stream_options()
client.run_setup(); client.get_variable_definitions(); client.get_partials_definitions()

print(client.run_solve_residuals({"a": np.array([1.0, 2.0, 3.0, 4.0, 5.0])}))

Actual:

ValueError: could not broadcast input array from shape (2,) into shape (3,)

Expected: {'x': array([1., 2., 3., 4., 5.])}

The identical discipline and chunk size through ExplicitServer works correctly and returns the right values, which isolates the cause to the implicit server's index encoding.

Cause

get_chunk_indices yields half-open (b, e) pairs, so a chunk carries e - b values.

explicit_server.py converts to the inclusive wire convention:

# philote_mdo/general/explicit_server.py:88 (also :137)
start=b,
end=e - 1,
data=value.ravel()[b:e],

implicit_server.py does not:

# philote_mdo/general/implicit_server.py:192 (also :259, :333)
start=b,
end=e,                      # <-- exclusive; should be e - 1
data=value.ravel()[b:e],    # carries e - b values

Both clients decode as inclusive:

# philote_mdo/general/discipline_client.py:329 (also :367, :412)
e = arr.end + 1

So the client computes a destination slice of end - b + 1 elements for a payload of end - b.

Why this hasn't shown up before

Single-chunk arrays survive by accident. For a whole-array chunk b=0, e=size, the client slices [0 : size + 1], and NumPy silently clips that to [0 : size] — which happens to match the payload length. Every implicit variable in examples/ and the test suite is a scalar, so the bug is invisible there. It appears as soon as a variable spans two or more chunks.

Suggested fix

implicit_server.py:192, :259, :333 — change end=e to end=e - 1, matching the explicit server.

A regression test with a variable larger than num_double would cover it; the snippet above works as-is.

Related

Array.end is inclusive per the standard, so the explicit server is correct and the implicit server is the outlier. Found while implementing Philote-Rust against the same proto — a Rust client validates the chunk length strictly, so it rejects implicit responses from this server rather than relying on the NumPy clipping that masks the bug in the single-chunk case.

Environment: Philote-Python 0.8.0 @ 1f3baf6, proto v0.8.0.