Graph analysis: god nodes, Leiden communities, layering, integration surface

[ako]

Summary

Add architecture/dependency-graph analysis on top of the catalog refs graph, to help understand an app and decide where to intervene (refactoring). Proposal: docs/11-proposals/PROPOSAL_graph_analysis.md.

Targets two refactoring journeys:

1. Spaghetti → layered/modular — dependency cycles + topological layer numbers + community groupings.
2. Monolith → multi-app — coarse communities = candidate apps; the cross-community "integration surface" classifies each crossing edge into the OData/REST/event contract a split would require.

Scope

• mxcli graph-report + CATALOG.graph_* views (god nodes, module coupling/cohesion, dead docs, refkind distribution, entity hotspots).
• Pure-Go graph engine (no CGO/deps): Leiden communities, Tarjan cycles, topological layering, PageRank, betweenness — refresh catalog communities [resolution n].
• SHOW COMMUNITIES / SHOW COMMUNITY [MEMBERS] OF.
• Starlark linter builtins (community_of, layer_of, cycles, module_dependencies, centrality, god_nodes, integration_surface, refs_from) so teams enforce their own architecture policy.

Status

Implemented on branch `misc` (Parts 1 & 2 of the proposal), validated against the `leidenalg` reference (105 communities, exact match) and live test apps; skills + docs-site added. Pending PR/merge.

Deferred follow-ons (out of scope here)

• Expression/XPath-embedded edges (entities/attributes/enums/constants referenced inside constraints/expressions) — gated behind expression type checking (Expression type checking for mxcli check (microflows + nanoflows) #580); enum/constant nodes fall out of it.
• Agent-context subgraph — a token-minimal query-scoped subgraph served as an MCP tool (the "71.5x" Graphify pitch) — a sibling proposal.
• Attribute-level graph nodes.

[github-actions]

Issue Triage

Summary:
The issue proposes adding graph‑analysis capabilities to mxcli (community detection, layering, integration‑surface analysis, Starlark linter builtins) to help users refactor Mendix apps from spaghetti to layered/modular or from monolith to multi‑app architectures.

Checklist for feature request completeness

Required info	Present?	Notes
Use case – why the feature is needed / what problem it solves	✅	Described as two refactoring journeys (spaghetti → layered/modular; monolith → multi‑app).
Proposed syntax or behavior (if applicable)	✅	Lists new commands (mxcli graph-report, SHOW COMMUNITIES, etc.), catalog views, and Starlark builtins.
Mendix version relevance (if version‑specific)	⚠️	Not explicitly mentioned; the feature appears to be version‑agnostic, so this is not required.

Conclusion:
All required information for a feature request is present. Thank you for the detailed proposal! 🙌

---

Automated triage via OpenRouter — workflow source

[github-actions]

Enhancement Implementation Guide

Completeness Assessment

The issue provides a clear summary and links to a detailed proposal (docs/11-proposals/PROPOSAL_graph_analysis.md). However, to implement this feature correctly, we need:

1. Mendix version compatibility: The graph analysis features likely depend on catalog data that requires a full catalog refresh. We need to know the minimum Mendix version where the necessary refs data is available (likely 10.18+ for v2 MPR format, but should confirm).
2. Example syntax validation: The proposal mentions SHOW COMMUNITIES and refresh catalog communities [resolution n], but we should verify exact syntax against MDL design guidelines (e.g., should it be REFRESH CATALOG COMMUNITIES or REFRESH CATALOG COMMUNITY DETECTION?).
3. Starlark builtin signatures: Need concrete examples of how linter builtins like community_of(module) or integration_surface(edge_type) would be used in policy files.

Questions to ask reporter:

• What is the minimum Mendix version required for this feature? (Catalog refs graph availability)
• Can you share the exact proposed syntax from docs/11-proposals/PROPOSAL_graph_analysis.md for:
  • REFRESH CATALOG COMMUNITIES command
  • SHOW COMMUNITIES output format
  • Starlark builtin function signatures
• Should the graph analysis work on the existing refs catalog table, or does it require a new catalog table?

Overlap Check

Existing features that partially cover this:

• SHOW CATALOG REFS already exposes the raw refs graph (source/destination edges)
• mxcli report includes some architectural metrics (but not graph-based)
• Starlark linter already has module_dependencies builtin (in mdl/linter/starlark/go.go)
• SHOW STRUCTURE shows module coupling at a high level
• However, no existing feature provides:
  • Community detection (Leiden algorithm)
  • Topological layering for cycle resolution
  • Integration surface analysis for split planning
  • God node identification via centrality metrics
  • Cross-community edge classification

Proposed MDL Syntax

Evaluating against MDL design guidelines (reads as English, standard verbs, qualified names):

Approved syntax from proposal (needs validation):

-- Graph analysis commands
REFRESH CATALOG COMMUNITIES [resolution 1.0];
SHOW COMMUNITIES;
SHOW COMMUNITY [MEMBERS] OF <qualified_name>;
SHOW COMMUNITY <id> MEMBERS;

Evaluation:

• ✅ REFRESH CATALOG follows existing pattern (like REFRESH CATALOG FULL)
• ❌ SHOW COMMUNITIES should be LIST COMMUNITIES per MDL guideline (new commands use list, not show)
• ❌ SHOW COMMUNITY ... should be DESCRIBE COMMUNITY ... for inspection
• ✅ Uses qualified names where needed (OF <qualified_name>)
• ❌ Missing property list syntax for configuration (should allow (resolution: 1.0) per MDL property format)

Revised MDL syntax (following guidelines):

-- Create/compute communities (alter-like operation on catalog)
ALTER CATALOG SET COMMUNITIES (resolution: 1.0);

-- Inspect communities
DESCRIBE COMMUNITIES;
DESCRIBE COMMUNITY <id> [MEMBERS];
DESCRIBE COMMUNITY OF <qualified_name>;

-- Starlark linter builtins (examples)
deny if community_of($module) == community_of($imported_module) and cyclic_dependency($module, $imported_module);
deny if layer_of($module) < layer_of($imported_module) and not allowed_downstream($imported_module);
deny if god_node($module) and not exempted($module);
allow if integration_surface($edge) in {"odata", "rest"} and split_safe($edge);

Implementation Roadmap

Following the full-stack pipeline (no new document types, so skip AST/visitor/SDK/writer steps):

1. Catalog Extension (mdl/catalog/):

   • Add Community struct in types.go (id, module, layer, centrality_score)
   • Extend Catalog interface with:

     SetCommunities(resolution float64) error
     Communities() []Community
     CommunityOf(module QualifiedName) (*Community, error)
     MembersOfCommunity(id int64) []QualifiedName
     IntegrationSurface() []IntegrationEdge // {source, target, contract_type}

   • Implement in mdl/backend/mpr/catalog.go:
     • Use pure-Go graph library (e.g., github.com/yourbasic/graph) for:
       • Leiden community detection (resolution parameter)
       • Tarjan's strongly connected components (cycles)
       • Topological layering (after removing cycles)
       • PageRank/betweenness centrality
     • Store results in catalog tables (new catalog_communities, catalog_community_members)

2. CLI Command (cmd/mxcli/):

   • Add graphReportCmd in cmd_mxcli.go (cobra command)
   • Implement alterCatalogCommunitiesCmd for ALTER CATALOG SET COMMUNITIES
   • Implement describeCommunitiesCmd for DESCRIBE COMMUNITIES variants
   • Output formatting: tables for communities, JSON for machine parsing

3. Starlark Builtins (mdl/linter/starlark/):

   • Add to go.go:

     "community_of": func(module string) (int64, error) { ... },
     "layer_of": func(module string) (int, error) { ... },
     "cycles": func() [][]string { ... }, // returns list of cycles
     "god_nodes": func(threshold float64) []string { ... }, // modules with centrality > threshold
     "integration_surface": func(edgeType string) [][]string { ... }, // edges crossing communities
     "refs_from": func(module string) []string { ... }, // direct dependencies

   • Backend must expose these via Catalog interface methods

4. Backend Abstraction (mdl/backend/):

   • Extend CatalogBackend interface with new methods (Step 1)
   • Add mock stubs in mdl/backend/mock/ (e.g., CommunitiesFunc)
   • Ensure MPR implementation satisfies interface

5. Documentation:

   • Update docs/01-project/MDL_QUICK_REFERENCE.md with new syntax
   • Add skill: .claude/skills/analyze-architecture.md (usage examples, gotchas)
   • Update mxcli help graph-report and command descriptions

BSON Investigation

Not directly applicable (no new BSON structures), but critical for catalog validity:

1. Verify refs graph completeness:

   • Create test project with known cycles/modules
   • Run mxcli bson dump -p test.mpr --type microflow --object "Test.MF" to confirm:
     • All microflow calls appear in refs table
     • All page references (show page) appear
     • All entity usages (in microflows/pages) appear
   • Check if expression/XPath references are missing (deferred follow-on)

2. Catalog population verification:

   • Run REFRESH CATALOG FULL on test project
   • Query: SELECT * FROM CATALOG.REFS WHERE SOURCE_TYPE = 'microflow'
   • Confirm edge counts match manual inspection

3. Storage name validation:

   • Ensure catalog uses qualified names (not storage names) for module/element references
   • Check CATALOG.OBJECTS for correct qualified name format (Module.Element)

Testing Strategy

1. Doctest script (mdl-examples/doctype-tests/10-graph-analysis.mdl):

   REFRESH CATALOG SET COMMUNITIES (resolution: 0.8);
   DESCRIBE COMMUNITIES;
   DESCRIBE COMMUNITY OF MyModule.CoreEntity;
   -- Starlark usage in .star file:
   deny if community_of($module) == 42 and layer_of($module) < 3;

2. mxcli validations:

   • mxcli check graph-analysis.mdl → Syntax OK
   • mxcli check graph-analysis.mdl -p test.mpr --references → Valid refs
   • mxcli exec graph-analysis.mdl -p test.mpr → Runs without error

3. Integration tests:

   • Test against known spaghetti app (e.g., mx-test-projects/spaghetti-app.mpr):
     • Verify Leiden communities match leidenalg reference implementation
     • Confirm topological layers resolve all cycles
     • Check god nodes align with high-degree modules
   • Test multi-app split scenario:
     • Verify integration surface correctly classifies edges as OData/REST/event
     • Confirm coarse communities align with business domains

4. Unit tests:

   • Catalog: Test community detection on synthetic graphs
   • Starlark: Test builtin outputs with mocked catalog data
   • CLI: Test command parsing and output formatting

Gotchas & Risks

1. Storage name vs qualified name in catalog:

   • Risk: Catalog refs table uses storage names internally but exposes qualified names via CATALOG.REFS view
   • Mitigation: Always use QualifiedName type from model/ package when interacting with catalog; never parse storage names directly

2. BSON defaults for new catalog tables:

   • Risk: New catalog tables (catalog_communities) must have correct BSON defaults when serialized
   • Mitigation: Follow pattern in mdl/catalog/types.go - use zero values for optional fields, verify with mx bson dump on test MPR

3. Performance on large apps:

   • Risk: Leiden algorithm O(E log V) may be slow for apps with 10k+ refs
   • Mitigation:
     • Cache community results in catalog until next REFRESH CATALOG FULL
     • Add progress reporting during long-running refresh
     • Allow resolution parameter to trade accuracy for speed (lower = faster)

4. Starlark security:

   • Risk: Linter builtins could expose sensitive graph data
   • Mitigation:
     • Builtins only return aggregated metrics (community IDs, not raw graphs)
     • No builtins that expose full adjacency matrix
     • Audit all Starlark exposed data for PII (none expected in architecture metrics)

5. Version gating:

   • Risk: Older Mendix versions (<10.18) may lack v2 MPR format needed for full refs
   • Mitigation:
     • Gate feature behind requireMinVersion(10.18) in executor
     • Fallback to legacy refs extraction for v1 MPR (with warning about missing expression edges)

Critical path: Implement catalog backend methods first, then CLI commands, then Starlark builtins. Verify with make test and manual validation against proposal's reference implementation.

---

Automated guide via OpenRouter — workflow source