Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 7 additions & 7 deletions unified/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,10 @@ by Apple's swift-syntax rather than by tree-sitter.
- To build the extractor, run `scripts/create-extractor-pack.sh`

## Swift Parser
- Swift source is parsed by `swift-syntax-parse`, a small Swift/Rust binary in
`swift-syntax-rs` that wraps Apple's swift-syntax and emits the parse tree as
JSON. There is no grammar in this repository to edit.
- Swift source is parsed by the `swift-syntax-rs` crate, which wraps Apple's
swift-syntax. The extractor calls `swift_syntax_rs::parse_to_json` in-process
to obtain the parse tree as JSON; the extractor does not invoke a separate
parser binary, and there is no grammar in this repository to edit.

- `extractor/src/languages/swift/adapter.rs` converts that JSON into a yeast AST.

Expand All @@ -23,10 +24,9 @@ by Apple's swift-syntax rather than by tree-sitter.
- The mapping from the parse tree to the target AST is found in `extractor/src/languages/swift/swift.rs`

- To run tests for the parser and mapping, run `cargo test` in the `extractor`
directory. The tests need the `swift-syntax-parse` binary: point
`CODEQL_EXTRACTOR_UNIFIED_SWIFT_SYNTAX_PARSE` at it, or put it on `PATH`.
Corpus tests skip themselves when it cannot be found, so check for skips
before concluding a change is clean.
directory. Since the parser is linked in-process, this needs a working Swift
toolchain (so `swift-syntax-rs` can build). The tests can also be run under
Bazel via `bazel test //unified/extractor:all_tests`.

- Extractor test cases are located at `extractor/tests/corpus/swift/*/*.swift`.

Expand Down
23 changes: 11 additions & 12 deletions unified/swift-syntax-rs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -132,27 +132,22 @@ The build does not depend on any particular version manager. You need:
(currently `6.3.2`), used to build `swift-syntax` `603.0.2`. Install it any way
you like — [swift.org](https://www.swift.org/install/) or
[swiftly](https://www.swift.org/swiftly/) (which reads `.swift-version`), or a
system package. Just make sure `swift` is on your `PATH` (or point `build.rs`
at it with the `SWIFT` environment variable).
system package. Just make sure `swift` (and `swiftc`) are on your `PATH` —
`build.rs` invokes them directly and does not read any environment variable
to locate them.

On Debian/Ubuntu the Swift runtime also needs `libncurses6` (and related libs)
available on the system.

## Building & testing

With `cargo` and `swift` on `PATH`:
With `cargo` and `swift`/`swiftc` on `PATH`:

```sh
cargo build
cargo test
```

If your `swift`/`swiftc` are not on `PATH`, point the build at them explicitly:

```sh
SWIFT=/path/to/swift SWIFTC=/path/to/swiftc cargo build
```

The first build compiles `swift-syntax` and can take several minutes.

## Building with Bazel (CI)
Expand All @@ -170,6 +165,10 @@ bazel test //unified/swift-syntax-rs:swift_syntax_rs_test
bazel run //unified/swift-syntax-rs:swift-syntax-parse < some.swift
```

The `swift-syntax-parse` binary is a debugging aid for looking at the raw
swift-syntax JSON for some input; it is not shipped as part of the extractor
pack, which links `swift-syntax-rs` directly instead.

Requirements:

- **`clang`** must be installed on the runner. `rules_swift` requires the Bazel
Expand Down Expand Up @@ -221,9 +220,9 @@ echo 'let x = 1' | cargo run --bin swift-syntax-parse
The JSON tree is consumed by the CodeQL extractor, which converts it into a
[`yeast::Ast`](../../shared/yeast) — the in-memory format its rewrite rules
operate on. That adapter is a pure-Rust module living in the extractor
(`unified/extractor/src/languages/swift/adapter.rs`), so the extractor never
needs the Swift toolchain: it consumes the JSON produced out-of-process by this
crate's `parse_to_json` / the `swift-syntax-parse` binary.
(`unified/extractor/src/languages/swift/adapter.rs`). The extractor links
`swift-syntax-rs` directly and consumes the JSON produced in-process by this
crate's `parse_to_json`.

## Layout

Expand Down
Loading