Run kson-lib tests against Rust and Python bindings#327
Open
aochagavia wants to merge 4 commits intomainfrom
Open
Run kson-lib tests against Rust and Python bindings#327aochagavia wants to merge 4 commits intomainfrom
kson-lib tests against Rust and Python bindings#327aochagavia wants to merge 4 commits intomainfrom
Conversation
We were accidentally converting the whole exception to a string, so the final error was something along the lines of "IllegalArgumentException: <the actual error>". This commit corrects that.
Since the `kson-lib` tests are written in Kotlin, we run them against Python in an indirect way: - We LLM-generated a JSON Schema based on the `kson-lib` API (with minimal manual tweaks). - We LLM-generated a clone of `kson-lib`'s `Kson.kt` that internally sends POST requests to a server based on the JSON Schema (with minimal manual tweaks). We called it `kson-lib-http` and wrote a descriptive readme for it. - We LLM-generated a Python server that receives requests and uses the bindings to handle them (with minimal manual tweaks). - We created a `:lib-python:kson-lib-tests` project responsible for actually running the `kson-lib` tests against the Python server. This setup is a bit convoluted, and I'm not sure I'd choose it for a different project. However, next to it being useful for testing, it doubles as an experiment to see how the KSON API could look like if we were to expose it through a text-based API (currently JSON, in the future... KSON!)
Here we follow the same approach used for testing the `lib-python` bindings. There is some code duplication in `ServerExtension.kt`, but I think we can tolerate that for now. Here is what we did: - We LLM-generated a Rust server that receives requests and uses the bindings to handle them (with minimal manual tweaks). - We created a `:lib-rust:kson-lib-tests` project responsible for actually running the `kson-lib` tests against the Rust server.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
See the individual commits (and their corresponding messages) for the implementation details. Note that the first one is merely a fix that I decided to pick up in this PR given it's so small.
Some remarks and questions:
kson-libtest suite / migrate tests over from the core library.Kson.ktbreaks the tests (they break when the public API oflib-ksondiverges fromlib-kson-http). Also, people will not necessarily have an easy time updating the JSON schema (provided they remember at all) and the test HTTP servers provided by the bindings. I thought about leaving instructions in one of the readmes (including prompts that could fully automate the updates), but even that doesn't cut it IMO (there will still be confusion). Ideas appreciated!