diff --git a/.github/AGENTS.md b/.github/AGENTS.md new file mode 100644 index 0000000..200749c --- /dev/null +++ b/.github/AGENTS.md @@ -0,0 +1,81 @@ +minecraft-data-generator extracts and generates Minecraft server-side data for multiple Minecraft versions via a Fabric mod that hooks into the vanilla Minecraft server. +This data that is generated is then stored inside the https://github.com/PrismarineJS/minecraft-data repo, a cross language repo holding minecraft data on blocks, items and more. + +There is a different mod for each Minecraft version this repo supports, under the `mc/` folder. +Each version has a Gradle project at `:mc:` and generating the data is done by running the Gradle task: +``` +./gradlew :mc::runServer +``` +Generated data ends up under `mc//run/server/minecraft-data` in the repo when run locally. + +## Repo Layout +- mc/ — per-Minecraft-version directories of OUR extractor code (e.g. `mc/1.21.8`) +- versions.json — canonical JSON array of versions that are supported (i.e. there are folders in mc/ for it) +- mc-source/ — (External) minecraft java edition code for relevant version with Mojang mappings that you can reference for changes. + * we store the last 2 most recent versions in the folder to save space, but if you need more versions you can do `git fetch origin client1.21.3 && git switch client1.21.3` + * there is a .diff file in mc-source/ like `1.21.7_to_1.21.8.diff` that contains result of `git diff --no-index old new` that you can reference for changes (note it's typically large) + * Since our generator is using Mojang mappings, the API naming is the same. +- README.md — info how to set up + +## Typical Flow + +minecraft-data-generator typically needs updates every time there is a new Minecraft version, particularly if there was code changes +around one of the areas that our extractors touches (e.g. an API changes, gets renamed, etc.). + +As explained in the README.md, there is an auto update workflow that typically creates scaffolding PRs (under the `bump` branch) whenever there is a new minecraft update. +These PRs are simply copying old version code into new, so there may be changes and other code fixes needed to support the new version. It's your +job to help plug the gaps by making these changes until the build passes for the given version. +It's also possible that there maybe missing or broken generated data even if the build is passing: in these cases, listen to the user's request +and figure out how to replicate, debug and fix the issues in data generation that may be a logical issue over a syntax one. + +## Troubleshooting + +Simply iterating over errors one by one is a good way to fix most issues. Inside mc-source/ you are provided the latest +code for the latest minecraft versions you can reference to investigate any code changes that we may need to accomodate in our +generator code. + +Sometimes, some APIs might change inside the mc code and make the data that the current code extracts no longer +valid. In these cases, do your best to conform the new data to the old structure even if it feels wrong. If you +can't, for example, let's say something about effects are removed from the game. But our data gen still expects it. +What to do? You should set the data to null, and inform the user about the issue -- perhaps a schema change is needed. +You can propose a new schema to the user (so that the relevant changes can be made to minecraft-data) and if +user confirms, you'd update the generator to output in that new schema. + +### Steps +- Reproduce the problem locally inside the container and run the generator task. + - Example run command: `./gradlew :mc:1.21.6:runServer --stacktrace` + - See if the problem exists on an older version -- if so inform the user, otherwise compare logs and code +- Inspect diffs in `mc-source/` to correlate Mojang API changes. + - See [mc-source/](mc-source/) +- Validate generated output at `mc//run/server/minecraft-data` to make sure all files are there + +--- + +## Examples + +Session 1 — Compile error after bump PR +- User prompt: "The bump PR for mc/1.21.8 fails to compile. Build error: cannot find symbol method RegistryKey.of(...)." +- Agent actions: + 1. Reproduce: `./gradlew :mc:1.21.8:runServer --stacktrace` + 2. Inspect compile error and mc-source/1.21.7_to_1.21.8.diff for API renames. + 3. Update generator code: edit mc/1.21.8/src/main/java/.../SomeRegistryAdapter.java to use the new Registry API. + 4. Build again and run generation. + +Session 2 — Runtime NPE during generation +- User prompt: "Generation starts but crashes with NullPointerException in EnchantmentsDataGenerator." +- Agent actions: + 1. Reproduce with stacktrace: `./gradlew :mc:1.20.4:runServer --stacktrace` + 2. Add temporary logging to EnchantmentsDataGenerator to identify the null object, or run in debugger. + 3. Fix by guarding against null Registries or empty lists and add defensive checks. + 4. Re-run generation and verify produced files. +- If problem persists: + * Add logging to old version too + * Check relevant code diff to see if anything changed related to this + +Session 3 — Semantically incorrect generated data +- User prompt: "Output JSON is valid but item tags are incorrect after the bump." +- Agent actions: + 1. Reproduce generation and open the produced JSON at mc//run/server/minecraft-data/tags/items.json + 2. Compare against previous version output (`git diff --no-index`) to identify mismatches. + 3. Inspect generator assumptions vs. mc-source diffs (names, tag keys, or logic changes). + 4. Correct mapping logic in generator class and run generation. diff --git a/.github/commands/setupCopilot.js b/.github/commands/setupCopilot.js index 7d248c9..d820365 100644 --- a/.github/commands/setupCopilot.js +++ b/.github/commands/setupCopilot.js @@ -61,10 +61,14 @@ async function main () { // Write outputs for downstream jobs, if GITHUB_OUTPUT available const ghOut = process.env.GITHUB_OUTPUT + const latestVersion = versions[versions.length - 1] + const secondLatestVersion = versions[versions.length - 2] if (ghOut) { try { fs.appendFileSync(ghOut, `success_versions=${JSON.stringify(successes)}\n`) fs.appendFileSync(ghOut, `failed_versions=${JSON.stringify(failures)}\n`) + fs.appendFileSync(ghOut, `latest_version=${JSON.stringify(latestVersion)}\n`) + fs.appendFileSync(ghOut, `second_latest_version=${JSON.stringify(secondLatestVersion)}\n`) } catch (e) { console.warn('Could not write to GITHUB_OUTPUT:', e && e.message) } @@ -72,6 +76,8 @@ async function main () { console.log('GITHUB_OUTPUT not found; printing summaries to stdout instead.') console.log('success_versions=', JSON.stringify(successes)) console.log('failed_versions=', JSON.stringify(failures)) + console.log('latest_version=', JSON.stringify(latestVersion)) + console.log('second_latest_version=', JSON.stringify(secondLatestVersion)) } console.log('') diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml index fb62535..7c1a3d9 100644 --- a/.github/workflows/build.yml +++ b/.github/workflows/build.yml @@ -2,10 +2,10 @@ name: Java CI with Gradle on: push: branches: - - ** + - '**' pull_request: branches: - - ** + - '**' permissions: pull-requests: write diff --git a/.github/workflows/copilot-setup-steps.yml b/.github/workflows/copilot-setup-steps.yml index da96427..7e3193f 100644 --- a/.github/workflows/copilot-setup-steps.yml +++ b/.github/workflows/copilot-setup-steps.yml @@ -14,6 +14,14 @@ jobs: runs-on: ubuntu-latest timeout-minutes: 20 + # Set the permissions to the lowest permissions possible needed for your steps. + # Copilot will be given its own token for its operations. + permissions: + # If you want to clone the repository as part of your setup steps, for example to install dependencies, you'll need the `contents: read` permission. If you don't clone the repository in your setup steps, Copilot will do this for you automatically after the steps complete. + contents: write + actions: write + pull-requests: write + steps: - name: Checkout repository uses: actions/checkout@v4 @@ -33,5 +41,31 @@ jobs: add-job-summary-as-pr-comment: on-failure - name: Run Copilot setup script + id: sb run: node .github/commands/setupCopilot.js - shell: bash \ No newline at end of file + shell: bash + + - name: Checkout latest mc code (external repo) + # If the external repo is public, the default token is fine. + # If it's private, create a personal access token with repo access and add it + # to this repo's secrets as EXTRA_REPO_TOKEN. + uses: actions/checkout@v4 + with: + repository: extremeheat/extracted_minecraft_data + path: mc-source/${{ steps.sb.outputs.latest_version }} + ref: client${{ steps.sb.outputs.latest_version }} + fetch-depth: 1 + + - name: Checkout second latest mc code (external repo) + # If the external repo is public, the default token is fine. + # If it's private, create a personal access token with repo access and add it + # to this repo's secrets as EXTRA_REPO_TOKEN. + uses: actions/checkout@v4 + with: + repository: extremeheat/extracted_minecraft_data + path: mc-source/${{ steps.sb.outputs.second_latest_version }} + ref: client${{ steps.sb.outputs.second_latest_version }} + fetch-depth: 1 + + - name: Write diff + run: cd mc-source && git diff --no-index ${{ steps.sb.outputs.second_latest_version }} ${{ steps.sb.outputs.latest_version }} > ${{ steps.sb.outputs.second_latest_version }}_to_${{ steps.sb.outputs.latest_version }}.diff diff --git a/.gitignore b/.gitignore index bd587ce..1f98e56 100644 --- a/.gitignore +++ b/.gitignore @@ -118,6 +118,7 @@ gradle-app.setting !gradle-wrapper.jar bin/ +mc-source/ node_modules package-lock.json \ No newline at end of file