anygatecloud/strom
2
1
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Fix minecraft-data as proper submodule

Browse Source
This commit is contained in:
adman
2026-01-13 15:37:52 +01:00
parent f97538fe19
commit ac2713433e
1083 changed files with 1 additions and 13761996 deletions
Download Patch File Download Diff File Expand all files Collapse all files
data/minecraft-data/doc/ARCHITECTURE.md
-20
View File
@@ -1,20 +0,0 @@
## minecraft-data architecture
Key Details:
* Langauge independent repository holding data for many Minecraft versions, including Java Edition versions after 1.7 and Bedrock Edition versions after 1.16
* Has data on blocks, items, network protocol packet structures, entities and more
* The data in this repo is generated using several tools as described in the `doc/add-data-new-version.md` file for Java Edition and `doc/bedrock.md` for Bedrock Edition
File Structure:
* Most data is stored in JSON. The JSON schemas for these files are stored inside the `schemas/` folder.
* `data/pc` contains data for minecraft java edition separated by folder and `data/bedrock` for bedrock edition
* `data/dataPaths.json` has a mapping for minecraft version to data type to folder to find that data.
* It looks like `{"pc": { "1.16": {"blocks":"pc/1.16"} } }`, meaning java edition block data for version 1.16 can be found in `pc/1.16` under the name `blocks.json`. If the data didn't change from 1.15, it might point to `pc/1.15` instead.
* `tools/js` contains Node.js scripts to do maintenance on the repository, such as running tests to ensure data validity and also contains some files used to generate data inside the repository.
Usage:
* As documented in the README.md, there exist several wrappers that expose the data inside minecraft-data. Notably, this includes PrismarineJS/node-minecraft-data, which exposes the data as a NPM package.
Contributing Tips:
* Files should generally be obtained with the use of a data extractor or generator, as mentioned above
* If you make a PR with new data, describe how you obtained the data, as otherwise we won't be able to obtain that data for future versions
* Checkout the `.github/workflows` directory to see how tests are run on the repo
data/minecraft-data/doc/add-data-new-version.md
-52
View File
@@ -1,52 +0,0 @@
# How to add data for a new version ?
For bedrock edition see [bedrock.md](bedrock.md)
| file name | auto? | how to get it | notes |
| -- | -- | -- | -- |
| protocol.json | No | Follow [Pre-release Protocol][1] | Always double check by looking at the Minecraft source files. |
| blocks.json | Yes | Use [minecraft-data-generator-server][2] |
| items.json | Yes | Use [minecraft-data-generator-server][2] |
| entities.json | Yes | Use [minecraft-data-generator-server][2] and run `extractPcEntityMetadata.js` script in tools/js to generate entity metadata in entities.json and protocol.json |
| recipes.json | Yes | Use [Burger][12], then use [burger-extractor][13] | should eventually be changed to native data generators |
| blockCollisionShapes.json | Yes | Use [minecraft-data-generator-server][2] |
| commands.json | No? |Use [mc-data-command-generator][3] | Link to jar files have to be manually added |
| biomes.json | Yes | Use [minecraft-data-generator-server][2] |
| instruments | Yes | Use [minecraft-data-generator-server][2] |
| materials.json | Yes | Use [minecraft-data-generator-server][2] |
| windows.json | No | Look at [wiki.vg inventories][4] |
| version.json | No | Look at [Protocol Versions][5] | [wiki.vg protocol numbers][6] |
| effects.json | Yes | Use [minecraft-data-generator-server][2] |
| enchantments.json | Yes | Use [minecraft-data-generator-server][2] |
| language.json | Yes | Use [minecraft-data-generator-server][2] |
| particles.json | Yes | Use [minecraft-data-generator-server][2] |
| blockLoot.json | No | Use [minecraft-jar-extractor][11] | Extractor can be very be tempermental at times
| entityLoot.json | No | Use [minecraft-jar-extractor][11] | Extractor can be very be tempermental at times
| mapIcons.json | No | Icons data can be found in the Minecraft source directly. Alternatively you might be able to look up the icons from the following page on the [Minecraft wiki][7] or from [wiki.vg][8]. | [minecraft-data pr mapIcons][9] |
| loginPacket.json | Yes | Running tests on [node-minecraft-protocol][10] |
| sounds.json | Yes | Use [minecraft-data-generator-server][2] | Make sure to check the packets that use this and the friendlybytebuffer functions to check if an offset is needed in the generator code.
| foods.json| Yes | Use [minecraft-data-generator-server][2] | move file obtained with [minecraft-data-generator-server][2] to the correct location and run `extractPcFoods.js` script located in `tools/js`|
* All files that say Yes for auto can be gotten from [minecraft-data-auto-updater][9] (that uses [minecraft-data-generator-server][2]). This is the preferred way to extract the data.
* **Note**, there is a script inside the `tools/js` folder that can be run with `npm run version` with usage of `npm run version <bedrock|pc> {version} {protocol_version}` to automatically copy over old data including the protocol in support for a new version and update entries in the dataPaths.json.
After generating and creating the new files for the version in its own directory, the version must be added to dataPaths.json in the data folder.
Additionally in data/pc/common the versions.json file needs to be updated with the new version.
## Protocol data
* See [protocol.md] for information on the protocol and how to update the data
[1]: https://wiki.vg/Pre-release_protocol
[2]: https://github.com/PrismarineJS/minecraft-data-generator-server
[3]: https://github.com/Miro-Andrin/mc-data-command-generator
[4]: https://wiki.vg/Inventory
[5]: https://github.com/PrismarineJS/minecraft-data/blob/master/data/pc/common/protocolVersions.json
[6]: https://wiki.vg/Protocol_version_numbers
[7]: https://minecraft.wiki/Map#Map_icons
[8]: https://wiki.vg/Protocol#Map_Data
[9]: https://github.com/PrismarineJS/minecraft-data-auto-updater
[10]: https://github.com/PrismarineJS/node-minecraft-protocol
[11]: https://github.com/PrismarineJS/minecraft-jar-extractor
[12]: https://github.com/Pokechu22/Burger
[13]: https://github.com/PrismarineJS/burger-extractor
data/minecraft-data/doc/bedrock.md
-54
View File
@@ -1,54 +0,0 @@
## minecraft bedrock edition data
### Obtaining data
| file name | info | how to get | schema |
|-|-|-|-|
| blockStates.json | Contains the global block palette for a Minecraft version. <br>Contains all of the possible block states. <br><br><br>The index of a Block State instance here is the paletted ID, <br>which can be used to idenfify this block state<br>instead of a string. | [bedrock-extractor][1] ("BlockStates.json" file) | |
| steve.json | Skin data for clients connecting to BE servers, for default Steve character | [bedrock-protocol][2] (capture from proxy) | |
| blockMappings.json | Output of bedrock-extractor |
| language.json | Translation keys and their English string values. | [extract-minecraft-lang][3] (run for new version, then copy the JSON file to minecraft-data) | |
### Updating protocol data
* To update bedrock protocol data, open `data/bedrock/latest` (or replace "latest" with some older version) and update the `proto.yml` and `types.yml` files.
* Go to `tools/js`, run `npm install` then `npm run build` to update the protodef JSON files
To update to a new protocol version:
* Copy `data/bedrock/latest` files into the old version's folder.
* Open `data/bedrock/latest` and update the `proto.yml` and `types.yml` files and update the `!version` part at the top
* Go to `tools/js`, run `npm install` then `npm run build` to update the protodef JSON files
then run `npm test` to test validity
### Status of data
<!--StartFragment-->
file name | have? | how to get it | notes
-- | -- | -- | --
protocol.json | ✔ | manual updated | wip
blocks.json | ✔ | bedrock-extractor | &nbsp;
items.json | ✔ | bedrock-extractor | &nbsp;
entities.json | ✔ | bedrock-extractor | &nbsp;
recipes.json | ✔ | bedrock-protocol dumper | new schema: adds furnace and other special recipes. <br/> Some recipes can have multiple outputs, also allow for recipe ingredients with damage values and specific counts.
blockCollisionShapes.json | ✔ | (from pc data) | New schema: block IDs map to an array of block state indexes, to get the block stateID, minStateId + index of array. This way different copies of the block with different state IDs can have unique collisions.
commands.json | ❌ | bedrock-protocol + dumper | must be custom schema, current JE schema too low level
biomes.json | ✔ | bedrock-extractor (via Amulet) |
instruments | ✔ | manual | &nbsp;
materials.json | 🔵 | (from pc data) | &nbsp;
windows.json | ✔ | manual obtained via proxy | &nbsp;
version.json | ✔ | from bedrock-protocol |
effects.json | ❌ | (from pc data) | &nbsp;
enchantments.json | ✔ | bedrock-extractor (via Geyser -> pc data) | &nbsp;
language.json | ✔ | [extract-minecraft-lang][3] (extract from dedicated server) | &nbsp;
particles.json | 🔵 | bedrock-protocol docs | IDs not needed, handled in protocol
blockLoot.json | 🔵 | manual data for 1.18 | &nbsp;
entityLoot.json | 🔵 | manual data for 1.18 | &nbsp;
mapIcons.json | 🔵 | ? | ?
<!--EndFragment-->
[1]: https://github.com/extremeheat/minecraft-data-extractor/tree/master/bedrock
[2]: https://github.com/PrismarineJS/bedrock-protocol
[3]: https://github.com/CreeperG16/extract-minecraft-lang
data/minecraft-data/doc/blockCollisionShapes.md
-73
View File
@@ -1,73 +0,0 @@
# Block Collision Shapes
Documentation for the block collision shapes data in `blockCollisionShapes.json`.
## Collision Boxes
Collision boxes are the axis-aligned bounding boxes that are checked when colliding entities and blocks.
They can differ from the hitbox, which is the box that is checked when clicking a block.
The full collision shape of a block state can be composed of multiple such boxes, for example stairs or brewing stands.
The way collision boxes are stored/computed has changed with The Flattening.
Until 1.12, a block's collision box also depended on the blocks around it,
e.g., cobblestone walls change their collision box if there's a solid block to N/S/E/W/above.
Starting with 1.13 the colision boxes seem to only depend on the block state itself.
`blockCollisionShapes.json` does not encode any of the neighbor block dependent collision box behavior.
It assumes all surrounding blocks are air, so it is only fully accurate for 1.13 and later.
For 1.12 and earlier, it is accurate for any blocks that don't depend on their neighbors.
For the neighbor dependent blocks it still serves as a better approximation than the previously available "full block"/"empty block" data in `blocks.json`.
## Data Format
The data are split into two collections, as a compromise between file size and consumer implementation simplicity.
`shapes` contains the actual shape coordinates, and `blocks` references these shapes to allow deduplication of data shared by several blocks.
### `blocks`
The `blocks` collection assigns each block (by string id) either an array of shape ids (one per block state)
or, if all block states have the same shape, just specifies that shape id instead of the array.
```js
"blocks":{
"rail":0, // Although not standardized, 0 usually is the "empty" shape (no collision).
"furnace":1, // all furnace states have the shape of id 1 (here: full block)
"oak_slab":[169,169,61,61,1,1], // six block states (upper/lower/double slab and subtypes), referencing the shape ids 169, 61, and 1.
```
### `shapes`
A shape consists of zero or more axis-aligned bounding boxes.
The `shapes` collection assigns each shape id the list of its boxes.
Each box is an array of the block-local coordinates in the order: west, bottom, north, east, top, south;
or x_min, y_min, z_min, x_max, y_max, z_max.
Here block-local means that if the block was at 0,0,0 the box coordinates would match the world coordinates.
Some blocks reach out of the 1x1x1 box, notably fences and piston heads.
The latter can thus have negative coordinate entries.
```js
"shapes":{
"0":[], // Although not standardized, 0 usually is the "empty" shape (no collision).
"1":[[0.0,0.0,0.0,1.0,1.0,1.0]], // full block: single box from (0, 0, 0) to (1, 1, 1)
"61":[[0.0,0.0,0.0,1.0,0.5,1.0]], // lower half slab: from (0, 0, 0) to (1, 0.5, 1)
"110":[[0.0,0.0,0.0,1.0,0.8125,1.0],[0.25,0.8125,0.25,0.75,1.0,0.75]],
// shape 110 (referenced in end_portal_frame) consists of two boxes:
// - the lower part of the block, up to y=0.8125
// - the eye: from y=0.8125 to y=1, from x&z=0.25 to 0.75
```
## Usage Examples
- [Java](https://github.com/Gjum/McDataExtracting/blob/04b573572c/src/main/java/mcextract/BlockCollisionBoxStorage.java)
- [Kotlin](https://github.com/Gjum/Botlin/blob/83a024e76dc76998e04fece55a2454678420c7e7/src/main/kotlin/com/github/gjum/minecraft/botlin/data/BlockInfo.kt#L96-L135)
## Data Source
Because the way that block collision boxes are stored/computed has changed between versions, there are multiple versions of the extraction code:
[1.12.2](https://github.com/Gjum/McDataExtracting/blob/85b564a2c00aa2e88b5b9bd6246f13df39cf215a/src/main/java/mcextract/Main.java#L87-L162),
[1.14.4](https://github.com/Gjum/McDataExtracting/blob/ff8fa51759/src/main/java/mcextract/Main.java#L57-L117),
[1.15](https://github.com/Gjum/McDataExtracting/blob/04b573572c4ccdaedab15383567c98dd63178f97/src/main/java/mcextract/Main.java#L57-L117).
In 1.12.2, the extraction failed for the `end_portal_frame` block, so it was added manually to the JSON file.
data/minecraft-data/doc/history.md
-1157
View File
File diff suppressed because it is too large Load Diff
data/minecraft-data/doc/make-a-new-wrapper.md
-20
View File
@@ -1,20 +0,0 @@
# Making a new wrapper
minecraft-data has currently a few wrapper (see the list in the README) for several languages.
If you want to use minecraft-data in a new language, you might want to create a wrapper for that language.
Here is a simple way to do it :
* create a git repo
* add minecraft-data as a submodule :
`git submodule add https://github.com/PrismarineJS/minecraft-data.git`
* add code in your language to read the json files
* add some code to index the data by name, by id to provide an api like
[api.md](https://github.com/PrismarineJS/node-minecraft-data/blob/master/doc/api.md)
* add some readme and doc to explain that api
* add a basic example
* add files to package your module (for example in js it's a package.json, in python a setup.py)
* publish it
If it makes sense to generate files of your language from the data instead of dynamically reading the .json,
feel free to do it (example of that in [ProtocolGen](https://github.com/Johni0702/ProtocolGen) for java)
data/minecraft-data/doc/protocol.md
-156
View File
@@ -1,156 +0,0 @@
## Updating Minecraft protocols
The protocol data is manually updated. It is updated by looking at changes made to the Minecraft protocol through various sources.
### Sources
#### Minecraft Java Edition
* https://wiki.vg/Protocol -- a manually updated version of the Minecraft protocol
* [Minecraft source code changes](https://github.com/extremeheat/extracted_minecraft_data/tree/clientlatest) - comparing decompiled Java output for the current version against the previous version
#### Minecraft Bedrock Edition
* [PocketMine](https://github.com/pmmp/PocketMine-MP) - PHP
* [gophertunnel](https://github.com/Sandertv/gophertunnel) - Go
* [CloudburstMC](https://github.com/CloudburstMC/Protocol) - Java
### Protocol schema
This project uses ProtoDef's schema for representing Minecraft protocol packets in JSON. These protocol.json files can then be used to generate code to serialize and deserialize Minecraft packets in any language. See the documentation for [node-protodef here](https://github.com/ProtoDef-io/node-protodef) for a Node.js implementation. In PrismarineJS/minecraft-protocol and PrismarineJS/bedrock-protocol, JavaScript code is generated from the JSON through the node-protodef compiler.
#### YAML syntax
For easier maintainability, the protocol JSON is generated from a more human readable YAML format. You can read more [here](https://github.com/extremeheat/protodef-yaml). Some documentation is below.
These YAML files can be found in data/pc/latest and data/bedrock/latest. Old protocol YAML data can be found in `data/<pc|bedrock>/<version>`.
In bedrock data, types and packets are sperated into proto.yml and types.yml files. On pc there is one single proto.yml file with all packets and types for all the states, and these states are sperated under different top-level regions like `^handshaking.toClient.types` and `^play.toServer.types` (which both inherit from the top `^types`).
Note: Minecraft Java Edition uses big endian encoding by default, whereas Minecraft Bedrock Edition uses little endian. Note that varints are always little endian.
```yml
# This defines a new data structure, a ProtoDef container.
Position:
# Variable `x` in this struct has a type of `li32`, a little-endian 32-bit integer
x: li32
# `z` is a 32-bit LE *unsigned* integer
z: lu32
# `b` is an **optional** 32-bit LE floating point
y?: lf32
# Fields starting with `packet_` are structs representing Minecraft packets
packet_player_position:
# Fields starting with ! are ignored by the parser. '!id' is used by the parser when generating the packet map
!id: 0x29 # This packet is ID #0x29
!bound: client # `client` or `server` bound, just for documentation purposes. This has no other effect.
# Read `on_ground` as a boolean
on_ground: bool
# Read `position` as custom data type `Position` defined above.
position: Position
# Reads a 8-bit unsigned integer, then maps it to a string
movement_reason: u8 =>
0: player_jump
1: player_autojump
2: player_sneak
3: player_sprint
4: player_fall
# A `_` as a field name declares an anonymous data structure which will be inlined. Adding a '?' at the end will start a `switch` statement
_: movement_reason ?
# if the condition matches to the string "player_jump" or "player_autojump", there is a data struct that needs to be read
if player_jump or player_autojump:
# read `original_position` as a `Position`
original_position: Position
jump_tick: li64
# if the condition matches "player_fall", read the containing field
if player_fall:
original_position: Position
default: void
# Another way to declare a switch, without an anonymous structure. `player_hunger` will be read as a 8-bit int if movement_reason == "player_sprint"
player_hunger: movement_reason ?
if player_sprint: u8
# The default statement as in a switch statement
default: void
# Square brackets notate an array. At the left is the type of the array values, at the right is the type of
# the length prefix. If no type at the left is specified, the type is defined below.
# Reads an array of `Position`, length-prefixed with a ProtoBuf-type unsigned variable length integer (VarInt)
last_positions: Position[]varint
# Reads an array, length-prefixed with a zigzag-encoded signed VarInt
# The data structure for the array is defined underneath
keys_down: []zigzag32
up: bool
down: bool
shift: bool
```
The above roughly translates to the following JavaScript code to read a packet:
```js
function read_position(stream) {
const ret = {}
ret.x = stream.readLI32()
ret.z = stream.readLU32()
if (stream.readBool()) ret.y = stream.readLF32()
return ret
}
function read_player_position(stream) {
const ret = {}
ret.on_ground = Boolean(stream.readU8())
ret.position = read_position(stream)
let __movement_reason = stream.readU8()
let movement_reason = { 0: 'player_jump', 1: 'player_autojump', 2: 'player_sneak', 3: 'player_sprint', 4: 'player_fall' }[__movement_reason]
switch (movement_reason) {
case 'player_jump':
case 'player_autojump':
ret.original_position = read_position(stream)
ret.jump_tick = stream.readLI64()
break
case 'player_fall':
ret.original_position = read_position(stream)
break
default: break
}
ret.player_hunger = undefined
if (movement_reason == 'player_sprint') ret.player_hunger = stream.readU8()
ret.last_positions = []
let __latest_positions_len = stream.readUnsignedVarInt()
for (let i = 0; i < __latest_positions_len; i++) {
ret.last_positions.push(read_player_position(stream))
}
ret.keys_down = []
let __keys_down_len = stream.readZigZagVarInt()
for (let i = 0; i < __keys_down_len; i++) {
const ret1 = {}
ret1.up = Boolean(stream.readU8())
ret1.down = Boolean(stream.readU8())
ret1.shift = Boolean(stream.readU8())
ret.keys_down.push(ret1)
}
return ret
}
```
and the results in the following JSON for the packet:
```json
{
"on_ground": false,
"position": { "x": 0, "y": 2, "z": 0 },
"movement_reason": "player_jump",
"original_position": { "x": 0, "y": 0, "z": 0 },
"jump_tick": 494894984,
"last_positions": [{ "x": 0, "y": 1, "z": 0 }],
"keys_down": []
}
```
Custom ProtoDef types can be inlined as JSON:
```yml
string: ["pstring",{"countType":"varint"}]
```
#### Compiling to JSON
Once the protocol YAML files have been updated, run `npm run build` inside `tools/js` to generate protocol.json for each of the protocol YAML files.
data/minecraft-data/doc/recipes.md
-151
View File
@@ -1,151 +0,0 @@
# Recipes
Documentation for the recipes json schema
### Items
An item can be represented different ways.
A single **numerical ID** or `null`.
**Examples:**
- `1` for any item with ID `1`
- `null` for empty
A **list** of `id` and `metadata`.
This is preferred if there are many items at once, e.g. in a shape.
**Example:** `[1, 2]` for any count of Polished Granite
A **dictionary** of at least `id`, optionally `metadata` and `count`.
This is preferred if there are not many items at once, e.g. `result` in a recipe.
**Examples:**
- `{"id": 1}` for any item with ID `1`
- `{"id": 1, "metadata": 3}` for any count of Diorite
- `{"id": 1, "metadata": 2, "count": 4}` for 4 Polished Granite
### Shapes
A shape is a list of rows, which are lists of items.
There must be at least one row with at least one item in it.
All rows must have the same length.
Empty rows at the beginning or end of a shape may be omitted.
Empty colums at the end may also be omitted.
When an item can be crafted in a 2x2 grid, the shape may not be larger than 2x2.
**Examples:**
Stick:
[
[5],
[5]
]
or
[
[5, null],
[5, null]
]
Stonebricks:
[
[1, 1],
[1, 1]
]
Polished Granite:
[
[[1,1], [1,1]],
[[1,1], [1,1]]
]
or
[
[{"id": 1, "metadata": 1}, {"id": 1, "metadata": 1}],
[{"id": 1, "metadata": 1}, {"id": 1, "metadata": 1}]
]
### Recipes
A shaped recipe is a dictionary of `result`, `inShape` and optionally `outShape`.
A shapeless recipe is a dictionary of `result` and `ingredients`.
`inShape` and `outShape` are shapes, as specified [above](#Shapes).
`ingredients` is a list of items.
`result` is a single item.
Items may be in any of the representations [above](#Items).
`outShape` must be of the same dimensions as `inShape`.
### File format
The file is in JSON format.
At the top level is a dictionary of quoted numerical item IDs.
Each ID maps to a list of recipes.
There may be multiple different recipes per item (same ID and metadata).
The recipes may not be sorted.
**Example:**
```js
{
"1": [
{ // Polished Granite
"result": {"id": 1, "metadata": 2, "count": 4},
"inShape": [
[[1,1], [1,1]],
[[1,1], [1,1]]
]
},
{ // Polished Diorite
"result": {"id": 1, "metadata": 4, "count": 4},
"inShape": [
[[1,3], [1,3]],
[[1,3], [1,3]]
]
}
],
"5": [
{
"result": {"id": 5, "metadata": -1, "count": 4},
"inShape": [
[17]
]
}
],
"280": [
{
"result": {"id": 280, "metadata": -1, "count": 4},
"inShape": [
[5],
[5]
]
}
],
"354": [ // cake
{
"result": 354,
"inShape": [
[335, 335, 335],
[353, 344, 353],
[296, 296, 296]
],
"outShape": [
[ 325, 325, 325],
[null, null, null],
[null, null, null]
]
}
],
"378": [ // magma cream
{
"result": 378,
"ingredients": [341, 377]
}
]
}
```
## Example implementations
- [Python](https://gist.github.com/Gjum/9c0491aad1c8ec8d6f38#file-recipe-py)
## Json schema
The [json schema](https://github.com/PrismarineJS/minecraft-data/blob/master/enums_schemas/recipes_schema.json) automatically
checks whether the recipes.json respects that format, see the [visualization](http://prismarinejs.github.io/minecraft-data/) here.