Modded Minecraft Reddit Feed
223 subscribers
34.2K photos
3.3K videos
5 files
37.7K links
This is a bot channel automatically posting content from /r/feedthebeast as it gets posted.

Supported by @r_channels & @reddit2telegram.
Download Telegram
always nice seeing a gregtech player in the wild
https://redd.it/1v379f6
@MinecraftModded
To the person who made Galosphere, I hope your bed and both sides of your pillows stay warm tonight.
https://redd.it/1v3frkn
@MinecraftModded
Media is too big
VIEW IN TELEGRAM
Little Logistics is now available on 1.21.1, with docking and routing overhaul!

https://redd.it/1v32zyb
@MinecraftModded
This media is not supported in your browser
VIEW IN TELEGRAM
I made a mod that adds roots beneath every vanilla tree in Minecraft, along with new Rooted Grass and Rooted Podzol blocks. Roots also generate beneath tall plants, bamboo, jungle bushes, and stumps. The mod is fully compatible with Muddy World. Mod name: Rooted Trees

https://redd.it/1v3nags
@MinecraftModded
I Spent Half a Year Chasing a Microstutter.. That was Caused by Windows

I just wanted to let anyone who has been experiencing a microstutter what worked for me. I was playing ATM10 and Better Minecraft modpacks. I had just purchased a new computer and noticed drastic FPS dips (into the 40s with shaders) and a pretty consistent microstutter. It wasn't game-breaking, but let me tell you, it drove me insane.

I tried EVERYTHING. Capping my FPS to various amounts, toggling V-sync, messing with NVIDIA settings, changing the Java argements, adjusting the RAM from 6-16 GB and everything in between, messing with the graphics settings in the game, toggling shaders off and on, you name it, I did it. I spent hours crawling Reddit and every other forum and website God ever made.

I had toyed with the idea of installing Linux and finally pulled the trigger by dual-booting Windows and Cachyos. As soon as I downloaded ATLauncher and got into the game, the difference was like night and day. Way better FPS out of the box even with shaders and no microstutter!

It was gone, guys. Windows made my computer run like shit for reasons unknown and also fixed a bunch of other minor issues I was having like my computer acting weird when waking up from hibernation and my trackpad occasionally taking a dump.

Oh, before I forget, I'm running a Core Ultra 9 275hx with a 5070 and 32GB of RAM. I bought my PC before the ram-pocalypse to play modded Minecraft. Since Minecraft is so CPU dependent, I knew it was capable of running Minecraft smoothly, I just didn't realize Windows was taking such a toll on the poor thing.

Anyways, that's my experience for anyone who's having issues with this. Cheers!

https://redd.it/1v3n1el
@MinecraftModded
Do y’all remember architecturecraft?? I am about to make a port. Is it a good idea?
https://redd.it/1v49f8z
@MinecraftModded
Media is too big
VIEW IN TELEGRAM
Ask me anything - Minecraft as a colony-builder (Sovereign early access preview)

https://redd.it/1v4k75j
@MinecraftModded
This is not a daily Quark post, It's a Zeta post
https://redd.it/1v4n2ac
@MinecraftModded
RAM's like a million dollars now so I saved you some in JEI

hey it's me back again with a bigass wall of text about a thing. enjoy!

tl;dr: JEI search has to index a surprisingly large amount of text in big modpacks. I made two new small Java libraries for immutable substring search, then built a JMH benchmark to test a new JEI search-index approach against the current one. The early result: on a synthetic 100k-item JEI-like workload, the most promising new index retained about 259 MiB instead of about 426 MiB, roughly 40% less memory, while also making short searches much faster in the benchmark.

# The optimization problem

JEI search feels simple from the player's side: type a few letters, the item list filters down to show what you're looking for.

Internally, that search box has to deal with a lot of text though. Item names are the obvious part, but JEI also searches things like tooltip lines, mod names, and tags. In a large pack, that can become hundreds of thousands of searchable strings. But I can't just remove those to save RAM! Searching copper, a mod name, a tag-like string, or some tooltip text is one of the things that makes JEI feel good to use.

Every nice search feature has a cost. The index has to be built, it has to be kept in memory, and then it has to be fast enough to query repeatedly as the player types.

Since ram is becoming infinitely expensive, I started poking at a very specific question: Can JEI keep the same search features, but use less memory?

# The two new projects

I made two small Java libraries for this:

[baked-substring-index](https://github.com/mezz/baked-substring-index)
baked-suffix-array-index

Both are designed around the same basic idea:

1. Add key/value pairs to a builder.
2. Build an immutable index.
3. Share that built index for read-only lookup.
4. If the source data changes, build a new index.

That maps pretty well to JEI's use case. The searchable data is built after loading/reloading, then queried many times while the player types. The existing library used in JEI is my adaptation of a generalized-suffix-tree, which is mutable. I wanted to look at immutable alternatives since they might be more optimized for the bulk of items, and I can keep the old mutable tree around for when mods add more items at runtime.

The libraries are intentionally not trying to solve every search problem that's out there in the wider computer science world. They do exact Java String substring matching. It's not pre-AI Google Search, there's no fuzzy search, no ranking, no locale-aware matching, no stemming, no edit distance. If callers want case folding or normalization, they can normalize before indexing and searching.

>References:
>
>Esko Ukkonen, "On-line construction of suffix trees," Algorithmica 14, 249-260, 1995. DOI: https://doi.org/10.1007/BF01206331
>
>Udi Manber and Gene Myers. "Suffix Arrays: A New Method for On-Line String Searches." SIAM Journal on Computing, 22(5), 935-948, 1993. DOI: https://doi.org/10.1137/0222058.
>
>Fei Shi. "Suffix Arrays for Multiple Strings: A Method for On-Line Multiple String Searches." ASIAN 1996, Lecture Notes in Computer Science 1179, 11-22, 1996. Bibliographic record: https://dblp.org/rec/conf/asian/Shi96.
>
>Gonzalo Navarro and Ricardo Baeza-Yates, "A Practical q-Gram Index for Text Retrieval Allowing Errors", CLEI Electronic Journal 1(2), 1998. DOI: https://doi.org/10.19153/cleiej.1.2.3.

(I'm lagging, it turns out everything was already figured out by 1998...)

# Baked substring index

The baked substring index uses a q-gram-style approach.

In plain terms: it indexes the unique 1-, 2-, and 3-character fragments in each key.

For short searches like c, co, or cop, it can look up the matching posting list
directly. For longer searches like copper, it looks up the 3-character fragments, intersects the candidate lists, and then verifies the remaining candidates with String.contains.

That last verification step is important. If a key contains cop and ppe, that does not automatically mean it contains copper, so the full substring still gets checked before returning results.

This design is especially interesting for JEI because players often type one character, then two, then three. Those early searches are broad, frequent, and easily the most expensive.

# Baked suffix array index

The baked suffix array index takes a different route.

A suffix array sorts every suffix of the indexed text. Once that sorted array exists, substring search becomes a binary search for the range of suffixes that start with the query. This implementation concatenates all keys into one encoded text array with separators between keys, so matches cannot cross from one key into the next. It stores the encoded text, suffix array, text-position-to-value mapping, and value references. It does not retain the original key strings after build.

If that sounds like insane nonsense to you, please check out the README where I give some examples using bananas, bandanas, and cabanas that even a highly caffeinated monkey could understand.

The suffix-array version is useful because Minecraft already has their own SuffixArray, and I wanted to make something comparable.

# The benchmark

To figure out what actually works, I made a separate benchmark project:

substring-search-benchmarks

It uses Java Microbenchmark Harness (JMH) and a synthetic large-catalog workload. It models:

item names
tooltip lines
mod names
tags
the combined default-search of everything together

For the 100,000-item benchmark, the full combined one has about:

80,292 item-name strings
729,520 tooltip-line strings
430 mod-name strings
7,424 tag strings

That is 817,666 searchable strings total.

The benchmark compares:

baked-substring-index
`baked-suffix-array-index`
Minecraft's built-in SuffixArray
Abahgat's suffix tree
my optimized fork of that suffix tree that has been in JEI for ages

The benchmark report with graphs is here:

https://mezz.github.io/substring-search-benchmarks/

# Results

Here are some results from the 100k-item default-search benchmark run.

Lower is better for all of these.

|Implementation|Approx retained memory|Build time|1-char search|3-char search|11-char search|
|:-|:-|:-|:-|:-|:-|
|Baked substring index|259 MiB|375 ms|0.656 ms|0.024 ms|0.073 ms|
|Baked suffix array index|352 MiB|2440 ms|34.851 ms|0.901 ms|0.030 ms|
|Minecraft suffix array|432 MiB|3695 ms|26.204 ms|0.974 ms|0.068 ms|
|JEI suffix tree|426 MiB|696 ms|5.629 ms|0.283 ms|0.016 ms|
|Abahgat suffix tree|590 MiB|995 ms|53.572 ms|2.121 ms|0.185 ms|

Benchmark: Typed Search Time

Benchmark: Build time per item

Benchmark: Retained Memory

The biggest result for JEI is the memory number.

In this benchmark, the baked substring index retained about 259 MiB, compared to about 426 MiB for the old suffix tree. That is roughly 40% less retained memory for the full default-search test.

The short-search result is also very promising. One-character searches are extremely broad, and JEI runs searches while the player is typing. The baked substring index is built for that case: short queries hit direct fragment posting lists instead of walking a huge
RAM's like a million dollars now so I saved you some in JEI

hey it's me back again with a bigass wall of text about a thing. enjoy!

tl;dr: JEI search has to index a surprisingly large amount of text in big modpacks. I made two new small Java libraries for immutable substring search, then built a JMH benchmark to test a new JEI search-index approach against the current one. The early result: on a synthetic 100k-item JEI-like workload, the most promising new index retained about 259 MiB instead of about 426 MiB, roughly 40% less memory, while also making short searches much faster in the benchmark.

# The optimization problem

JEI search feels simple from the player's side: type a few letters, the item list filters down to show what you're looking for.

Internally, that search box has to deal with a lot of text though. Item names are the obvious part, but JEI also searches things like tooltip lines, mod names, and tags. In a large pack, that can become hundreds of thousands of searchable strings. But I can't just remove those to save RAM! Searching `copper`, a mod name, a tag-like string, or some tooltip text is one of the things that makes JEI feel good to use.

Every nice search feature has a cost. The index has to be built, it has to be kept in memory, and then it has to be fast enough to query repeatedly as the player types.

Since ram is becoming infinitely expensive, I started poking at a very specific question: Can JEI keep the same search features, but use less memory?

# The two new projects

I made two small Java libraries for this:

* [baked-substring-index](https://github.com/mezz/baked-substring-index)
* [baked-suffix-array-index](https://github.com/mezz/baked-suffix-array-index)

Both are designed around the same basic idea:

1. Add key/value pairs to a builder.
2. Build an immutable index.
3. Share that built index for read-only lookup.
4. If the source data changes, build a new index.

That maps pretty well to JEI's use case. The searchable data is built after loading/reloading, then queried many times while the player types. The existing library used in JEI is my adaptation of a [generalized-suffix-tree](https://github.com/mezz/generalized-suffix-tree), which is mutable. I wanted to look at immutable alternatives since they might be more optimized for the bulk of items, and I can keep the old mutable tree around for when mods add more items at runtime.

The libraries are intentionally not trying to solve every search problem that's out there in the wider computer science world. They do exact Java `String` substring matching. It's not pre-AI Google Search, there's no fuzzy search, no ranking, no locale-aware matching, no stemming, no edit distance. If callers want case folding or normalization, they can normalize before indexing and searching.

>References:
>
>Esko Ukkonen, "On-line construction of suffix trees," Algorithmica 14, 249-260, 1995. DOI: [https://doi.org/10.1007/BF01206331](https://doi.org/10.1007/BF01206331)
>
>Udi Manber and Gene Myers. "Suffix Arrays: A New Method for On-Line String Searches." SIAM Journal on Computing, 22(5), 935-948, 1993. DOI: [https://doi.org/10.1137/0222058](https://doi.org/10.1137/0222058).
>
>Fei Shi. "Suffix Arrays for Multiple Strings: A Method for On-Line Multiple String Searches." ASIAN 1996, Lecture Notes in Computer Science 1179, 11-22, 1996. Bibliographic record: [https://dblp.org/rec/conf/asian/Shi96](https://dblp.org/rec/conf/asian/Shi96).
>
>Gonzalo Navarro and Ricardo Baeza-Yates, "A Practical q-Gram Index for Text Retrieval Allowing Errors", CLEI Electronic Journal 1(2), 1998. DOI: [https://doi.org/10.19153/cleiej.1.2.3](https://doi.org/10.19153/cleiej.1.2.3).

(I'm lagging, it turns out everything was already figured out by 1998...)

# Baked substring index

The [baked substring index](https://github.com/mezz/baked-substring-index) uses a q-gram-style approach.

In plain terms: it indexes the unique 1-, 2-, and 3-character fragments in each key.

For short searches like `c`, `co`, or `cop`, it can look up the matching posting list
directly. For longer searches like `copper`, it looks up the 3-character fragments, intersects the candidate lists, and then verifies the remaining candidates with `String.contains`.

That last verification step is important. If a key contains `cop` and `ppe`, that does not automatically mean it contains `copper`, so the full substring still gets checked before returning results.

This design is especially interesting for JEI because players often type one character, then two, then three. Those early searches are broad, frequent, and easily the most expensive.

# Baked suffix array index

The [baked suffix array index](https://github.com/mezz/baked-suffix-array-index) takes a different route.

A suffix array sorts every suffix of the indexed text. Once that sorted array exists, substring search becomes a binary search for the range of suffixes that start with the query. This implementation concatenates all keys into one encoded text array with separators between keys, so matches cannot cross from one key into the next. It stores the encoded text, suffix array, text-position-to-value mapping, and value references. It does not retain the original key strings after build.

If that sounds like insane nonsense to you, please [check out the README](https://github.com/mezz/baked-suffix-array-index/blob/main/README.md) where I give some examples using `banana`s, `bandana`s, and `cabana`s that even a highly caffeinated monkey could understand.

The suffix-array version is useful because Minecraft already has their own `SuffixArray`, and I wanted to make something comparable.

# The benchmark

To figure out what actually works, I made a separate benchmark project:

[substring-search-benchmarks](https://github.com/mezz/substring-search-benchmarks)

It uses [Java Microbenchmark Harness (JMH)](https://github.com/openjdk/jmh) and a synthetic large-catalog workload. It models:

* item names
* tooltip lines
* mod names
* tags
* the combined default-search of everything together

For the 100,000-item benchmark, the full combined one has about:

* 80,292 item-name strings
* 729,520 tooltip-line strings
* 430 mod-name strings
* 7,424 tag strings

That is 817,666 searchable strings total.

The benchmark compares:

* `baked-substring-index`
* `baked-suffix-array-index`
* Minecraft's built-in `SuffixArray`
* Abahgat's suffix tree
* my optimized fork of that suffix tree that has been in JEI for ages

The benchmark report with graphs is here:

[https://mezz.github.io/substring-search-benchmarks/](https://mezz.github.io/substring-search-benchmarks/)

# Results

Here are some results from the 100k-item default-search benchmark run.

Lower is better for all of these.

|Implementation|Approx retained memory|Build time|1-char search|3-char search|11-char search|
|:-|:-|:-|:-|:-|:-|
|Baked substring index|259 MiB|375 ms|0.656 ms|0.024 ms|0.073 ms|
|Baked suffix array index|352 MiB|2440 ms|34.851 ms|0.901 ms|0.030 ms|
|Minecraft suffix array|432 MiB|3695 ms|26.204 ms|0.974 ms|0.068 ms|
|JEI suffix tree|426 MiB|696 ms|5.629 ms|0.283 ms|0.016 ms|
|Abahgat suffix tree|590 MiB|995 ms|53.572 ms|2.121 ms|0.185 ms|

[Benchmark: Typed Search Time](https://preview.redd.it/gszx694ojafh1.png?width=995&format=png&auto=webp&s=d7f6e74764f0ca32d72f3dbf8ca58e18d7f3c421)

[Benchmark: Build time per item](https://preview.redd.it/fmqcdd5kfafh1.png?width=888&format=png&auto=webp&s=a793cc1d4665b2bce9b1e62b6975b055acfc0b63)

[Benchmark: Retained Memory](https://preview.redd.it/kmkro6xvfafh1.png?width=870&format=png&auto=webp&s=d0a835abd146d325144d40ad549450df005dd54f)

The biggest result for JEI is the memory number.

In this benchmark, the baked substring index retained about 259 MiB, compared to about 426 MiB for the old suffix tree. That is roughly 40% less retained memory for the full default-search test.

The short-search result is also very promising. One-character searches are extremely broad, and JEI runs searches while the player is typing. The baked substring index is built for that case: short queries hit direct fragment posting lists instead of walking a huge
suffix range.

Longer searches take longer for the Baked substring index, but keep in mind a frame rate of 120 FPS equals approximately 8.33 milliseconds per frame, so 0.073 milliseconds is not even 1% of a fast frame. It's still really fast.

The baked suffix array is more of an exploration of the suffix-array path. It retains less memory than Minecraft's suffix array here, and it does well on longer selective searches, but it is not great for very short broad searches.

# What this means for JEI

This is not me saying "JEI now uses 40% less memory." because JEI uses memory for other stuff too (like recipes) that I haven't bothered testing because I've been writing about computer science, and making little benchmark graphs about banana cabanas and whatever.

What I do think it shows is:

* JEI search memory can be optimized!
* Immutable baked indexes are a good fit for this
* A q-gram substring index looks like a strong candidate for JEI's new default search behavior

The next step was to integrate the new library into JEI [(committed here)](https://github.com/mezz/JustEnoughItems/commit/31b52164944e8f5824c091845016fb538e67bbf5). It's already released in JEI 30.13.0 for Minecraft 26.2, Jei 29.19.0 for Minecraft 26.1, and I'll continue backporting it to older versions. After that we can try measuring the retained memory in some real packs that people really actually use on real versions of Minecraft released a real long time ago.

# Why post about this?

JEI is 10+ years old and squeezing performance out of it is getting harder, there aren't as many low-hanging fruit as there used to be in the beginning! A lot of performance work is not obvious, and it's rarely one giant fix. It is usually a long trial of staring into the void, followed by a bunch of small tools, benchmarks, failed ideas, and hopefully eventually a useful improvement.

I made the libraries separately so they can be tested, documented, reused, and compared outside JEI. If they turn out to be useful for other mods or tools that need exact substring search over mostly-constant data, cool! Everything is released under the MIT license so people can copy and edit it.

# What you can do

If you are interested in this kind of thing:

1. Check out the benchmark report: [https://mezz.github.io/substring-search-benchmarks/](https://mezz.github.io/substring-search-benchmarks/)
2. Look at the two libraries, there's a lot of details in the READMEs for how they work:
* [https://github.com/mezz/baked-substring-index](https://github.com/mezz/baked-substring-index)
* [https://github.com/mezz/baked-suffix-array-index](https://github.com/mezz/baked-suffix-array-index)
3. If you have search-heavy Java code with data that is built once and queried many times, try them and see if the tradeoffs fit.
4. If you know of some better approaches for this, let me know!

I hope to keep JEI search useful, fast, and leave a little more memory for the game and the mods that really need it. I didn't do the math (and I'm not going to apply logic either) but if every player using JEI saves some ram, I have to reasonably assume that's probably saving billions of dollars, nice!

edit: fixed a graph with a weird y-axis label

https://redd.it/1v5vtib
@MinecraftModded