Jump to content
Gopher's Minions

Hopper Lag - Problem and Solution


Recommended Posts

Hoppers in and of themselves do not create significant server lag. Only uncovered hoppers create rule-worthy lag. The lag is not being caused by the item-moving code of the hopper blocks. The lag is being caused by the search-for-item code only on uncovered hoppers. When the top of a hopper is not covered by a block-type having an inventory slot (so that the hopper will switch to pull-from-container mode), the hopper block constantly checks the block above it for things to suck in. Solid blocks (non-inventory-slot blocks) don't toggle the hopper mode.

In respect of "hopper lag", we currently have an informal rule (per Rodzyn on in-game chat) of about 5 hoppers in length for a hopper chain. However, we're still seeing an escalation of server lag as we build out the map. Looking into it briefly, I find that limiting length of hopper chains is not going to help us, as is shown in this recorded test:

 

Furthermore, one of the most common reasons to build a long hopper chain is not creating significant hopper lag:

6dJepnI.png

 

A hopper-driven inventory sorter creates hopper lag only on its top row of hoppers, and that can be fixed with inventory slot blocks, per Accidental Gaming's demonstration. Simply place furnaces on top of the hoppers on the top row, and the entire mechanism will be relatively lag-free. A single-hopper chicken egg machine is probably creating more lag per-machine than any item sorter would do (if all of the item sorter hoppers are covered properly), because the sorter is effectively the same structure as an auto-furnace, and its redstone circuit is nearly always emitting no signal. The player placing items into a sorter or furnace "input" chest is the same as the player toggling an on/off switch.

 

http://www.minecraftforum.net/forums/mapping-and-modding/minecraft-tools/1262291-tool-blockfinder-0-9-1

If you feed the server's region files into the blockfinder tool, looking for hoppers, you'll probably find much of the ongoing lag problem: instances of uncovered hoppers, increasing over time.

Make it a rule to put "inventory slot" blocks on top of hopper chains, and discourage uncovered hopper machines instead.

 

Gamepedia Minecraft Wiki has another review of the problem, showing that "activated hoppers" are the problem, and if you understand what "activated" means in this context, it matches with the rest. Covering a hopper with an inventory-slot block deactivates the hopper until an item is in the neighbor block's inventory. Thusly-covered hoppers are essentially lag-free because they are rarely activated.

Edited by All_Of_My_Nope
Link to post
Share on other sites

That's very interesting and helpful research. I wasn't aware of that difference between "activated" and "deactivated" hopper, from now on it will be like you suggested, amount of hoppers won't be big issue as long as they're all properly covered with inventory blocks.

Main reason I opted for low hopper rule was to keep it down to reasonable levels, single person might not be able to cause significant lag but having dozens of people putting dozens of hoppers piles up. However as shown at the video, as long as hoppers are covered, you need really huge numbers to make it a problem.

Link to post
Share on other sites
1 hour ago, Rodzyn said:

Main reason I opted for low hopper rule was to keep it down to reasonable levels, single person might not be able to cause significant lag but having dozens of people putting dozens of hoppers piles up.

Definitely; I agreed and left it at that until someone on in-game chat mentioned the furnace thing. Google showed a reddit thread and that video. That left me to wonder if the server's unique player count isn't high enough for permanently activated hoppers to accumulate over time as people craft them for normal open-topped hopper purposes (they're ostensibly allowed about 5 of those). That's at least 1:5 of player:hopper, and we can count on some people building none while others build more than 5 and hide them in the world.

That's where the blockfinder tool comes in. >.>

Link to post
Share on other sites

That is actually really interesting to learn, but makes a reasonably strong amount of sense now that I know it.  The code to detect item entities in the world would have to, logically, be more complex and more lines of code to run than the items in inventory code which (last I checked) is still much more significant than mod added equivalents to hoppers (such as Buildcraft chutes).

Link to post
Share on other sites

Also quite relavent would be the fact that item drop checks would most likely be quite frequent, container checks on the other hand are not just less frequent in general but Bukkit configs have customisation options. I know for a fact that Xelphos set container checks to be 3 times less frequent and to balance it made hoppers move 3 items at a time. Those settings would make the lag gap between active and inactive hopper even more noticable.

Link to post
Share on other sites
2 hours ago, Rodzyn said:

I know for a fact that Xelphos set container checks to be 3 times less frequent and to balance it made hoppers move 3 items at a time. Those settings would make the lag gap between active and inactive hopper even more noticable.

Good man, that one.

https://hub.spigotmc.org/jira/browse/SPIGOT-2523

That person reports going to similar but further lengths in tackling the problem, without completely solving it.

 

https://www.spigotmc.org/wiki/spigot-configuration/

Check if max-tick-time is tuned to their suggestion—

Quote

Values between 10 - 20 for tiles and 20 - 25 for entities have been reported to provide a good performance increase.

—but note that hopper-alt-ticking, hopper-amount, and ticks-per hopper-check have all been removed from Spigot code at MC version 1.8.3; If those are in your config, they're being ignored. The code was removed because it broke hoppers and machines depending on them.

Edit: On second reading, it looks like "ticks-per hopper-transfer" is still in.

 

7 minutes ago, Just Louise said:

Minion Standard of Excellency: Platinum Level gopherVaultStiv

Thanks for doing the research. Now if only I had some iron for a hopper... :P

Google is a powerful drug. gopherVaultStiv

Edited by All_Of_My_Nope
Link to post
Share on other sites

https://www.spigotmc.org/wiki/reducing-lag/

This was updated last month. Some thoughts as I read it:

Feels like Xel already lowered merge-radius.

I know item-despawn-rate is higher than 1 minute, and agree that setting that aggressively low will also require keep-inventory to be on (therefore lowering risk and difficulty). Yet, Minion Land is already effectively a build-only server, so assuming that in a BungeeCord setup you have separate config for each linked server, just let Minion Land (the big laggy one) have that, and let Far Lands be the riskier map.

Our view-distance is at 10 (I think), and that's already too low for preference, so maybe that's the big lag we just live with.

Entity-activation-range is already fairly low; low enough to break most mob farming attempts.

Wiki author is unaware that ticks-per hopper-check isn't in play, or they're running a <= 1.8.2 server, or they're using PaperMC with use-hopper-check set true.

 

On that note:

https://github.com/PaperMC/Paper/issues/197

They're attacking the problem by forking Spigot, and in May claimed to have a performance improvement fix that replaces their need for use-hopper-check.

https://www.beastnode.com/portal/knowledgebase/164/How-to-Optimize-Performance-using--Spigot-or-PaperSpigot.html

Basically the same information as before, but aimed at PaperMC.

Edited by All_Of_My_Nope
Link to post
Share on other sites

I was curious about difference between hopper handling between vanilla and Spigot and stumbled upon this test:

If what's shown here (it's for 1.8 version of Minecraft) then placing furnaces might actually be counter productive. At the same time it shows that hoppers even in their active state should not be significant culprits in lag situation.

I don't know our current settings in spigot.yml but I already had in mind taking interest in max-tick-time setting. I took a quick peek at my config for Stivarstead and apparently I had it set to 1000.. can't remember why I had it set so high, if that's the default or suggestion I picked up but if that's default and Xelphos didn't change it then it's way higher than suggested values.

Link to post
Share on other sites
15 hours ago, Rodzyn said:

I was curious about difference between hopper handling between vanilla and Spigot and stumbled upon this test:

If what's shown here (it's for 1.8 version of Minecraft) then placing furnaces might actually be counter productive. At the same time it shows that hoppers even in their active state should not be significant culprits in lag situation.

I don't know our current settings in spigot.yml but I already had in mind taking interest in max-tick-time setting. I took a quick peek at my config for Stivarstead and apparently I had it set to 1000.. can't remember why I had it set so high, if that's the default or suggestion I picked up but if that's default and Xelphos didn't change it then it's way higher than suggested values.

 

Good catch with this video. That extra TPS loss in Spigot 1.8.* for covering (what was that, 250,000?) hoppers with an inventory block is surprising. Hopper lag is still present (with a much higher ceiling on hopper count) in large Spigot servers, and the Internets hath lead me astray on the furnace thing for Spigot. Glad you found the video, cuz I felt like we'd either have to test it ourselves or assume that Spigot referenced Vanilla for what lead to that quirk in Vanilla.

I still wouldn't take from that demonstration that hoppers are insignificant across the lifetime of a server. They get built for a purpose and then left behind to tick away at nothing. It's an overhead problem. Video shows that Spigot at c. 1.8.* changed their hopper code significantly enough to remove their previous non-Vanilla behavior at 1.8.3 (the work-around code that gave the hopper tick config variable they removed), then being something like 4 times as efficient at calculating empty hopper ticks, and Spigot also changed hoppers so that they are more efficient in uncovered state.

It's still significant, but few times less significant than some other MC admin discussions lead me to believe, where they were failing to differentiate between server implementations.

With only hoppers on an empty stage, he took Spigot 1.8.* TPS to 17 average at 250,000 hoppers. For a server of 1,000 people having built over the server's lifetime, that is 250 hoppers per person. It isn't unlikely that we could approach that many hoppers before the end of this server's life, through MC folks doing MC things, and that creates a baseline amount of always-ticking CPU work that erodes the server's free time for every other normal MC thing (player actions and mob actions).

Approaching that point would mean that we'd be ticking server-side at nearly half of 30FPS, just from hopper overhead alone. We don't want that, so it is significant.

Suppose that means we should still be looking for "excesses" in hoppers, but can safely leave alone most well-made automations using them, and single-use instances (mailboxes, donation boxes, etc.) That makes hoppers something to occasional look at, to manage their contribution to overhead, rather than something to actively enforce daily.

Edited by All_Of_My_Nope
Link to post
Share on other sites
  • 2 months later...
On 15.10.2016 at 5:43 AM, Rodzyn said:

I know for a fact that Xelphos set container checks to be 3 times less frequent and to balance it made hoppers move 3 items at a time. Those settings would make the lag gap between active and inactive hopper even more noticable.

Sorry to necro the thread - but I've been racking my brain for the past two days trying to make an auto-shop design work on the Farlands.

The setup mentioned could finally be the explanation I've been trying to find.

The design I'm using is supposed to do a 1:3 ratio transaction utilizing a simple dual hopper filter system. Somehow though for every transaction the lower hopper would pull 3 instead of one item, kinda ruining the system. (filter setup: 18/1/1/1 filter with a comparator sending a signal for any additional items defined by the filter, activating the pulling hopper for one check then locking up again - it keeps depleting the filter to 16/1/1/1 though instead of the expected 18/1/1/1).

Do I understand that correctly? And if so - is there a way to adapt? Or am I stuck with making my shop's "cost" in multiples of 3?

Edited by Bob_Bungle
Link to post
Share on other sites

I did find a way to work around the limitation utilizing a seperate circuit to refill the hopper with 2 items for every item thrown in - but that kinda means i'll have to resort the items in the system so often that it kind of defeats the purpose of it being an "auto"-shop (besides it making the build unreasonably big)

Edited by Bob_Bungle
Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

  • Recently Browsing   0 members

    No registered users viewing this page.

×
×
  • Create New...