Rendered at 21:23:17 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
exabrial 3 hours ago [-]
Huge thanks to the OpenJDK Team! I still can't believe we are finding incredible improvements. The JVM is an engineering marvel
kasperni 1 days ago [-]
Are people still dealing with GC issues?
I find that it basically just more or less works out of the box on modern JVMs.
hylaride 1 days ago [-]
There are edge cases where GC issues can crop up, in particular specific "serverless" models (eg AWS lambdas) where the JVM can get "paused" between executions and GC doesn't cleanly run, causing memory to trend upwards until the next cold-start happens (especially if you're running it within a docker container yourself). Limited CPU situations that can exist in these kinds of runtime environments also limit GC in several ways, too.
skullone 1 days ago [-]
What's it like running JVM inside serverless? Python gets interesting enough when it pauses waiting for another call sometimes, the JVM seems like it'd introduce its own interesting things :o
hylaride 1 days ago [-]
Other than some GC issues, meaning we had to give it more memory than we'd otherwise like thereby making them a bit more expensive, it's mostly been fine. The key to ephemeral java is keeping scope limited and not using it when cold starts matter (which can cause a bit of jitter in execution lag). Our use case was asynchronous, so it worked.
We wanted to investigate using the native AWS java lambdas with snapstart and/or using parallel GC, but the place I used them at a year ago was under the tyranny of product having complete control of our backlog, so we didn't get to go that far with it or even more serious tech debt for that matter.
More memory may or may not even be much of an issue depending on your workload. If you can handle the odd OOM error it would have been fine, but for our use case lambdas were so cheap that it didn't really matter, outside of us techies preferring to do it "right". Having the max execution time be less than 15m could also minimize the heap bloat at the cost of more cold starts.
thangalin 24 hours ago [-]
> Are people still dealing with GC issues?
Have you tried real-time audio processing for digital radio communications on a JVM that requires sub-millisecond latency on older, temperature-hardened CPUs?
pron 23 hours ago [-]
Use ZGC.
hedora 21 hours ago [-]
Does it provide hard latency bounds like Azul does (did?), and are they lower than disk/network latencies on modern hardware?
I moved to c++/rust years ago because those languages do, and tens of milliseconds matter for network services. At the time Java could pause for 10’s of seconds, which was 1000x worse than waiting for a spinning disk to seek.
These days, disks are 100s micros to single digit millis, so I guess if Java GC is finally working 30 years after they “fixed” its performance problems, then I’d want to be able to tune ZGC to not pause the app for more than ~ 500us, max.
This article is from last year, but suggests they’re still off by an order of magnitude:
If you hammer a 100GB-1TB heap in steady state with small allocations for, say, a month at 100% CPU, does it eventually do the typical Java thing, where a major compaction takes the process down for seconds or even minutes, or does it just slow down application requests so it can keep up with load?
mdavidn 19 hours ago [-]
> If you hammer a 100GB-1TB heap in steady state with small allocations for, say, a month at 100% CPU, does it eventually do the typical Java thing, where a major compaction takes the process down for seconds or even minutes, or does it just slow down application requests so it can keep up with load?
No. Every garbage collection in Java relocates objects. Compared to malloc, memory fragmentation in long-lived processes is less of a concern. Freelists track only large segments of available memory. The allocator reserves a segment per thread and simply advances a pointer. Small short-lived objects are never visited by the collector. Instead, live siblings are relocated elsewhere before the entire segment is reclaimed.
The above holds true for all of the collectors. The difference is how they deal with concurrent changes to object pointers by the application. Generally, stopping the world uses less net CPU than the memory barriers required by G1GC and ZGC, but most applications are willing to provide more memory and CPU in exchange for shorter pauses.
pron 20 hours ago [-]
> Does it provide hard latency bounds like Azul does (did?), and are they lower than disk/network latencies on modern hardware?
Yes and yes (although we need to be more precise when we talk about latencies; see next paragraph).
> These days, disks are 100s micros to single digit millis, so I guess if Java GC is finally working 30 years after they “fixed” its performance problems, then I’d want to be able to tune ZGC to not pause the app for more than ~ 500us, max.
1. You don't need to tune it. The algorithm simply doesn't collect garbage in stop-the-world pauses.
2. Hiccups are sporadic. They should not be compared to the average latency of normal operation. The relevant question is, is ZGC introducing longer hiccups than those a non-realtime kernel would, and the answer is no.
> This article is from last year, but suggests they’re still off by an order of magnitude
The article doesn't measure GC pauses when it shows latencies (it says: "With ZGC on the other hand, the longest GC pause time observed is ~50 microseconds"). It measures the response latencies of some service. Note that allocation stalls also occur with malloc, it just isn't reported conveniently.
Of course, one of the greatest advantages of moving collectors still applies: Under high allocation rates, moving collectors (but not malloc/free!) allow you to compensate for increased CPU spent on memory management by increasing the heap (i.e. if your allocation rate doubles, you can increase the heap and keep the CPU cost of memory management the same). In the past, this advantage translated to higher throughputs compared to malloc/free, but suffered from GC pauses. Those pauses are gone today.
> If you hammer a 100GB-1TB heap in steady state with small allocations for, say, a month at 100% CPU, does it eventually do the typical Java thing, where a major compaction takes the process down for seconds or even minutes, or does it just slow down application requests so it can keep up with load?
No, it does not. You could, of course, construct some pathological cases where you'd have a high allocation rate for long-lived objects which would result in high CPU utilisation by the GC, but it's easier to get into pathological malloc/free cases in C++ (or Rust) than with ZGC. Let me put it another way: no matter your memory management strategy, it's possible to overwhelm it, but the likelihood that a real, "naive" program would overwhelm a malloc/free allocator is higher than it would the JDK's GCs.
kelseyfrog 1 days ago [-]
It's selection bias. There's a huge number of GC language users, but those who experience problems tend to be the ones who comment.
You're right; the vast majority of people use GC just fine and go about their day. We should update little to none when we see evidence of GC hardship.
2001zhaozhao 22 hours ago [-]
I sometimes run into browser JavaScript GC issues for browser games specifically but on the JVM side i have not run into any pain points for a long time.
(I run a first-person shooter Minecraft server, and for this and other fast-paced gaming in general pause times under a millisecond or so is generally good.)
stmw 1 days ago [-]
It depends a lot on the scale of the system, which often means that those fewer users are actually doing things that are more complex and more valuable - either at the lower end or at the high end.
geodel 24 hours ago [-]
Perhaps. That's why actual JVM GC developers are talking about it.
smallnix 1 days ago [-]
> selecting something else depending on arcane environmental conditions was more of a burden than an advantage
What is being referred to here? Does "else" refer to ZGC, Shenandoah?
> We made G1 the default collector for server environments in JDK 9 (JEP 248). At that time, testing showed that Serial had significant advantages in throughput and footprint in constrained environments with a single CPU or less than 1792 MB of physical memory. We therefore adjusted the JVM's GC selection algorithm to choose Serial in such environments.
Traubenfuchs 12 hours ago [-]
If any JDK wizard is present:
In my opinion, the thing the JRE is REALLY missing is a single process level memory limit setting.
Nowadays you still have to consider the off heap memory when limiting a JRE processes max memory.
To clarify: -XX:MaxRAMPercentage should not exist. Instead the process should be told: „You can use x mb/gb of memory. Use of that what you need for offheap and use the rest for heap.“
That setting is embarassing. Figuring out its value is a mixture of voodoo, vibe-driven guessing and playing the game of „how much wasted memory do you want to risk to prevent a crash?“.
Give us -XX:MaxProcessMemory=4g please. Why doesn‘t this exist?
stmw 1 days ago [-]
GC's haven't freed us from manual memory management, you just do all that manual work with environment variables, or making sure to "pick the right collector for the job", or debugging performance or heap size issues, or chasing down weak references or confused finalizers.
pron 23 hours ago [-]
> you just do all that manual work with environment variables
You really don't anymore. For the past several years, Java's GCs mostly pick the right settings automatically, except for heap size, which will be taken care of soon (https://openjdk.org/jeps/8377305). The reason heap size isn't automatic is that with moving collectors it determines the CPU/RAM tradeoff, and doing that in a more natural way isn't trivial, but we have the algorithm now and will merge it soon.
> or making sure to "pick the right collector for the job"
There are really only four options, most of which are easy to choose among: Parallel for batch jobs where only throughput matters, ZGC for interactive applications where latency matters a lot, and then consider either G1 or Serial if there's a problem with those choices.
As someone who's worked for a long, long time solving manual memory management issues, the amount of effort required isn't just in a different ballpark, but in a different city. Sure, spending a few hours a year to reconsider your settings isn't nothing, but it isn't even remotely in the same category of pain with manual memory management (or even automatic memory management, but with malloc/free underneath).
hedora 20 hours ago [-]
Concretely, what are current tail latencies, worst case?
Ten years ago, “rewrite in C++” was definitely easier than getting the Java GC to stay up under server load.
Most servers I work with run on big machines and are the only process, so figure a 100-250GB heap that lives for months, all async, small requests, so insane amounts of Future and String allocation spam.
Optimizing that stuff away in Java is harder than writing Rust, so assume idiomatic Java.
Also, is there any work on statically enforcing data race freedom in Java? That’s a bigger rust selling point than memory safety for me. I think swift has done some interesting work in that space. It would be nice to get those sorts of safety properties without manually writing borrow checker annotations.
stmw 20 hours ago [-]
As the OC, I think my view is somewhere in the middle - I am neither as optimistic about it being "great now" nor do I think that "rewrite in C++" 10 years ago was easier.
My reason for disagreeing with the former view is that improvements in physical RAM available and tendency towards smaller workloads have allowed many Java (or other GC runtimes) to essentially "fix their problems because hardware got better". So you can waste more RAM, waste more cycles, but "it doesn't matter", and likely it is fine in many cases - but it's is not the same thing as claiming the GC algorithms are responsivle for that outcome. We have been 3 years away from GC solving memory management for at least 30 years.
My reason for disagreeing with the latter view is that for those who don't have 100-250 GB long-lived heaps (or whatever the contemporary version of that is), the pain level is far lower than rewriting in C++ or Rust, or likely the pain level of hiring enough engineers who can do either. It's a completely different engineering culture.
pron 9 hours ago [-]
> So you can waste more RAM, waste more cycles
Just to be clear, the main reason for the use of moving collectors in the first place is to waste less cycles on memory management (otherwise we wouldn't use them). They exist to serve as an optimisation.
> We have been 3 years away from GC solving memory management for at least 30 years.
It's now 3 years in the past (since Generational ZGC); e.g. see https://netflixtechblog.com/bending-pause-times-to-your-will.... Of course, it doesn't solve all imaginable memory management issues, but in practice it makes it a non-issue for a large class of interesting and very common programs.
stmw 3 hours ago [-]
Maybe I should've mentioned at the start that I've implemented several GCs and worked on several Java VM implementations, so I am generally familiar with the tradeoffs between GC algorithms and other runtime details.
Even in the very positive blog you linked, you see statements like
* "ZGC has a fixed overhead 3% of the heap size, requiring more native memory than G1. .." and
* "Reference processing is also only performed in major collections with ZGC. We paid particular attention to deallocation of direct byte buffers, but we haven’t seen any impact thus far. This difference in reference processing did cause a performance problem with JSON thread dump support, but that’s a unusual situation caused by a framework accidentally creating an unused ExecutorService instance for every request."
This was my point about how this sort of thing is a type of manual memory management.
As for waste more RAM, waste more cycles wasn't a statemnt about whether a particular GC is better-performing for certain situations, but that the overall improvement likely has more to do with improvements in CPU speeds and RAM size, than the latest GC version (which tends to simply make a different set of engineering tradeoffs).
pron 2 hours ago [-]
You're right that managing any resource that isn't just Java heap memory requires manual management, but it isn't what we normally mean by memory management. The two have been somewhat tied together traditionally through reference processing (in the sense of reference queues), which is generally something we now discourage in Java programs, and may be deprecated and removed altogether at some point. It's traditionally been used as a convenience. The framework used in that post is, indeed, based on an old library that relies on manual or reference-processing-assisted management of non-heap resources.
> but that the overall improvement likely has more to do with improvements in CPU speeds and RAM size, than the latest GC version (which tends to simply make a different set of engineering tradeoffs).
Well, the biggest improvement has been the creation of a new "pauseless" collector, ZGC, with a novel GC algorithm (at least for OpenJDK), which does _zero_ GC work in STW pauses, i.e. no scanning, no marking, no moving. In particular, even roots, including stacks, are processed entirely concurrently with the program. The main practical impact of that has been saying goodbye to GC pauses, and getting low latency, that is perhaps even more predictable than malloc/free (and obviously, still has higher throughputs in a large class of interesting programs). The tradeoff is the usual footprint tradeoff, which is the core of moving algorithms, as well as more CPU cycles compared to STW collectors (but again, still less than malloc/free in many programs). The additional CPU can, of course, be compensated for with an even larger heap.
The general idea is to use RAM chips as hardware program accelerators, but in the past latency was also something you had to sacrifice, and this is no longer the case today.
hedora 19 hours ago [-]
It definitely was easier for the projects I worked on, but they are exactly the use case where the heap is long lived and most of the machine.
I’ve also worked on systems with lots of small processes, and the operational issues that creates dwarfs GC problems: It takes one middle tier machine, and adds 64-128 network boundaries, and also creates an extremely difficult static memory allocation problem.
I know people do it anyway, but it’s rare that they can articulate a decent technical reason for it, and it wastes something like 90% of the hardware (even in carefully optimized code bases / deployments).
Anyway, I’m not the target market for such stuff.
pron 20 hours ago [-]
> Concretely, what are current tail latencies, worst case?
Well under 1ms for ZGC (to the point that OS-caused hiccups are of similar magnitudes).
> Ten years ago, “rewrite in C++” was definitely easier than getting the Java GC to stay up under server load.
Both could have been hard in some cases, but open-source "pauseless" GCs are only 3 years old (and all of the JDK's GCs are nothing like what they were ten years ago).
> Optimizing that stuff away in Java is harder than writing Rust, so assume idiomatic Java.
Quite the opposite. Performance issues due to memory management are, in practice, more serious in Rust than they are in modern Java.
> Also, is there any work on statically enforcing data race freedom in Java?
There isn't much demand for that atm. If we see growing demand, we could prioritise it.
hedora 19 hours ago [-]
In rust, I usually just make sure stuff is not Box<>, and try to reuse buffers. That generally gets the memory allocator completely out of the way (except for async).
The remaining allocator performance problems are mostly due to it zeroing allocated memory unless I use unsafe. Is java able to stackify most new Object calls and elide default initialization of object members these days?
I’m surprised to hear there is no demand for compiler enforced/facilitated thread safety in Java. That was a major pain point in all the Java code bases I’ve worked with in the past, and is a headline safety feature for rust (which goes even further and enforces aliasing rules) and JS. Could you be seeing selection bias in your user base?
noelwelsh 11 hours ago [-]
> Is java able to stackify most new Object calls and elide default initialization of object members these days?
Escape analysis in OpenJDK will stack allocate values where it can show it is safe to do so. Project Valhalla is also reducing the memory footprint of objects.
As for thread safety, that is more of a language concern than a runtime one. Amongst JVM languages Scala is leading here AFAIK. Its "capture checking"[1] provides thread safety (e.g. [2]) and actually covers escape analysis as well. On Scala Native (the native code backend for Scala) capture checking can be used for safe stack allocation and safe arena allocation.
> In rust, I usually just make sure stuff is not Box<>, and try to reuse buffers. That generally gets the memory allocator completely out of the way (except for async).
You say "just", but this is easy when programs are small. The problem is that this gets harder and harder and harder as programs grow large (the whole point of the JVM's design was to address the performance issues that plague large C++ programs). E.g. someone who works at one of the world's largest tech companies just told me that they have problems with Rust programs spending 30% of their CPU on memory management even when they're as small as a couple hundreds of thousands of LOC.
> Is java able to stackify most new Object calls and elide default initialization of object members these days?
No, the general idea is to just make memory management efficient (although some objects are "stackified" and the compiler will elide zeroing when non-defaults are passed to a constructor). Now, I say "just", but this used to come at the cost of GC pauses and larger footprint. Now it only comes at the cost of a larger footprint.
But there is a definite choice here when it comes to performance. Low level languages give you control that means performance is attained through manual effort. Java takes away control to improve effort-per-performance. Roughly speaking, these tradeoffs mean that when programs are small and the extra effort is manageable, low-level languages are hard to beat, but when programs are large, it is Java that is hard to beat.
> I’m surprised to hear there is no demand for compiler enforced/facilitated thread safety in Java. That was a major pain point in all the Java code bases I’ve worked with in the past, and is a headline safety feature for rust (which goes even further and enforces aliasing rules) and JS.
This used to be a bigger problem when locks were the main mechanism for sharing data among threads. Now, with the wide selection of concurrent data structures, such problems don't occur as much. I'm not saying they don't occur at all, just not frequently enough to become a major priority.
Also, safe Rust's data-race freedom comes at the cost of requiring unsafe for benign races, which are not uncommon in concurrent algorithms (i.e. it excludes even "good" races). This may be fine in languages whose view on performance is "with enough effort you can get good performance", but, as I said, Java is about making more "naive" programs fast with little effort.
stmw 3 hours ago [-]
It is fair that there are many ways to be slow in any number of programming languages. I'm surprised to hear "Rust programs spending 30% of their CPU on memory management even when they're as small as a couple hundreds of thousands of LOC", although I can visualize some unique workloads where that's unavoidable irrespective of language & runtime.
I find that it basically just more or less works out of the box on modern JVMs.
We wanted to investigate using the native AWS java lambdas with snapstart and/or using parallel GC, but the place I used them at a year ago was under the tyranny of product having complete control of our backlog, so we didn't get to go that far with it or even more serious tech debt for that matter.
More memory may or may not even be much of an issue depending on your workload. If you can handle the odd OOM error it would have been fine, but for our use case lambdas were so cheap that it didn't really matter, outside of us techies preferring to do it "right". Having the max execution time be less than 15m could also minimize the heap bloat at the cost of more cold starts.
Have you tried real-time audio processing for digital radio communications on a JVM that requires sub-millisecond latency on older, temperature-hardened CPUs?
I moved to c++/rust years ago because those languages do, and tens of milliseconds matter for network services. At the time Java could pause for 10’s of seconds, which was 1000x worse than waiting for a spinning disk to seek.
These days, disks are 100s micros to single digit millis, so I guess if Java GC is finally working 30 years after they “fixed” its performance problems, then I’d want to be able to tune ZGC to not pause the app for more than ~ 500us, max.
This article is from last year, but suggests they’re still off by an order of magnitude:
https://www.morling.dev/blog/lower-java-tail-latencies-with-...
Also, that’s measuring a 30 second window.
If you hammer a 100GB-1TB heap in steady state with small allocations for, say, a month at 100% CPU, does it eventually do the typical Java thing, where a major compaction takes the process down for seconds or even minutes, or does it just slow down application requests so it can keep up with load?
No. Every garbage collection in Java relocates objects. Compared to malloc, memory fragmentation in long-lived processes is less of a concern. Freelists track only large segments of available memory. The allocator reserves a segment per thread and simply advances a pointer. Small short-lived objects are never visited by the collector. Instead, live siblings are relocated elsewhere before the entire segment is reclaimed.
The above holds true for all of the collectors. The difference is how they deal with concurrent changes to object pointers by the application. Generally, stopping the world uses less net CPU than the memory barriers required by G1GC and ZGC, but most applications are willing to provide more memory and CPU in exchange for shorter pauses.
Yes and yes (although we need to be more precise when we talk about latencies; see next paragraph).
> These days, disks are 100s micros to single digit millis, so I guess if Java GC is finally working 30 years after they “fixed” its performance problems, then I’d want to be able to tune ZGC to not pause the app for more than ~ 500us, max.
1. You don't need to tune it. The algorithm simply doesn't collect garbage in stop-the-world pauses.
2. Hiccups are sporadic. They should not be compared to the average latency of normal operation. The relevant question is, is ZGC introducing longer hiccups than those a non-realtime kernel would, and the answer is no.
> This article is from last year, but suggests they’re still off by an order of magnitude
The article doesn't measure GC pauses when it shows latencies (it says: "With ZGC on the other hand, the longest GC pause time observed is ~50 microseconds"). It measures the response latencies of some service. Note that allocation stalls also occur with malloc, it just isn't reported conveniently.
Of course, one of the greatest advantages of moving collectors still applies: Under high allocation rates, moving collectors (but not malloc/free!) allow you to compensate for increased CPU spent on memory management by increasing the heap (i.e. if your allocation rate doubles, you can increase the heap and keep the CPU cost of memory management the same). In the past, this advantage translated to higher throughputs compared to malloc/free, but suffered from GC pauses. Those pauses are gone today.
> If you hammer a 100GB-1TB heap in steady state with small allocations for, say, a month at 100% CPU, does it eventually do the typical Java thing, where a major compaction takes the process down for seconds or even minutes, or does it just slow down application requests so it can keep up with load?
No, it does not. You could, of course, construct some pathological cases where you'd have a high allocation rate for long-lived objects which would result in high CPU utilisation by the GC, but it's easier to get into pathological malloc/free cases in C++ (or Rust) than with ZGC. Let me put it another way: no matter your memory management strategy, it's possible to overwhelm it, but the likelihood that a real, "naive" program would overwhelm a malloc/free allocator is higher than it would the JDK's GCs.
You're right; the vast majority of people use GC just fine and go about their day. We should update little to none when we see evidence of GC hardship.
(I run a first-person shooter Minecraft server, and for this and other fast-paced gaming in general pause times under a millisecond or so is generally good.)
What is being referred to here? Does "else" refer to ZGC, Shenandoah?
What does arcane env conditions mean?
> We made G1 the default collector for server environments in JDK 9 (JEP 248). At that time, testing showed that Serial had significant advantages in throughput and footprint in constrained environments with a single CPU or less than 1792 MB of physical memory. We therefore adjusted the JVM's GC selection algorithm to choose Serial in such environments.
In my opinion, the thing the JRE is REALLY missing is a single process level memory limit setting.
Nowadays you still have to consider the off heap memory when limiting a JRE processes max memory.
To clarify: -XX:MaxRAMPercentage should not exist. Instead the process should be told: „You can use x mb/gb of memory. Use of that what you need for offheap and use the rest for heap.“
That setting is embarassing. Figuring out its value is a mixture of voodoo, vibe-driven guessing and playing the game of „how much wasted memory do you want to risk to prevent a crash?“.
Give us -XX:MaxProcessMemory=4g please. Why doesn‘t this exist?
You really don't anymore. For the past several years, Java's GCs mostly pick the right settings automatically, except for heap size, which will be taken care of soon (https://openjdk.org/jeps/8377305). The reason heap size isn't automatic is that with moving collectors it determines the CPU/RAM tradeoff, and doing that in a more natural way isn't trivial, but we have the algorithm now and will merge it soon.
> or making sure to "pick the right collector for the job"
There are really only four options, most of which are easy to choose among: Parallel for batch jobs where only throughput matters, ZGC for interactive applications where latency matters a lot, and then consider either G1 or Serial if there's a problem with those choices.
As someone who's worked for a long, long time solving manual memory management issues, the amount of effort required isn't just in a different ballpark, but in a different city. Sure, spending a few hours a year to reconsider your settings isn't nothing, but it isn't even remotely in the same category of pain with manual memory management (or even automatic memory management, but with malloc/free underneath).
Ten years ago, “rewrite in C++” was definitely easier than getting the Java GC to stay up under server load.
Most servers I work with run on big machines and are the only process, so figure a 100-250GB heap that lives for months, all async, small requests, so insane amounts of Future and String allocation spam.
Optimizing that stuff away in Java is harder than writing Rust, so assume idiomatic Java.
Also, is there any work on statically enforcing data race freedom in Java? That’s a bigger rust selling point than memory safety for me. I think swift has done some interesting work in that space. It would be nice to get those sorts of safety properties without manually writing borrow checker annotations.
My reason for disagreeing with the former view is that improvements in physical RAM available and tendency towards smaller workloads have allowed many Java (or other GC runtimes) to essentially "fix their problems because hardware got better". So you can waste more RAM, waste more cycles, but "it doesn't matter", and likely it is fine in many cases - but it's is not the same thing as claiming the GC algorithms are responsivle for that outcome. We have been 3 years away from GC solving memory management for at least 30 years.
My reason for disagreeing with the latter view is that for those who don't have 100-250 GB long-lived heaps (or whatever the contemporary version of that is), the pain level is far lower than rewriting in C++ or Rust, or likely the pain level of hiring enough engineers who can do either. It's a completely different engineering culture.
Just to be clear, the main reason for the use of moving collectors in the first place is to waste less cycles on memory management (otherwise we wouldn't use them). They exist to serve as an optimisation.
> We have been 3 years away from GC solving memory management for at least 30 years.
It's now 3 years in the past (since Generational ZGC); e.g. see https://netflixtechblog.com/bending-pause-times-to-your-will.... Of course, it doesn't solve all imaginable memory management issues, but in practice it makes it a non-issue for a large class of interesting and very common programs.
Even in the very positive blog you linked, you see statements like * "ZGC has a fixed overhead 3% of the heap size, requiring more native memory than G1. .." and * "Reference processing is also only performed in major collections with ZGC. We paid particular attention to deallocation of direct byte buffers, but we haven’t seen any impact thus far. This difference in reference processing did cause a performance problem with JSON thread dump support, but that’s a unusual situation caused by a framework accidentally creating an unused ExecutorService instance for every request."
This was my point about how this sort of thing is a type of manual memory management.
As for waste more RAM, waste more cycles wasn't a statemnt about whether a particular GC is better-performing for certain situations, but that the overall improvement likely has more to do with improvements in CPU speeds and RAM size, than the latest GC version (which tends to simply make a different set of engineering tradeoffs).
> but that the overall improvement likely has more to do with improvements in CPU speeds and RAM size, than the latest GC version (which tends to simply make a different set of engineering tradeoffs).
Well, the biggest improvement has been the creation of a new "pauseless" collector, ZGC, with a novel GC algorithm (at least for OpenJDK), which does _zero_ GC work in STW pauses, i.e. no scanning, no marking, no moving. In particular, even roots, including stacks, are processed entirely concurrently with the program. The main practical impact of that has been saying goodbye to GC pauses, and getting low latency, that is perhaps even more predictable than malloc/free (and obviously, still has higher throughputs in a large class of interesting programs). The tradeoff is the usual footprint tradeoff, which is the core of moving algorithms, as well as more CPU cycles compared to STW collectors (but again, still less than malloc/free in many programs). The additional CPU can, of course, be compensated for with an even larger heap.
The general idea is to use RAM chips as hardware program accelerators, but in the past latency was also something you had to sacrifice, and this is no longer the case today.
I’ve also worked on systems with lots of small processes, and the operational issues that creates dwarfs GC problems: It takes one middle tier machine, and adds 64-128 network boundaries, and also creates an extremely difficult static memory allocation problem.
I know people do it anyway, but it’s rare that they can articulate a decent technical reason for it, and it wastes something like 90% of the hardware (even in carefully optimized code bases / deployments).
Anyway, I’m not the target market for such stuff.
Well under 1ms for ZGC (to the point that OS-caused hiccups are of similar magnitudes).
> Ten years ago, “rewrite in C++” was definitely easier than getting the Java GC to stay up under server load.
Both could have been hard in some cases, but open-source "pauseless" GCs are only 3 years old (and all of the JDK's GCs are nothing like what they were ten years ago).
> Optimizing that stuff away in Java is harder than writing Rust, so assume idiomatic Java.
Quite the opposite. Performance issues due to memory management are, in practice, more serious in Rust than they are in modern Java.
> Also, is there any work on statically enforcing data race freedom in Java?
There isn't much demand for that atm. If we see growing demand, we could prioritise it.
The remaining allocator performance problems are mostly due to it zeroing allocated memory unless I use unsafe. Is java able to stackify most new Object calls and elide default initialization of object members these days?
I’m surprised to hear there is no demand for compiler enforced/facilitated thread safety in Java. That was a major pain point in all the Java code bases I’ve worked with in the past, and is a headline safety feature for rust (which goes even further and enforces aliasing rules) and JS. Could you be seeing selection bias in your user base?
Escape analysis in OpenJDK will stack allocate values where it can show it is safe to do so. Project Valhalla is also reducing the memory footprint of objects.
As for thread safety, that is more of a language concern than a runtime one. Amongst JVM languages Scala is leading here AFAIK. Its "capture checking"[1] provides thread safety (e.g. [2]) and actually covers escape analysis as well. On Scala Native (the native code backend for Scala) capture checking can be used for safe stack allocation and safe arena allocation.
[1]: https://docs.scala-lang.org/scala3/reference/experimental/cc... [2]: https://softwaremill.com/understanding-capture-checking-in-s...
You say "just", but this is easy when programs are small. The problem is that this gets harder and harder and harder as programs grow large (the whole point of the JVM's design was to address the performance issues that plague large C++ programs). E.g. someone who works at one of the world's largest tech companies just told me that they have problems with Rust programs spending 30% of their CPU on memory management even when they're as small as a couple hundreds of thousands of LOC.
> Is java able to stackify most new Object calls and elide default initialization of object members these days?
No, the general idea is to just make memory management efficient (although some objects are "stackified" and the compiler will elide zeroing when non-defaults are passed to a constructor). Now, I say "just", but this used to come at the cost of GC pauses and larger footprint. Now it only comes at the cost of a larger footprint.
But there is a definite choice here when it comes to performance. Low level languages give you control that means performance is attained through manual effort. Java takes away control to improve effort-per-performance. Roughly speaking, these tradeoffs mean that when programs are small and the extra effort is manageable, low-level languages are hard to beat, but when programs are large, it is Java that is hard to beat.
> I’m surprised to hear there is no demand for compiler enforced/facilitated thread safety in Java. That was a major pain point in all the Java code bases I’ve worked with in the past, and is a headline safety feature for rust (which goes even further and enforces aliasing rules) and JS.
This used to be a bigger problem when locks were the main mechanism for sharing data among threads. Now, with the wide selection of concurrent data structures, such problems don't occur as much. I'm not saying they don't occur at all, just not frequently enough to become a major priority.
Also, safe Rust's data-race freedom comes at the cost of requiring unsafe for benign races, which are not uncommon in concurrent algorithms (i.e. it excludes even "good" races). This may be fine in languages whose view on performance is "with enough effort you can get good performance", but, as I said, Java is about making more "naive" programs fast with little effort.