10B Integers Walk into an Array
An experiment using Pharo Smalltalk demonstrated storing 10 billion SmallInteger instances, highlighting 64-bit addressing advantages over Java's limitations, particularly in memory optimization and handling large data sets.
Read original article
The article discusses an experiment conducted using Pharo Smalltalk to store 10 billion SmallInteger instances in an array, highlighting the capabilities of 64-bit addressing in modern programming environments. The author reflects on their past experiences with 32-bit Smalltalk and the limitations of Java's array size, which is capped at 2³¹-1 elements. With a new MacBook Pro M2 Max equipped with 96GB of RAM, the author successfully created arrays exceeding 2 billion elements, ultimately reaching 10 billion without running out of memory. The experiment revealed that Pharo Smalltalk optimizes memory usage by storing SmallInteger values inline rather than as references, contrasting with Java's handling of large numbers, which can lead to overflow issues. The author also attempted a similar experiment in Java using third-party libraries like fastutil and Eclipse Collections, noting the differences in indexing and data handling between the two languages. The results demonstrated the challenges of managing large data sets in Java, particularly with respect to data type limitations. Overall, the article emphasizes the advancements in programming languages and hardware that allow for the manipulation of vast amounts of data.
- Pharo Smalltalk can store over 10 billion SmallInteger instances in an array due to its 64-bit architecture.
- Java arrays are limited to 2³¹-1 elements, necessitating third-party libraries for larger data sets.
- Pharo optimizes memory by storing SmallInteger values inline, unlike Java's reference-based storage.
- The author conducted experiments on a MacBook Pro M2 Max with 96GB of RAM.
- Differences in data handling between Smalltalk and Java can lead to overflow issues in Java.
Related
Smalltalk syntax in 7 minutes [video]
The YouTube video explains Smalltalk syntax, emphasizing readability and message-based object interaction. It covers keywords, arrays, closures, and method execution in Pharo Smalltalk, providing a practical example and additional learning resources.
Some Tricks from the Scrapscript Compiler
The Scrapscript compiler implements optimization tricks like immediate objects, small strings, and variants for better performance. It introduces immediate variants and const heap to enhance efficiency without complexity, seeking suggestions for future improvements.
Overview of cross-architecture portability problems
Michał Górny discusses cross-architecture portability challenges between 32-bit and 64-bit systems, highlighting issues with memory allocation, file size limitations, and the Y2K38 problem affecting multiple programming languages.
Eliminating Intermediate Array Allocations
The article highlights Ruby's memory allocation optimization, particularly in tokenizers and array operations, emphasizing that simple literals do not allocate memory and that idiomatic Ruby can be performant.
The costs of the i386 to x86-64 upgrade
The article examines the shift from 32-bit to 64-bit architecture, discussing its impact on code density, memory usage, performance, and the evolution of software demands and hardware capabilities.
2 comments"Oh, we're a numerical application, not a server one."
10b integers get pushed into an array. Does one need an unfold operation to C the union of (&*p) bits get "twidded n by n, ..." and 'ants go marching n x n' [1].
Couldn't find an appropriate utf-32 glyph for the base case.
Does the 'BOOM' happen at 0, NULL or NILL?
------
Limited HN ability to Hollerith when there's to much Babbage[2], (aka insufficient HN article space), for Simon/Garfunkel electric glow wall for the perl camel example.
------
[0]: duff's device : https://en.wikipedia.org/wiki/Duff%27s_device proto threads : https://dunkels.com/adam/pt/ coroutines : http://www.chiark.greenend.org.uk/~sgtatham/coroutines.html
[1] : file:///home/surf/Downloads/counting-and-math-rhymes-002-the-ants-go-marching.pdf
Related
Smalltalk syntax in 7 minutes [video]
The YouTube video explains Smalltalk syntax, emphasizing readability and message-based object interaction. It covers keywords, arrays, closures, and method execution in Pharo Smalltalk, providing a practical example and additional learning resources.
Some Tricks from the Scrapscript Compiler
The Scrapscript compiler implements optimization tricks like immediate objects, small strings, and variants for better performance. It introduces immediate variants and const heap to enhance efficiency without complexity, seeking suggestions for future improvements.
Overview of cross-architecture portability problems
Michał Górny discusses cross-architecture portability challenges between 32-bit and 64-bit systems, highlighting issues with memory allocation, file size limitations, and the Y2K38 problem affecting multiple programming languages.
Eliminating Intermediate Array Allocations
The article highlights Ruby's memory allocation optimization, particularly in tokenizers and array operations, emphasizing that simple literals do not allocate memory and that idiomatic Ruby can be performant.
The costs of the i386 to x86-64 upgrade
The article examines the shift from 32-bit to 64-bit architecture, discussing its impact on code density, memory usage, performance, and the evolution of software demands and hardware capabilities.