Fun with Go Iterators

I work with Go a lot at my job, and I definitely prefer functional programming so I chafe at the language a bit. I was excited to incorporate iterators into our code. Unfortunately, I'm also working in an environment where we are memory constrained, so on our hot path we need to not allocate at all. I tried playing with the iterators a bit, and couldn't get it to produce something that didn't allocate. I got close, but as much as a tried I couldn't get below 1 allocation per loop (not per loop iteration, per loop). Which in any other setting would be fine, but not for our use case.

Call me crazy, but I don't like any of this. Make more named functions. Keep your logic flat and explicit. I believe go wants you to code this way as well. Imagine the horrors this kind of function chaining creates. Actually, you don't have to. It's JavaScript.

I absolutely love it when we can take advantage of Go's type system and add additional traits and behaviors to existing types like this.

That said, I noticed something odd here. In order for a module like this to really shine, I think all these operations need to be functionally pure. Right now, some of these mutate the iterator's `iter` method mid-stream, which is about as side-effect-ful as you can get.

```

func (i Iterator[V]) Map(f func(V) V) Iterator[V] {

cpy := i.iter

i.iter = func(yield func(V) bool) {

  for v := range cpy {

   v = f(v)

   if !yield(v) {

    return

   }

  }

 }

 return i

} ```

Unless I'm misreading that, `i.iter` has new behavior after this call. A better way would be to return a new _iterator_ with the custom iter behavior instead.

``` func (i Iterator[V]) Map(f func(V) V) Iterator[V] {

        // create a fresh iterator around a custom closure (NewIterator() is hypothetical in this case)

 return NewIterator(func(yield func(V) bool) {

  for v := range i.iter {

   v = f(v)

   if !yield(v) {

    return

   }

  }

 })

} ```

Go people will do this and they'll be content:

  a := []int{1,2,3,4}
  it := slices.All(a)
  it = slices.Reverse(it)
  it = slices.Map(it)
  it = slices.Filter(it, func(i int) bool { return i % 2 == 0 })
  slices.ForEach(it, func(i int) { fmt.Println(i) })

I don't judge the Go enjoyers, but I prefer writing TypeScript to Go which says it all.

Type-inferred arrow lambda for function arguments would go such a long way in making this code nicer... And not make compilation slower at all.

  it = slices.Filter(it, i => i % 2 == 0)
  slices.ForEach(it, i => fmt.Println(i))

Unfortunately the chain approach breaks down when if you need to `map` to a different type, for example. Go does not allow generic typed methods.

Might be interesting to make a library that competes with https://github.com/samber/lo?

> My issue with the go way of iterators is, you can’t chain them like you would in JavaScript

You are not supposed to chain them. This addiction to try and chain everything everywhere all the time is so freaking weird and has been for a very long time.

Not only you are completely losing grasp on what is going on and write code prone to errors, but you are making it unreadable for other people that will be maintaining or just reading your code who will come long after you are gone from the company or abandon your library.

This is where Go's simplicity approach and splitting each action into its own for loop or block of code is a godsend for maintainability.

The Reverse implementation seems off to me -- it runs through the iterator twice, once collecting into a slice, and then a second time filling the same slice in reverse. (So basically the first Collect call is only being used to find the length of the iterated sequence.) I'm not sure about Go conventions†, but I imagine it would be considered better form to only run through the iterator once, reversing the collected slice in-place via a series of swaps.

(† Are iterators even expected/required to be reusable? If they are reusable, are they expected to be stable?)

> My issue with the go way of iterators is, you can’t chain them like you would in JavaScrip

Because it’s not JavaScript, and that is a good thing.

I’m trying to understand whether this is intended to make Go seem bad or whether it’s just coming across that way to me.

Shameless plug. I had experimented with Go iterators a while ago and did a https://github.com/gomoni/it

It was updated to 1.23, so it is as idiomatic as I can get. And yes it has a map method between two types. Just a single simple trick used.

I like how the author uses a test to run arbitrary code, this is exactly how I do it too!

We just released a go pkg that uses the new iter pkg. We were so excited by the interface in large part because of how simple iterators are to use.

https://github.com/picosh/pubsub/blob/main/pubsub.go#L18

We have seen in other languages like JS and python the power of iterators and we are happy to see it in Go

Since the article is making a not-so-subtle jab at python being unable to do chain operations, I'm making my annual rounds to point out that implementing simple, straightforward chain functionality in python is as simple as a two-line function definition:

  def chain( Accumulant, *Functions_list ):
      for f in Functions_list: Accumulant = f( Accumulant )
      return Accumulant

https://sr.ht/~tpapastylianou/chain-ops-python/

I think it would be more idiomatic to use statements, not expressions. That is, it’s ok to use local variables for intermediate values in a pipeline.

This reminds me of RxJS (https://rxjs.dev/)

1. That's a good looking Hugo theme!

2. Implicitly chain everything all the time!

In Factor, you might do it as:

    reverse [ sq ] [ even? ] map-filter [ . ] each

Or with a little less optimizing:

    reverse [ sq ] map [ even? ] filter [ . ] each

The least obvious thing is that the period is the pretty-print function.

It's funny the author throws a dig at Python for its syntax that actively discourages this kind of code. Like… my guy you're not taking the hint. Python makes things it doesn't want you to do ugly as sin. It's why lambda is so awkward and clunky.