Groups


192 of 99+  



julia-users ›
Should Iterators be nonmutating?
4 posts by 2 authors  



Lyndon White 	

Apr 22


I am writing some iterators that work with `IO`.

I thought, so that my iterators would not mutate,
I would `deepcopy` the IO item, and store it as part of my state.
(then only my state would mutate, and no-one cares probably (unless they were trying to `tee` for free))

But `deepcopying` IO items apparently does not actually cause them to become separate.
At least not for IOStreams.



io=open("./data/text8/text8","r")
ia = deepcopy(io)
>IOStream(<file ./data/text8/text8>)


position(io)
>0

position(seekend(io))
>100000000

position(ia)
>100000000



I was under the assumption that iterators never mutated, as there is no `!` in `next`.
(then again there is also no `!` in `seekend`)

So I was a bit stuck for how to implement.

So I thought I would see how `Eachline` is implemented.
It is mutating.
So maybe my assumptions are wrong, and iterators can be mutating?
They would be a lot easier to write that way, I guess.

But julia does not have a `tee` function



 


Dan 	

Apr 22


Looks like it might be appropriate to make the `stream` in `Eachline` be the state of the iterator (instead of the `nada` returned by `next`).

Opening an issue (perhaps with a PR of a fix) could be the next step. Unless someone finds some other logic for keeping `Eachline` the way it is.
- show quoted text -
 


Lyndon White 	

Apr 22


I'm not sure changing this behavior is possible,
at least not without making IOStream clonable.
And I suspect there are filesystem constraints against this.

Well there is a scary way, when the state just holds the current file position,
then seeks to it when next is called,
reads a line,
then seeks back to where it was before.
So it looks like we are not mutating the underlying object.
(cos we undo our changes)
This would not be threadsafe though.
- show quoted text -
 


Dan 	

Apr 23


The OS resource cost of creating a new stream for each Eachline does not seem like a problem, since the stream is closed at the end of the iteration and massive parallel Eachlines on the same file are the only blocking usecase. Additionally, since the `done` function for Eachline closes the stream, it seems weird not to create a new one on `start`.

So, IOStream should be clonable in the sense you suggest (not totally sure I got what you meant).

Furthermore, the semantics of an IOStream is to allow stream access to an underlying resource (a file for example). This resource should have a "name", and IOStream should allow you to get the name. With this name, you could try to open a new IOStream to the same resource (possibly with different permissions).

Currently, a file IOStream has a `name` field with the filename in some format. But perhaps a `getURI` (or `getURL` or `name`) method should return a representation of the underlying resource of IOStream which could be used to create another IOStream for that resource.
- show quoted text -