xoutil.iterators
- Functions creating iterators for efficient looping¶
Several util functions for iterators
-
xoutil.iterators.
dict_update_new
(target, source)[source]¶ Update values in source that are new (not present) in target.
-
xoutil.iterators.
first_n
(iterable, n=1, fill=Unset)[source]¶ Takes the first n items from iterable.
If there are less than n items in the iterable and fill is
Unset
, a StopIteration exception is raised; otherwise it’s used as a filling pattern as explained below.Parameters: - iterable – An iterable from which the first n items should be collected.
- n (int) – The number of items to collect
- fill –
The filling pattern to use. It may be:
- a collection, in which case first_n fills the last items by cycling over fill.
- anything else is used as the filling pattern by repeating.
Returns: The first n items from iterable, probably with a filling pattern at the end.
Return type: generator object
New in version 1.2.0.
Changed in version 1.4.0: The notion of collection for the fill argument uses
xoutil.types.is_collection
instead of probing for the__iter__
method.
-
xoutil.iterators.
first_non_null
(iterable, default=None)[source]¶ Returns the first value from iterable which is non-null.
This is roughly the same as:
next((x for x in iter(iterable) if x), default)
New in version 1.4.0.
-
xoutil.iterators.
slides
(iterable, width=2, fill=None)[source]¶ Creates a sliding window of a given width over an iterable:
>>> list(slides(range(1, 11))) [(1, 2), (3, 4), (5, 6), (7, 8), (9, 10)]
If the iterator does not yield a width-aligned number of items, the last slice returned is filled with fill (by default None):
>>> list(slides(range(1, 11), width=3)) [(1, 2, 3), (4, 5, 6), (7, 8, 9), (10, None, None)]
Changed in version 1.4.0: If the fill argument is a collection is cycled over to get the filling, just like in
first_n()
.Changed in version 1.4.2: The fill argument now defaults to None, instead of Unset.
-
xoutil.iterators.
continuously_slides
(iterable, width=2, fill=None)[source]¶ Similar to
slides()
but moves one item at the time (i.e continuously).fill is only used to fill the fist chunk if the iterable has less items than the width of the window.
Example (generate a texts tri-grams):
>>> slider = continuously_slides(str('maupassant'), 3) >>> list(str('').join(chunk) for chunk in slider) ['mau', 'aup', 'upa', 'pas', 'ass', 'ssa', 'san', 'ant']
-
xoutil.iterators.
ungroup
(iterator)[source]¶ Reverses the operation of
itertools.groupby()
(or similar).The iterator should produce pairs of
(_, xs)
; wherexs
is another iterator (or iterable).It’s guaranteed that the iterator will be consumed at the boundaries of each pair, i.e. before taking another pair
(_, ys)
from iterator the firstxs
will be fully yielded.Demonstration:
>>> def groups(): ... def chunk(s): ... for x in range(s, s+3): ... print('Yielding x:', x) ... yield x ... ... for g in range(2): ... print('Yielding group', g) ... yield g, chunk(g)
>>> list(ungroup(groups())) Yielding group 0 Yielding x: 0 Yielding x: 1 Yielding x: 2 Yielding group 1 Yielding x: 1 Yielding x: 2 Yielding x: 3 [0, 1, 2, 1, 2, 3]
This is not the same as:
>>> import itertools >>> xs = itertools.chain(*(xs for _, xs in groups())) Yielding group 0 Yielding group 1
Notice that the iterator was fully consumed just to create the arguments to
chain()
.New in version 1.7.3.
-
xoutil.iterators.
delete_duplicates
(seq[, key=lambda x: x])[source]¶ Remove all duplicate elements from seq.
Two items
x
andy
are considered equal (duplicates) ifkey(x) == key(y)
. By default key is the identity function.Works with any sequence that supports
len()
,__getitem__()
andaddition
.Note
seq.__getitem__
should work properly with slices.The return type will be the same as that of the original sequence.
New in version 1.5.5.
Changed in version 1.7.4: Added the key argument. Clarified the documentation: seq should also implement the
__add__
method and that its__getitem__
method should deal with slices.
-
xoutil.iterators.
iter_delete_duplicates
(iter[, key=lambda x: x])[source]¶ Yields non-repeating items from iter.
key has the same meaning as in
delete_duplicates()
.Examples:
>>> list(iter_delete_duplicates('AAAaBBBA')) ['A', 'a', 'B', 'A']
>>> list(iter_delete_duplicates('AAAaBBBA', key=lambda x: x.lower())) ['A', 'B', 'A']
New in version 1.7.4.
-
xoutil.iterators.
fake_dict_iteritems
(source)[source]¶ Iterate (key, value) in a source that have defined method “keys” and
__getitem__()
.Warning
Deprecated since 1.7.0. This was actually in risk since 1.4.0.
-
xoutil.iterators.
flatten
(sequence, is_scalar=xoutil.types.is_scalar, depth=None)[source]¶ Flattens out a sequence. It takes care of everything deemed a collection (i.e, not a scalar according to the callabled passed in is_scalar):
>>> from xoutil.eight import range >>> range_ = lambda *a: list(range(*a)) >>> tuple(flatten((1, range_(2, 5), range(5, 10)))) (1, 2, 3, 4, 5, 6, 7, 8, 9)
If depth is None the collection is flattened recursiverly until the “bottom” is reached. If depth is an integer then the collection is flattened up to that level. depth=0 means not to flatten. Nested iterators are not “exploded” if under the stated depth:
# In the following doctest we use ``...range(...X)`` because the string # repr of range differs in Py2 and Py3k. >>> tuple(flatten((range_(2), range(2, 4)), depth=0)) # doctest: +ELLIPSIS ([0, 1], ...range(2, 4)) >>> tuple(flatten((range(2), range_(2, 4)), depth=0)) # doctest: +ELLIPSIS (...range(...2), [2, 3])
-
xoutil.iterators.
zip
([iter1[, iter2[, ...]]])¶ Return a zip-like object whose next() method returns a tuple where the i-th element comes from the i-th iterable argument. The next() method continues until the shortest iterable in the argument sequence is exhausted and then it raises StopIteration.
This method is actually the standard
itertools.izip()
when in Python 2.7, and the builtinzip
when in Python 3.
-
xoutil.iterators.
map
(func, *iterables)¶ Make an iterator that computes the function using arguments from each of the iterables. It stops when the shortest iterable is exhausted instead of filling in None for shorter iterables.
This method is actually the stardard
itertools.imap
when in Python 2.7, and the builtinmap
when in Python 3.
-
xoutil.iterators.
zip_longest
(*iterables, fillvalue=None)¶ Make an iterator that aggregates elements from each of the iterables. If the iterables are of uneven length, missing values are filled-in with fillvalue. Iteration continues until the longest iterable is exhausted.
If one of the iterables is potentially infinite, then the
zip_longest()
function should be wrapped with something that limits the number of calls (for exampleislice()
ortakewhile()
). If not specified, fillvalue defaults to None.This function is actually an alias to
itertools.izip_longest()
in Python 2.7, and an alias toitertools.zip_longest()
in Python 3.3.